Fine Automatic Spraying Method for Lure Baits Based on Machine Vision

Abstract

The present invention relates to a refined automatic spraying method for lure baits based on machine vision, which solves the technical problems of high labor intensity, high labor cost and low efficiency in the painting process of existing artificial spraying methods for fake baits. It identifies the bait through a vision algorithm, calculates and generates a spraying trajectory, optimizes the spraying trajectory using deviation information, and controls the end action of the robotic arm to perform spraying. The present invention is widely used in the painting and coloring process of lure baits.

Description

Technical Field

[0001] The present invention relates to the technical field of manufacturing artificial fishing baits, and more particularly, to a refined automatic spraying method for lure fishing baits based on machine vision. Background Art

[0002] During the process of fishing with a lure fishing rod, artificial baits are used to attract fish to bite. The shapes of artificial baits are similar to small fish, shrimps or insects. In the production process of artificial baits, after being produced with plastics, metals or silica gels as raw materials and shaped, the artificial baits need to be painted and colored.

[0003] Currently, the main operation method is manual spraying. An operator holds a spray gun to paint and color the artificial baits hanging on the assembly line, spraying the front side first and then the back side. This method has technical problems such as high labor intensity, high labor cost and low efficiency. Summary of the Invention

[0004] The present invention is to solve the technical problems of high labor intensity, high labor cost and low efficiency in the existing production process of artificial fishing baits by using manual spraying. A refined automatic spraying method for lure fishing baits based on machine vision is provided to replace manual spraying and improve operation efficiency.

[0005] The refined automatic spraying method for lure fishing baits based on machine vision provided by the present invention includes a refined spraying robot for lure fishing baits. The refined spraying robot for lure fishing baits includes a robotic arm connecting bracket, a robotic arm, an execution end, a workbench and a bait positioning device. The robotic arm is connected to the robotic arm connecting bracket, the execution end is connected to the end of the robotic arm, and the bait positioning device is connected to the middle of the workbench. The refined automatic spraying method for lure fishing baits includes the following steps:

[0006] The first step is to identify the bait through a visual recognition algorithm and calculate deviation information;

[0007] Step 1: Collect a color image and a depth image of a primerless bait connected to the bait positioning device through a binocular camera, and transmit the color image and the depth image to the controller;

[0008] Step 2: The controller extracts the outer contour of the bait from the color image to facilitate the polygon approximation algorithm to scan feature points;

[0009] Step 3: Scan the outer contour of the bait with the polygon approximation algorithm to obtain feature points;

[0010] Step 4: Align the depth image with the color image, read the depth corresponding to the two-dimensional pixel coordinates of the feature points obtained in the previous step, and thus obtain the three-dimensional coordinates of the feature points;

[0011] Step 5: First, select three feature points A, B, and C from the detected feature points, find the corresponding points in the three-dimensional model of the fishing bait, and obtain through the formula

[0012] to obtain Establish a coordinate system for the model feature points, and calculate the transformation matrix between the coordinate system of the model feature points and the origin coordinate system of the model o T f ;

[0013] Step 6: Based on the detected feature points, establish a coordinate system for the feature points in the camera coordinate system Perform a cross product operation on the coordinate system of the feature points and the transformation matrix o T f to obtain the attitude information T of the center point of the fishing bait in the camera coordinate system

[0014] ; cam-o ;

[0015] Second step: Determine the spraying trajectory;

[0016] Step 1: Convert the three-dimensional model of the fishing bait drawn using SolidWorks software that has been prepared in advance into the stl format, import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the fishing bait, and display the information of the model point cloud of the fishing bait;

[0017] Step 2: Obtain the normal vectors of all points in the information of the model point cloud of the fishing bait for reading pose information;

[0018] Step 3: Perform a halving process on the point cloud information. Cut along the central axis of the model point cloud of the fishing bait to only show the upper half, which is convenient for reading the information of the fish back of the fishing bait;

[0019] Step 4: Perform a slicing process on the upper half of the point cloud information formed in Step 3. Cut from top to bottom on both sides of the central axis in the directions of Section 1 and Section 2. The distance between Section 1 and Section 2 is δ, where d < δ < 4 * d << w. Here, d = max(di), and d i represents the distance between the i-th point of the point cloud and the other point closest to it, and w represents the width of the upper half of the point cloud. After the slicing process, the fish back curve is presented;

[0020] Step 5: Equally spaced sample points on the fish back curve;

[0021] Step 6: Translate the points obtained in Step 5 along their normal vectors by a preset height to obtain the spraying position points. The preset height is denoted as the spraying height, which is the distance between the nozzle of the pneumatic paint spray gun and the fishing bait; A number of spraying position points form a curve, and this curve is the spraying trajectory; The pose information of the spraying position points is T p ;

[0022] Step 7: Optimize the deviation of the spraying position points, and calculate the pose matrix P of the spraying position points in the camera coordinate system c , p C = T cam-o × T p ;

[0023] The third step: The manipulator moves the execution end to the working position;

[0024] The hand-eye calibration obtains the calibration matrix Tr; Multiply the optimized pose matrix P c by Tr crosswise, and the spraying position corresponding to the execution end can be obtained; The manipulator drives the execution end to move to the working position;

[0025] The fourth step: The controller controls the drive motor in the execution end to work, starts the pneumatic paint spray gun, and the pneumatic paint spray gun sprays the fishing bait.

[0026] The present invention also provides a fine automatic spraying method for lure fishing bait based on machine vision, including a fine spraying robot for lure fishing bait. The fine spraying robot for lure fishing bait includes a manipulator connection bracket, a manipulator, an execution end, a workbench and a fishing bait positioning device. The manipulator is connected to the manipulator connection bracket, the execution end is connected to the end of the manipulator, and the fishing bait positioning device is connected to the middle of the workbench; The fine automatic spraying method for lure fishing bait includes the following steps:

[0027] The first step: Identify the fishing bait through a visual recognition algorithm and calculate the deviation information;

[0028] Step 1: Collect the color image and depth image of the fishing bait with primer connected to the fishing bait positioning device through a binocular camera, and transmit the color image and depth image to the controller;

[0029] Step 2: Convert the depth image into a point cloud map to obtain the point cloud map of the fishing bait with primer;

[0030] Step 3: Combine the point cloud map with the RGB map;

[0031] Step 4: Convert the three-dimensional model of the fishing bait drawn by SolidWorks software prepared in advance into the stl format, and import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the fishing bait. The model point cloud of the fishing bait is used as the model point cloud of the fishing bait with primer;

[0032] Step 5: Perform point cloud registration on the point cloud map of the fishing bait with primer and the model point cloud through the ICP algorithm, and then obtain the deviation information from the registration result;

[0033] The second step: Determine the spraying trajectory;

[0034] Step 1: Convert the 3D model of the fishing bait drawn by SolidWorks software into the stl format. Import the 3D model in stl format into the open3D library to obtain the model point cloud of the fishing bait, and display the information of the fishing bait model point cloud;

[0035] Step 2: Obtain the normal vectors of all points in the fishing bait model point cloud information for reading pose information;

[0036] Step 3: Cut the point cloud information in half. Cut along the central axis of the fishing bait model point cloud and only show the upper half to facilitate reading the information of the fish back of the fishing bait;

[0037] Step 4: Slice the upper half of the point cloud information formed in Step 3. Cut from top to bottom on both sides of the central axis in the directions of section one and section two. The distance between section one and section two is δ, where d < δ < 4*d << w. Here, d = max(di), and d i represents the distance between the i-th point of the point cloud and the other point closest to itself, and w represents the width of the upper half of the point cloud. After slicing, the fish back curve is presented;

[0038] Step 5: Take equally spaced points on the fish back curve;

[0039] Step 6: Translate the points obtained in Step 5 along their normal vectors by a preset height to obtain the spraying position points. The preset height is denoted as the spraying height, which is the distance between the nozzle of the pneumatic paint spray gun and the fishing bait; A number of spraying position points form a curve, and this curve is the spraying trajectory; The pose information of the spraying position points is T p ;

[0040] Step 7: Optimize the deviation of the spraying position points. The cross product of the deviation information and the pose information is T p ;

[0041] Third step: The robotic arm moves the end effector to the working position;

[0042] Perform hand-eye calibration to obtain the calibration matrix Tr; Cross multiply the optimized pose matrix P c by Tr, that is, obtain the spraying position corresponding to the end effector; The robotic arm drives the end effector to move to the working position;

[0043] Fourth step: The controller controls the drive motor in the end effector to work, starts the pneumatic paint spray gun, and the pneumatic paint spray gun sprays the fishing bait.

[0044] The beneficial effects of the present invention are as follows: It replaces manual painting operations, saves labor, greatly reduces labor costs, has a low labor intensity, and greatly improves the painting operation efficiency. The spraying effect is better. Precise spraying is achieved.

[0045] Further features and aspects of the present invention will be clearly described in the following detailed description with reference to the accompanying drawings. Description of the Drawings

[0046] Figure 1 is an axonometric view of the fine spraying robot system for fishing lures;

[0047] Figure 2 is the front view of the fine spraying robot system for fishing lures;

[0048] Figure 3 is the side view of the fine spraying robot system for fishing lures;

[0049] Figure 4 is the rear view of the fine spraying robot system for fishing lures;

[0050] Figure 5 is Figure 3 a cross-sectional view taken along the A-A direction in

[0051] Figure 6 is Figure 1 an axonometric view of the fishing lure positioning device without the outer shell in the fine spraying robot system for fishing lures shown;

[0052] Figure 7 is an axonometric view of the end effector;

[0053] Figure 8 is the front view of the end effector;

[0054] Figure 9 is the top view of the end effector;

[0055] Figure 10 is the left view of the end effector;

[0056] Figure 11 is an axonometric view of the first spray gun automatic opening mechanism;

[0057] Figure 12 is the front view of the first spray gun automatic opening mechanism;

[0058] Figure 13 is the top view of the first spray gun automatic opening mechanism;

[0059] Figure 14 is the bottom view of the first spray gun automatic opening mechanism;

[0060] Figure 15 is the left view of the first spray gun automatic opening mechanism;

[0061] Figure 16 is the right view of the first spray gun automatic opening mechanism;

[0062] Figure 17 is Figure 1 a partial enlarged view of the structure shown;

[0063] Figure 18 is Figure 6 a partial enlarged view of the structure shown;

[0064] Figure 19 is an isometric view of the bait positioning device after removing the outer shell;

[0065] Figure 20 is Figure 19 the front view of the structure shown;

[0066] Figure 21 is Figure 19 the right view of the structure shown;

[0067] Figure 22 is Figure 20 the sectional view in the direction of B - B in;

[0068] Figure 23 is Figure 22 the sectional view in the direction of C - C in;

[0069] Figure 24 is Figure 22 the sectional view in the direction of D - D in;

[0070] Figure 25 is the structural schematic diagram of the bait positioning device;

[0071] Figure 26 is the structural schematic diagram of the bait to be processed installed on the refined spraying robot system for fishing lures;

[0072] Figure 27 is Figure 26 a partial enlarged view of the structure shown;

[0073] Figure 28 is the principle block diagram of the control system;

[0074] Figure 29 is the architecture diagram of the automatic spraying method;

[0075] Figure 30 is the overall flowchart of the refined automatic spraying method for fishing lures based on machine vision;

[0076] Figure 31 is the flowchart of deviation calculation for baits without primer;

[0077] Figure 32 is the schematic diagram of xyz coordinate system establishment;

[0078] Figure 33 is the flowchart of point cloud registration;

[0079] Figure 34 It is a flow chart for generating a spraying trajectory from point cloud slices;

[0080] Figure 35 It is a structural schematic diagram of a hand-eye calibration board;

[0081] Figure 36 It is a schematic diagram of the slice processing process, where Figure (a) is the point cloud of the fishing bait model, Figure (b) is the result of cutting Figure (a) to only retain the upper half, and Figure (c) is the top view of Figure (b);

[0082] Figure 37 It is a schematic diagram of equidistant point sampling;

[0083] Figure 38 It is a schematic diagram of extracting the outer contour of the fishing bait from a color image;

[0084] Figure 39 It is a diagram of characteristic points scanned from the outer contour of the fishing bait using a polygon approximation algorithm.

[0085] Explanation of symbols in the figure:

[0086] 100. Robotic arm connection bracket, 200. Robotic arm, 300. Execution end, 301. Fixing bracket, 302. First pneumatic paint spray gun, 302-1. Trigger, 302-2. Nozzle, 302-3. Paint storage pot, 303. Second pneumatic paint spray gun, 303-1. Trigger, 304. First spray gun automatic opening mechanism, 304-1. First fixing block, 304-1-1. Connection hole, 304-1-2. First blocking inclined plane, 304-2. First support frame, 304-3. First telescopic frame, 304-3-1. First pressing rod, 304-4. First nut seat, 304-5. First lead screw, 304-6. First driving motor, 305. Second spray gun automatic opening mechanism, 306. Support rod, 307. Binocular camera, 308. Second U-groove photoelectric switch, 309. First U-groove photoelectric switch, 310. Second trigger plate, 311. First trigger plate; 400. Workbench, 401. First paint drainage and collection channel, 402. Second paint drainage and collection channel; 500. Bait positioning device, 501. Shell, 502. Base, 503. First bait connecting rod, 504. First driving gear, 505. Bearing, 506. First left bait connecting rod, 507. Second left bait connecting rod, 508. Third left bait connecting rod, 509. First left driven gear, 510. Second left driven gear, 511. Third left driven gear, 512. First left intermediate transition gear, 513. Second left intermediate transition gear, 514. Third left intermediate transition gear, 515. First right bait connecting rod, 516. Second right bait connecting rod, 517. Third right bait connecting rod, 518. First right driven gear, 519. Second right driven gear, 520. Third right driven gear, 521. First right intermediate transition gear, 522. Second right intermediate transition gear, 523. Third right intermediate transition gear, 524. Driven synchronous pulley, 525. Driving synchronous pulley, 526. Synchronous belt, 527. Flip driving motor. 600. Bait. 5. Controller, 6. Power supply module, 12. Electro-hydraulic proportional valve, 13. Analog output card, 14. First electromagnetic reversing valve, 17. Three-way joint, 18. Second electromagnetic reversing valve. Detailed implementation mode

[0087] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0088] As Figures 1-6 shown, the refined spraying robot system for lure bait includes a robotic arm connection bracket 100, a robotic arm 200, an execution end 300, a workbench 400, and a bait positioning device 500. The robotic arm 200 is installed on the robotic arm connection bracket 100, the execution end 300 is installed at the end of the robotic arm 200, and the bait positioning device 500 is installed in the middle of the workbench 400.

[0089] like Figures 7-10 As shown, the execution terminal 300 includes a fixed frame 301, a first pneumatic paint spray gun 302, a second pneumatic paint spray gun 303, a first spray gun automatic opening mechanism 304, and a second spray gun automatic opening mechanism 305. The first spray gun automatic opening mechanism 304 and the second spray gun automatic opening mechanism 305 are respectively fixedly installed on the fixed frame 301, the first pneumatic paint spray gun 302 is connected and positioned on the first spray gun automatic opening mechanism 304, and the second pneumatic paint spray gun 303 is connected and positioned on the second spray gun automatic opening mechanism 305.

[0090] like Figures 11-16 As shown, the first spray gun automatic opening mechanism 304 includes a first fixed block 304-1, a first support frame 304-2, a first telescopic frame 304-3, a first nut seat 304-4, a first screw rod 304-5, and a first drive motor 304-6. The first support frame 304-2 is fixedly connected to the first fixed block 304-1, the first drive motor 304-6 is fixedly installed on the first support frame 304-2, the first screw rod 304-5 is fixedly connected to the output shaft of the first drive motor 304-6, the first screw rod 304-5 is located in the space of the first support frame 304-2, and the first nut seat 304-4 is fixedly connected to the first fixed block 304-1. The first drive motor 304-6 is fixedly installed on the first support frame 304-2. The first screw rod 304-5 is fixedly connected to the output shaft of the first drive motor 304-6. The first screw rod 304-5 is located in the space of the first support frame 304-2. The internal threaded hole of the first screw rod 304-5 is connected and matched with the external thread of the first screw rod 304-5, the first nut seat 304-4 is located in the space of the first support frame 304-2, the rear end of the first telescopic frame 304-3 is fixedly connected with the two sides of the first nut seat 304-4, the front end of the first telescopic frame 304-3 is provided with a first pressure rod 304-3-1, the first telescopic frame 304-3 is located on both sides of the first fixed block 304-1, and the first pressure rod 304-3-1 is located in front of the first fixed block 304-1; the first fixed block 304-1 is provided with a connecting hole 304-1-1, and the connecting hole 304-1-1 is used to install a pneumatic paint spray gun. The front part of the first fixed block 304-1 is provided with a first blocking inclined surface 304-1-2. When the first drive motor 304-6 works, it drives the first screw rod 304-5 to rotate, and the first screw rod 304-5 drives the first nut seat 304-4 to translate, and the first nut seat 304-4 drives the first telescopic frame 304-3 to translate backward or forward, and the first pressure rod 304-3-1 on the first telescopic frame 304-3 translates backward or forward accordingly, and the first pressure rod 304-3-1 moves backward and gradually approaches the first blocking slope 304-1-2, and the first pressure rod 304-3-1 moves forward and gradually moves away from the first blocking slope 304-1-2.

[0091] The bottom of the first support frame 304-2 is provided with screw holes, and the first support frame 304-2 is fixedly installed on the fixing frame 301 through screws. The bottom of the first fixing block 304-1 is provided with screw holes, and the first fixing block 304-1 is fixedly installed on the fixing frame 301 through screws. It should be noted that it is also feasible to only fixedly install the first fixing block 304-1 on the fixing frame 301 without fixedly installing the first support frame 304-2 on the fixing frame 301.

[0092] The structure of the second paint gun automatic opening mechanism 305 is the same as that of the first paint gun automatic opening mechanism 304. The second paint gun automatic opening mechanism 305 includes a second fixing block, a second support frame, a second telescopic frame, a second nut seat, a second lead screw, and a second driving motor. The second support frame is fixedly connected to the second fixing block. The second driving motor is fixedly installed on the second support frame. The second lead screw is fixedly connected to the output shaft of the second driving motor. The second lead screw is located in the space of the second support frame. The internal threaded hole of the second nut seat is in threaded connection and cooperation with the external thread of the second lead screw. The second nut seat is located in the space of the second support frame. The rear end of the second telescopic frame is fixedly connected to both sides of the second nut seat. The front end of the second telescopic frame is provided with a second pressure rod. The second telescopic frame is located on both sides of the second fixing block. The second pressure rod is located in front of the second fixing block. The second fixing block is provided with a connection hole for installing a pneumatic paint gun. The front part of the second fixing block is provided with a second blocking inclined surface.

[0093] The second support frame is fixedly installed on the fixing frame 301 through screws. The second fixing block is fixedly installed on the fixing frame 301 through screws.

[0094] The pneumatic paint gun is a conventional product that can be directly purchased and used in the market. Usually, the pneumatic paint gun is manually operated. As Figures 7-10 shown, when installing two pneumatic paint guns, insert the handle of the first pneumatic paint gun 302 into the connection hole 304-1-1 of the first fixing block 304-1 for connection and positioning (the removal process is also convenient). The trigger 302-1 of the first pneumatic paint gun 302 is located in the space between the first pressure rod 304-3-1 and the first fixing block 304-1, that is to say, the trigger 302-1 is located behind the first pressure rod 304-3-1 and in front of the first blocking inclined surface 304-1-2. Similarly, insert the handle of the second pneumatic paint gun 303 into the connection hole of the second fixing block. The trigger 303-1 of the second pneumatic paint gun 303 is located in the space between the second pressure rod and the second fixing block, that is to say, the trigger 303-1 is located behind the second pressure rod and in front of the second blocking inclined surface.

[0095] It can be seen from the figure that the two pneumatic paint guns are arranged side by side.

[0096] When the first pneumatic paint spray gun 302 is in operation, connect the air interface of the first pneumatic paint spray gun 302 to an external air pipe for outputting compressed air. Start the first drive motor 304-6 to move the first pressure lever 304-3-1 backward. The first pressure lever 304-3-1 then presses the trigger 302-1, and thus the nozzle 302-2 of the first pneumatic paint spray gun 302 sprays atomized paint. When the first pressure lever 304-3-1 moves forward, the pressure applied to the trigger 302-1 disappears, and the trigger 302-1 will return to its initial position, stopping the operation, and the nozzle 302-2 stops spraying atomized paint. The working process of the second pneumatic paint spray gun is the same and will not be elaborated here.

[0097] Fix the fixing bracket 301 at the end of the robotic arm 200. Since the execution end 300 is connected to the end of the robotic arm 200, the robotic arm 200 can place the execution end 300 at any position in space suitable for painting the fishing bait.

[0098] As Figures 17-25As shown, the bait positioning device 500 includes a housing 501, a base 502, a first bait connecting rod 503, a first driving gear 504, a bearing 505, a first left bait connecting rod 506, a second left bait connecting rod 507, a third left bait connecting rod 508, a first left driven gear 509, a second left driven gear 510, a third left driven gear 511, a first left intermediate idler gear 512, a second left intermediate idler gear 513, a third left intermediate idler gear 514, a first right bait connecting rod 515, a second right bait connecting rod 516, a third right bait connecting rod 517, a first right driven gear 518, a second right driven gear 519, a third right driven gear 520, a first right intermediate idler gear 521, a second right intermediate idler gear 522, a third right intermediate idler gear 523, a driven synchronous pulley 524, a driving synchronous pulley 525, a synchronous belt 526, and a flipping drive motor 527. The first bait connecting rod 503 passes through the base 502, and the first bait connecting rod 503 is rotatably connected to the base 502 through two bearings 505. The first driving gear 504 is fixedly connected to the middle of the first bait connecting rod 503. The first left bait connecting rod 506 passes through the base 502, and the first left bait connecting rod 506 is rotatably connected to the base 502 through a bearing. The second left bait connecting rod 507 passes through the base 502 and is rotatably connected to the base 502 through a bearing. The third left bait connecting rod 508 passes through the base 502 and is rotatably connected to the base 502 through a bearing. The first left driven gear 509 is fixedly connected to the middle of the first left bait connecting rod 506. The second left driven gear 510 is fixedly connected to the middle of the second left bait connecting rod 507. The third left driven gear 511 is fixedly connected to the middle of the third left bait connecting rod 508. The first left intermediate idler gear 512 is rotatably connected to the base 502 through a bearing. The first left intermediate idler gear 512 meshes with the first driving gear 504 and the first left driven gear 509 at the same time, and the first left intermediate idler gear 512 is disposed between the first driving gear 504 and the first left driven gear 509. The second left intermediate idler gear 513 is rotatably connected to the base 502 through a bearing. The second left intermediate idler gear 513 meshes with the first left driven gear 509 and the second left driven gear 510 at the same time, and the second left intermediate idler gear 513 is disposed between the second left driven gear 510 and the first left driven gear 509. The third left intermediate idler gear 514 is rotatably connected to the base 502 through a bearing. The third left intermediate idler gear 514 meshes with the second left driven gear 510 and the third left driven gear 511 at the same time;The first right fishing bait connecting rod 515 passes through the base 502 and is rotatably connected to the base 502 through a bearing. The second right fishing bait connecting rod 516 passes through the base 502 and is rotatably connected to the base 502 through a bearing. The third right fishing bait connecting rod 517 passes through the base 502 and is rotatably connected to the base 502 through a bearing. The first right driven gear 518 is fixedly connected to the middle part of the first right fishing bait connecting rod 515. The second right driven gear 519 is fixedly connected to the middle part of the second right fishing bait connecting rod 516. The third right driven gear 520 is fixedly connected to the middle part of the third right fishing bait connecting rod 517. The first right intermediate idler gear 521 is rotatably connected to the base 502 through a bearing. While meshing with the first driving gear 504, the first right intermediate idler gear 521 also meshes with the first right driven gear 518. The first right intermediate idler gear 521 is arranged between the first driving gear 504 and the first right driven gear 518. The second right intermediate idler gear 522 is rotatably connected to the base. While meshing with the first right driven gear 518, the second right intermediate idler gear 522 also meshes with the second right driven gear 519. The third right intermediate idler gear 523 is rotatably connected to the base. While meshing with the second right driven gear 519, the third right intermediate idler gear 523 also meshes with the third right driven gear 520. The third right intermediate idler gear 523 is arranged between the second right driven gear 519 and the third right driven gear 520. The driven synchronous pulley 524 is fixedly connected to the first fishing bait connecting rod 503. The driving synchronous pulley 525 is connected to the output shaft of the flipping drive motor 527. The synchronous belt 526 is connected between the driving synchronous pulley 525 and the driven synchronous pulley 524. The housing 501 covers the base 502 and is fixedly connected by screws. The housing 501 functions to prevent dust and external substances from entering. When the flipping drive motor 527 is started, the first fishing bait connecting rod 503 is driven to rotate 180° through the synchronous belt transmission mechanism. Driven by the gear set, the first left fishing bait connecting rod 506, the second left fishing bait connecting rod 507, the third left fishing bait connecting rod 508, the first right fishing bait connecting rod 515, the second right fishing bait connecting rod 516, and the third right fishing bait connecting rod 517 also rotate 180° in the same direction.;

[0099] As Figure 18 shown, the base 502 is fixedly connected to the middle of the workbench 400. The flipping drive motor 527 is fixedly installed on the workbench 400. Each end of the fishing bait connecting rod is provided with a notch for inserting the fishing bait.

[0100] As Figure 1 、 17 、18 shown, the workbench 400 is provided with a first paint drainage and collection channel 401 and a second paint drainage and collection channel 402. The first paint drainage and collection channel 401 and the second paint drainage and collection channel 402 are respectively located on both sides of the fishing bait positioning device 500.

[0101] AsFigure 28 As shown in the figure, the control system includes a controller 5, a power supply module 6, an electro-hydraulic proportional valve 12, an analog output card 13, a tee 17, a first electromagnetic directional valve 14, and a second electromagnetic directional valve 18. The power supply module 6 is used to supply power to the power-consuming units such as the controller 5, the electro-hydraulic proportional valve 12, and the analog output card 13. The inlet of the tee 17 is connected to the outlet of the electro-hydraulic proportional valve 12. The inlet of the first electromagnetic directional valve 14 is connected to the first outlet of the tee 17 through a pipeline. The inlet of the second electromagnetic directional valve 18 is connected to the second outlet of the tee 17 through a pipeline. The compressed air interface of the first pneumatic paint spray gun 302 is connected to the outlet of the first electromagnetic directional valve 14 through a pipeline. The compressed air interface of the second pneumatic paint spray gun 303 is connected to the outlet of the second electromagnetic directional valve 18 through a pipeline. The second U-groove photoelectric switch 308 and the first U-groove photoelectric switch 309 are respectively connected to the controller 5. The first drive motor 304-6 is electrically connected to the controller 5. The second drive motor is electrically connected to the controller 5. The flipping drive motor 527 is electrically connected to the controller 5. The first drive motor 304-6 and the second drive motor preferably adopt closed-loop stepper motors with higher precision, which is beneficial to accurately control the opening and closing degree of the pneumatic paint spray gun and ensure the accuracy and stability of spraying. The electro-hydraulic proportional valve 12 is used to control the size of the air pressure supplied to the pneumatic paint spray gun and achieve precise adjustment of the paint spraying amount. Both the first electromagnetic directional valve 14 and the second electromagnetic directional valve 18 adopt two-position five-way electromagnetic directional valves. The first two-position five-way electromagnetic directional valve is provided with a P port, an A port, and a B port. The P port is connected to the first outlet of the tee 17 through a pipeline. The B port is connected to the compressed air interface of the first pneumatic paint spray gun 302 through a pipeline. When the coil of the two-position five-way electromagnetic directional valve is energized, the P port and the B port are conducted, and compressed air is supplied from the B port to the first pneumatic paint spray gun 302. When the coil is de-energized, the P port and the B port are cut off. The second two-position five-way electromagnetic directional valve is provided with a P port, an A port, and a B port. The P port is connected to the second outlet of the tee 17 through a pipeline. The B port is connected to the compressed air interface of the second pneumatic paint spray gun 303 through a pipeline. When the coil of the second two-position five-way electromagnetic directional valve is energized, the P port and the B port are conducted, and compressed air is supplied from the B port to the second pneumatic paint spray gun 303. When the coil is de-energized, the P port and the B port are cut off. During the spraying process, the analog output card 13 and the electro-hydraulic proportional valve 12 are used to accurately control the adjustment of the air pressure, ensure the accuracy and stability of the spraying process, realize the accurate automatic control of the spray flow rate and speed, and accurately control the air pressure and the opening and closing degree of the spray gun through the drive motor to improve the spraying quality.

[0102] The working process of the above-mentioned refined spraying robot system for fishing lures is introduced as follows:

[0103] Refer to Figure 26 、 27, manually insert the bait 600 to be painted into the slot at the end of the bait connecting rod of the bait positioning device 500. Baits are connected to both ends of each bait connecting rod, thus forming two rows of baits. One row of 7 baits is located above the opening of the first paint drainage and collection channel 401, and another row of 7 baits is located above the second paint drainage and collection channel 402.

[0104] Next, connect the compressed air delivery pipes to the first pneumatic paint spray gun 302 and the second pneumatic paint spray gun 303. Control the movement of the robotic arm 200 so that the execution end 300 is located above the bait positioning device 500. The nozzle of the first pneumatic paint spray gun 302 is located above one row of baits, and the nozzle of the second pneumatic paint spray gun 303 is located above the other row of baits.

[0105] Next, start the first drive motor 304-6 and the second drive motor, so that the first pressure rod 304-3-1 presses the trigger 302-1 of the first pneumatic paint spray gun 302, thereby causing the nozzle 302-2 of the first pneumatic paint spray gun 302 to spray atomized paint. At the same time, the nozzle of the second pneumatic paint spray gun also sprays atomized paint. The first pneumatic paint spray gun 302 performs a painting operation on one row of baits above the opening of the first paint drainage and collection channel 401, and the second pneumatic paint spray gun performs a painting operation on one row of baits above the opening of the second paint drainage and collection channel 402. During the painting process, the robotic arm drives the execution end 300 to move integrally along the length direction of one row of baits to ensure that the front of each bait can be sprayed with paint.

[0106] Next, start the flipping drive motor 527 of the bait positioning device 500 to flip each bait so that the back of the bait faces up, and then the robotic arm drives the two pneumatic paint spray guns to perform a painting operation on the back of the baits.

[0107] Finally, stop the machine, and manually pull out each bait from the end of the bait connecting rod.

[0108] During the painting process, the paint will flow downward along the first paint drainage and collection channel 401 and the second paint drainage and collection channel 402 and can be collected.

[0109] It can be seen that the above automatic painting process significantly reduces labor costs, saves time and effort; greatly improves the painting operation efficiency; has a better spraying effect; the baits to be processed are fixed, and the robotic arm realizes the stable and accurate positioning of the pneumatic paint spray gun, and finally enables the pneumatic paint spray gun to move stably, avoiding the shaking and instability during the spraying process of manually holding the spray gun and moving it, and realizing precise spraying.

[0110] During the operation of the spray gun at the execution end 300, for higher safety and reliability, a first blocking inclined surface 304-1-2 is provided at the front of the first fixing block 304-1 to prevent the trigger 302-1 from being pushed backward excessively and causing fracture or damage. The extreme position where the trigger 302-1 moves backward is to contact the first blocking inclined surface 304-1-2.

[0111] During the process of the robotic arm driving the execution end to move and position above the fishing bait, for more accurate positioning, as Figure 7 , 8 , as shown in 10, a binocular camera is provided at the execution end. The support rod 306 is fixedly connected to the fixing frame 301, and the binocular camera 307 is fixed on the support rod 306. Images are collected by the binocular camera 307 and sent to the controller. The controller identifies the fishing bait through machine vision technology and controls the movement of the robotic arm to position the execution end above the fishing bait.

[0112] It should be noted that those skilled in the art can understand that the two spray guns can work independently and do not have to operate simultaneously.

[0113] To further optimize the structure and function of the execution end 300, referring to Figures 7-10 , a second U-groove photoelectric switch 308, a first U-groove photoelectric switch 309, a second trigger plate 310, and a first trigger plate 311 are provided. The second U-groove photoelectric switch 308 is fixedly installed on the second fixing block, and the second trigger plate 310 is fixedly installed on the second telescopic frame. When the second telescopic frame moves forward, the second trigger plate 310 moves forward accordingly. When the second trigger plate 310 enters the U-shaped groove of the second U-groove photoelectric switch 308, the second U-groove photoelectric switch 308 generates a signal to the controller, and the controller instructs the second driving motor to stop working, and the second telescopic frame stops moving, ensuring that the second telescopic frame accurately moves forward to the predetermined position and then stops, ensuring the accuracy of the forward and backward movement strokes of the second telescopic frame. Similarly, the first U-groove photoelectric switch 309 is fixedly installed on the first fixing block 304-1, and the first trigger plate 311 is fixedly installed on the first telescopic frame 304-3. When the first telescopic frame 304-3 moves forward, the first trigger plate 311 moves forward accordingly. When the first trigger plate 311 enters the U-shaped groove of the first U-groove photoelectric switch 309, the first U-groove photoelectric switch 309 generates a signal to the controller, and the controller instructs the first driving motor to stop working, and the first telescopic frame 304-3 stops moving, ensuring that the first telescopic frame 304-3 accurately moves forward to the predetermined position and then stops, ensuring the accuracy of the forward and backward movement strokes of the first telescopic frame 304-3. Prevent the first telescopic frame and the second telescopic frame (that is, prevent the execution end 300) from hitting the workbench 400 and the fishing bait positioning device 500.

[0114] The first drive motor 304-6 and the second drive motor preferably use closed-loop stepper motors with higher precision to achieve more stable and accurate control of the opening and closing degree of the pneumatic paint spray gun.

[0115] like Figure 29 As shown, the controller includes a visual algorithm positioning module, a spraying trajectory generation and optimization module, a robotic arm control module, and an execution terminal control module. The visual algorithm positioning module controls the binocular camera to collect depth and color images, scans the image for feature points, calculates the deviation information of transparent baits and baits with primers, and stores the bait position and deviation information. The spraying trajectory generation and optimization module is configured to convert the three-dimensional model of the bait into an STL format to generate a point cloud model, perform point cloud slicing on the model, generate a spraying trajectory, optimize the spraying trajectory using the deviation information detected by the visual algorithm positioning module, and verify and store the optimized trajectory. The robot control module is used to first perform hand-eye calibration to obtain the conversion matrix between the binocular camera and the actuator end and store it, model the device, generate a URDF file, perform Moveit configuration on the URDF file, and generate a configuration file; then cross-multiply the optimized spray trajectory information by the conversion matrix obtained by hand-eye calibration, obtain the deviation optimized spray trajectory in the actuator end coordinate system, simulate and control the robot arm, realize the spray trajectory reproduction, and perform spraying operations according to the optimized spray trajectory. The execution end control module can control the first drive motor 304-6 and the second drive motor to perform automatic painting operations, and control the first electromagnetic reversing valve 14 and the second electromagnetic reversing valve 18 to switch between two pneumatic paint spray guns, thereby achieving spraying of different colors.

[0116] Connect the bait to be processed to the bait positioning device. The placement of the bait to be processed may not reach the ideal placement state. The spraying effect is better in the ideal placement state. Figure 32 , establish an xyz coordinate system, with the connection position of the end of the bait and the bait connecting rod as the origin, the x-axis is parallel to the long side of the base 502 of the bait positioning device, the y-axis is perpendicular to the long side of the base 502 of the bait positioning device, and the z-axis is along the vertical direction. When the bait is located in the xy plane and the bait is parallel to the y-axis, it is defined that there is no deviation between the bait and the xyz coordinate system (considered to be an ideal placement state), otherwise it is considered that there is a deviation. A certain angle between the bait and the xy plane is considered to be a deviation, and a certain angle between the bait and the y-axis is considered to be a deviation.

[0117] The method for fine automatic spraying of lure bait based on machine vision includes the following steps:

[0118] In the first step, the bait is identified through a visual recognition algorithm and deviation information is calculated.

[0119] It is described in two cases. The first case is for the transparent fishing bait. The transparent fishing bait is a semi-finished fishing bait made of silica gel, without a primer, that is, a fishing bait without a primer. The second case is for the semi-finished fishing bait after soaking and attaching a primer to the fishing bait without a primer, that is, a fishing bait with a primer.

[0120] The first case:

[0121] Step 1, collect the color image and depth image of the fishing bait without a primer connected to the fishing bait positioning device 500 through the binocular camera 307, and transmit the color image and depth image to the controller.

[0122] Step 2, the controller extracts the outer contour of the fishing bait from the color image to facilitate the scanning of feature points by the polygon approximation algorithm. The outer contour is as Figure 38 shown.

[0123] Step 3, scan the feature points on the outer contour of the fishing bait with the polygon approximation algorithm. As Figure 39 shown, the 9 points around are the feature points.

[0124] Step 4, align the depth image with the color image to obtain the aligned image frame and camera parameters; read the depth corresponding to the two-dimensional pixel coordinates of the feature points obtained in Step 3, and then obtain the three-dimensional coordinates of the feature points.

[0125] Step 5, first select three feature points A, B, and C from the detected feature points. The three feature points A, B, and C need to be on the mid-section of the fishing bait, find the corresponding points in the three-dimensional model of the fishing bait, and through the formula get establish the coordinate system of the model feature points, and calculate and obtain the transformation matrix o T f ;

[0126] Step 6, through the detected feature points, establish the coordinate system of the feature points in the camera coordinate system according to the previous method Cross-multiply the coordinate system of the feature points with the transformation matrix o T f to perform a cross-product operation,

[0127] obtain the attitude information of the center point of the fishing bait in the camera coordinate system T cam-o , where, represents the position information, represents the attitude information. In R, represents three mutually perpendicular unit vectors; the pose transformation is usually represented by three methods: rotation matrix, RPY angle, and quaternion. The pose rotation around x, y, and z is respectively represented as:

[0128] (Sometimes the s, c notation is used)

[0129]

[0130] Any rotation in space can be represented as Rot(z, φ)Rot(y, θ)Rot(x, ψ). For the convenience of calculation, the rotation is written as a four-row and four-column homogeneous matrix of

[0131] Translation in space is represented by the following expression:

[0132]

[0133] Thus, a single coordinate transformation in space can be written as:

[0134]

[0135] R contains the angular deviations of the x, y, and z axes of the center point coordinate system of the primerless bait from those of the center point coordinate system of the bait three-dimensional model with respect to the center point coordinate system of the bait three-dimensional model. t contains the position deviations of the center point of the primerless bait from the center point of the bait three-dimensional model along the x, y, and z axes with respect to the center point of the bait three-dimensional model.

[0136] The second case:

[0137] Step 1: Collect the color image and depth image of the baited bait connected to the bait positioning device 500 through the binocular camera 307, and transmit the color image and depth image to the controller.

[0138] Step 2: Convert the depth image into a point cloud map to obtain the point cloud map of the baited bait.

[0139] Step 3: Combine the point cloud map with the RGB map. Refer to Figure 33 .

[0140] Step 4: Convert the format of the bait three-dimensional model drawn by SolidWorks software prepared in advance into the stl format, and import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the bait, and this model point cloud of the bait is used as the model point cloud of the baited bait.

[0141] Step 4: Perform point cloud registration on the point cloud map of the baited bait and the model point cloud through the ICP algorithm, and then obtain the deviation information from the registration result. The deviation information is as follows:

[0142]

[0143] The second step: Determine the spraying trajectory. Refer to Figure 34 .

[0144] Taking the fish back of the fishing bait as an example, the specific process is as follows:

[0145] Step 1: Convert the three-dimensional model of the fishing bait drawn by SolidWorks software in advance into the stl format, import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the fishing bait, and display the information of the fishing bait model point cloud;

[0146] Step 2: Obtain the normal vectors of all points in the fishing bait model point cloud information for reading pose information;

[0147] Step 3: Cut the point cloud information in half. Cut along the central axis of the fishing bait model point cloud, and only show the upper half to facilitate reading the information of the fish back of the fishing bait;

[0148] Step 4: Perform slicing processing on the upper half of the point cloud information formed in Step 3. As Figure 36 shown, cut from top to bottom on both sides of the central axis 1-1 in the directions of section plane 1-2 and section plane 1-3. The distance from the central axis 1-1 to section plane 1-2 is equal to the distance from the central axis 1-1 to section plane 1-3. The distance between section plane 1-2 and section plane 1-3 is δ, that is, the thickness of the cut part is δ, d < δ < 4*d << w, where d = max(di), d i represents the distance between the i-th point of the point cloud and the other point closest to itself, and w represents the width of the upper half of the point cloud. After slicing processing, the fish back curve can be presented.

[0149] Step 5: Take equally spaced points on the fish back curve. Refer to Figure 37 , L represents the distance of taking equally spaced points. If the equally spaced specified position is at a point in the original point cloud, then determine the point in the point cloud. If the equally spaced specified position is not at a point in the original point cloud, then take the point closest to the specified position.

[0150] Step 6: Translate the points obtained in Step 5 along their normal vectors by a preset height to obtain the spraying position points. The preset height is recorded as the spraying height, that is, the distance between the nozzle of the pneumatic paint spray gun and the fishing bait. A number of spraying position points form a curve, and this curve is the spraying trajectory. The number of spraying position points can be displayed to facilitate the user's observation.

[0151] The position information of the spraying position points is x, y, z, and the pose information is qx, qy, qz, w. The quaternion qx, qy, qz, w represents the rotation of the spray gun pose matrix relative to the world coordinate system. Therefore, the pose information of the spraying position points is T p . The following table shows the data of 7 spraying position points:

[0152] x y z w qx qy qz -62.53818 0.03354 20.32650 -0.00984 -0.02141 0.99608 0.08515 -42.01204 0.03267 23.38727 0.00571 0.01663 0.99757 0.06733 -21.21179 0.03596 25.62323 -0.00833 0.00025 0.99914 0.04042 -0.25477 0.03737 26.67834 -0.00825 -0.00448 0.99991 0.00853 21.13692 0.03681 26.02264 -0.00457 -0.00119 0.99926 -0.03791 42.99507 0.03544 22.89105 -0.00265 0.00134 0.99494 -0.10034 59.81892 0.035781 18.92455 -0.00462 -0.01611 0.99159 -0.12830

[0153] Step 7: Optimize the deviation of the spraying position points, which is divided into two cases. The first case is for the bait without primer. Calculate the pose matrix P of the spraying position points in the camera coordinate system c , p C = T cam-o × T p ; The second case is for the bait with primer. Calculate the pose matrix P of the spraying position points in the camera coordinate system c , and the cross product of the deviation information and the pose information is T p , P c = T cam-o2 × T p That is

[0154] Step 8: Store the optimized pose matrix P c .

[0155] Thirdly, the manipulator moves the execution end to the working position

[0156] Step 1: Since the coordinates for controlling the manipulator movement are in the manipulator base coordinate system, it is necessary to measure the transformation matrix from the camera to the execution end through hand-eye calibration. Then, the transformation matrices Tr from the bait to the camera, from the camera to the execution end, and from the execution end to the manipulator base coordinate can be constructed. By obtaining this matrix Tr, the above pose matrix P c can be directly cross-multiplied to control the manipulator movement

[0157] The hand-eye calibration process is selected for equipment calibration, and the calibration results obtained by measuring 25 groups of calibration data have been verified with an error within 3 mm. The specific steps of hand-eye calibration are as follows: Store the internal parameters of the binocular camera in the controller; Prepare a hand-eye calibration board (as shown in Figure 35 ), and place the hand-eye calibration board on the bait positioning device 500; The controller controls the manipulator to run to the working position, calls the binocular camera to collect pictures of the hand-eye calibration board, and reads the pose information of the execution end; The controller controls the manipulator to change to another pose, calls the binocular camera to collect pictures of the hand-eye calibration board again, and repeats taking 25 groups of data; Obtain the calibration matrix Tr based on the 25 groups of data and the pose information of the execution end

[0158] Step 2: Cross-multiply the optimized pose matrix P c by Tr to obtain the corresponding spraying position of the execution end

[0159] Step 3: Conduct simulation control on the manipulator. The specific process is as follows

[0160] First, model the entire robot. The file format for modeling in the ROS system is the URDF file. Secondly, check the modeled URDF file in the ROS system to display its structural information. Thirdly, after importing the modeled URDF file into the ROS system, perform MoveIt configuration to generate a collision matrix, create virtual joints, add motion planning groups, add set poses, mark the end effector, and other configuration files. Then, link the generated MoveIt configuration files with the Gazebo simulation platform for joint simulation. Finally, control the robotic arm through the ROS system to drive the end effector to move to the working position.

[0161] The state velocity planning and state monitoring of the robotic arm can be carried out through Matlab software. It is necessary to establish a Matlab model of the robotic arm, which is established using the improved DH parameter method, and then import the established robotic arm model into Matlab.

[0162] In the fourth step, the controller controls the drive motor in the end effector to work, starts the pneumatic paint spray gun, and the pneumatic paint spray gun sprays the bait. Fine automatic spraying is achieved.

Claims

1. An automatic fine spraying method for lure fishing baits based on machine vision, characterized in that, It includes a refined spraying robot for lure baits. The refined spraying robot for lure baits includes a robotic arm connection bracket, a robotic arm, an execution end, a workbench, and a bait positioning device. The robotic arm is connected to the robotic arm connection bracket, the execution end is connected to the end of the robotic arm, and the bait positioning device is connected to the middle of the workbench. The refined automatic spraying method for lure baits includes the following steps: In the first step, identify the bait through a visual recognition algorithm and calculate deviation information. Step 1: Collect the color image and depth image of the primerless bait connected to the bait positioning device through a binocular camera, and transmit the color image and depth image to the controller. Step 2: The controller extracts the outer contour of the bait from the color image to facilitate the polygon approximation algorithm to scan feature points. Step 3: Scan the feature points of the outer contour of the bait using the polygon approximation algorithm. Step 4: Align the depth image with the color image, read the depth corresponding to the two-dimensional pixel coordinates of the feature points obtained in the previous step, and thus obtain the three-dimensional coordinates of the feature points. Step 5: First, select three feature points A, B, and C from the detected feature points, find the corresponding points in the three-dimensional model of the bait, and obtain through the formula to get Establish a coordinate system for the model feature points, and calculate the transformation matrix between the coordinate system of the model feature points and the origin coordinate system of the model o T f ; Step 6: Establish a feature point coordinate system in the camera coordinate system based on the detected feature points Multiply the feature point coordinate system with the transformation matrix o T f by cross product operation to obtain the attitude information T of the center point of the fishing bait in the camera coordinate system cam-o ; In the second step, determine the spraying trajectory. Step 1: Convert the three-dimensional model of the bait drawn by SolidWorks software prepared in advance into the stl format, import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the bait, and display the model point cloud information of the bait. Step 2: Obtain the normal vectors of all points in the model point cloud information of the bait for reading pose information. Step 3: Cut the point cloud information in half. Cut along the central axis of the model point cloud of the bait, and only show the upper half to facilitate reading the back information of the bait. Step 4: Slice the upper half of the point cloud information formed in Step 3. Cut from top to bottom on both sides of the central axis in the directions of Section 1 and Section 2. The distance between Section 1 and Section 2 is δ, where d < δ < 4*d << w. Here, d = max(di), and d i represents the distance between the i-th point of the point cloud and the other point closest to itself, and w represents the width of the upper half of the point cloud. After slicing, a fish-back curve is presented; Step 5: Take equally spaced points on the back curve of the bait. Step 6: Translate the points obtained in Step 5 along their normal directions by a preset height to obtain the spraying position points. The preset height is denoted as the spraying height, which is the distance between the nozzle of the pneumatic paint spray gun and the fishing bait. A number of spraying position points form a curve, and this curve is the spraying trajectory. The pose information of the spraying position points is T p ; Step 7, optimize the deviation of the spraying position points, and calculate the pose matrix P of the spraying position points in the camera coordinate system c , p C = T cam-o × T p ; In the third step, the robotic arm moves the execution end to the working position. The hand-eye calibration yields the calibration matrix Tr; the optimized pose matrix P c is cross-multiplied by Tr to obtain the spraying position corresponding to the end effector; the robotic arm drives the end effector to move to the working position; In the fourth step, the controller controls the driving motor in the execution end to work, starts the pneumatic paint spray gun, and the pneumatic paint spray gun sprays the bait.

2. A refined automatic spraying method for lure fishing baits based on machine vision, characterized in that, It includes a refined spraying robot for lure baits. The refined spraying robot for lure baits includes a robotic arm connection bracket, a robotic arm, an execution end, a workbench, and a bait positioning device. The robotic arm is connected to the robotic arm connection bracket, the execution end is connected to the end of the robotic arm, and the bait positioning device is connected to the middle of the workbench. The refined automatic spraying method for lure baits includes the following steps: In the first step, identify the bait through a visual recognition algorithm and calculate deviation information. Step 1: Collect the color image and depth image of the primed bait connected to the bait positioning device through a binocular camera, and transmit the color image and depth image to the controller. Step 2: Convert the depth image into a point cloud map to obtain the point cloud map of the primed bait. Step 3: Combine the point cloud map with the RGB map. Step 4: Convert the three-dimensional model of the bait drawn by SolidWorks software prepared in advance into the stl format, import the three-dimensional model in the stl format into the open3D library to obtain the model point cloud of the bait, and this model point cloud of the bait is used as the model point cloud of the primed bait. Step 5: Perform point cloud registration on the point cloud map of the bait with primer and the model point cloud through the ICP algorithm, and then obtain the deviation information T from the registration result cam-o2 ; In the second step, determine the spraying trajectory. Step 1: Convert the 3D model of the fishing bait drawn with SolidWorks software in advance into the stl format, import the 3D model in stl format into the open3D library to obtain the model point cloud of the fishing bait, and display the information of the fishing bait model point cloud; Step 2: Obtain the normals of all points in the fishing bait model point cloud information for reading the pose information; Step 3: Cut the point cloud information in half. Cut along the central axis of the fishing bait model point cloud and only show the upper half to facilitate reading the information of the fish back of the fishing bait; Step 4: Slice the upper part of the point cloud information formed in Step 3. Cut from top to bottom on both sides of the central axis in the directions of section one and section two. The distance between section one and section two is δ, where d < δ < 4*d << w. Here, d = max(di), and d i represents the distance between the i-th point of the point cloud and the other point closest to itself, and w represents the width of the upper part of the point cloud. After slicing, a fish-back curve is presented; Step 5: Take equidistant points on the fish back curve; Step 6: Translate the points obtained in Step 5 along their normal directions by a preset height to obtain spray position points. The preset height is denoted as the spray height, which is the distance between the nozzle of the pneumatic paint spray gun and the fishing bait. A number of spray position points form a curve, and this curve is the spray trajectory. The pose information of the spray position points is T p ; Step 7, perform deviation optimization on the spraying position points, P c = T cam-o2 × T p ; The third step: The manipulator moves the execution end to the operation position; The hand-eye calibration yields the calibration matrix Tr; the optimized pose matrix P c is cross-multiplied by Tr to obtain the spraying position corresponding to the end effector; the robotic arm drives the end effector to move to the working position; The fourth step: The controller controls the driving motor in the execution end to work, starts the pneumatic paint spray gun, and the pneumatic paint spray gun sprays the fishing bait.

Images