Method and system for guiding deviation correction of mechanical arm in visual mode and mechanical arm equipment

Abstract

The invention relates to a method, system and equipment for guiding a manipulator to rectify deviation in a visual mode. The method comprises the steps that when a material taking and placing structure reaches a preset deviation rectification calibration condition, a deviation rectification calibration instruction for the material taking and placing structure is sent out; in response to the deviation rectification calibration instruction, controlling a manipulator driving mechanism to drive a material taking and placing structure to repeatedly move to a position right above a camera vision calibration mechanism, and controlling the camera vision calibration mechanism to perform vision identification on the material taking and placing structure with a vision calibration identification structure to obtain a plurality of vision identification calibration images; according to the obtained visual identification calibration images of the multiple material taking and placing structures, deviation correction calibration positions of the material taking and placing structures are obtained; and when it is confirmed that the material taking and placing structure deviates, the deviation correction calibration position is set as the new coordinate origin position of the material taking and placing structure. The problems that after a traditional mechanical arm circularly acts for a long time, point position deviation is likely to happen, and then the feeding accuracy, the product quality and the production efficiency are affected can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of flexible circuit board production, and in particular to a method, system and robot device for guiding a robot to correct deviation using a visual method. Background Art

[0002] During the production of printed circuit boards, testing is often required. To improve testing efficiency, automated gantry robots are often used to load and unload test fixtures. However, with traditional technologies, these robots are prone to positional deviation after prolonged cycling, which in turn affects feeding accuracy, product quality, and production efficiency. Summary of the Invention

[0003] The present invention provides a method, system and robot equipment for guiding a robot to correct deviation using visual means, which can solve the technical problem in traditional technology that a robot is prone to point deviation after long-term cyclic action, thereby affecting feeding accuracy, product quality and production efficiency.

[0004] In order to solve the above technical problems, the present invention provides a method for guiding a robot to correct deviation using a visual method, which is applied to a robot device; the robot device includes a robot truss, a robot drive mechanism provided on the robot truss, a material picking and placing structure provided on the robot drive mechanism, a camera vision calibration mechanism provided on the robot truss, and a vision calibration mark structure provided on the material picking and placing structure and correspondingly located above the camera vision calibration mechanism;

[0005] The method comprises:

[0006] When the real-time material taking and putting times of the material taking and putting structure reaches a preset number of material taking and putting times, or / and the real-time working time reaches a preset time, a correction calibration instruction for the material taking and putting structure is issued;

[0007] In response to the deflection correction and calibration instruction, the robot drive mechanism is controlled to drive the material picking and placing structure to repeatedly move to directly above the camera vision calibration mechanism, and the camera vision calibration mechanism is controlled to perform visual recognition on the material picking and placing structure having the vision calibration mark structure to obtain a plurality of visual recognition calibration images;

[0008] Obtaining a deviation correction calibration position of the material picking and placing structure based on the obtained plurality of visual recognition calibration images of the material picking and placing structure;

[0009] When it is confirmed that the material picking and placing structure has deviated, the deviation correction calibration position is set as the new coordinate origin position of the material picking and placing structure to complete the deviation correction calibration operation of the material picking and placing structure.

[0010] Optionally, the controlling manipulator driving mechanism drives the material picking and placing structure to repeatedly move to directly above the camera vision calibration mechanism, and controls the camera vision calibration mechanism to perform visual recognition on the material picking and placing structure having the vision calibration identification structure, and obtains a plurality of vision recognition calibration images, including:

[0011] Controlling the manipulator drive mechanism to drive the material picking and placing structure to move to the top of the camera vision calibration mechanism in the X-axis direction, Y-axis direction, Z-axis direction, XY-axis direction, and XYZ-axis direction for a preset number of repetitions;

[0012] Each time the material taking and placing structure moves to the position directly above the camera vision calibration mechanism, the camera vision calibration mechanism is controlled to perform visual recognition on the material taking and placing structure having the vision calibration identification structure, and a plurality of vision recognition calibration images are acquired.

[0013] Optionally, the material picking and placing structure includes a material picking and placing mounting seat provided on the manipulator drive mechanism, a material picking and placing adsorption plate provided on the material picking and placing mounting seat, and a plurality of vacuum suction nozzles protruding from the bottom of the material picking and placing adsorption plate;

[0014] The visual calibration mark structure includes a visual calibration mark block provided at the bottom of the material pickup and placement adsorption plate, the bottom surface of the visual calibration mark block is provided with a visual calibration mark pattern, and the visual calibration mark pattern corresponds to the camera visual calibration mechanism in the upper and lower directions;

[0015] The controlling the camera vision calibration mechanism to perform visual recognition on the material picking and placing structure having the vision calibration identification structure to obtain a plurality of vision recognition calibration images includes:

[0016] When the material picking and placing structure moves to the position directly above the camera vision calibration mechanism, the camera vision calibration mechanism is controlled to visually photograph the bottom surface of the material picking and placing adsorption plate of the material picking and placing structure and the visual calibration mark pattern on the bottom surface of the visual calibration mark structure thereon, to obtain a visual recognition calibration image of the visual calibration mark structure having the visual calibration mark pattern;

[0017] The above steps are repeated multiple times to obtain multiple vision recognition calibration images of the vision calibration mark pattern having the vision calibration mark structure.

[0018] Optionally, the visual calibration mark block is a rectangular block, and the visual calibration mark pattern is a cross-shaped mark pattern; and the bottom surface of the rectangular block is a black bottom surface or a white bottom surface, and the cross-shaped mark pattern is a white pattern or a black pattern correspondingly;

[0019] The acquiring of the visual recognition calibration image having the visual calibration mark pattern of the visual calibration mark structure includes:

[0020] Acquire a visual recognition calibration image of the bottom surface of the material picking and placing plate of the material picking and placing structure;

[0021] The visual recognition calibration image has the visual calibration mark structure which is a rectangular block provided on the bottom surface of the pick-up and unloading plate, and the bottom surface of the visual calibration mark structure has the visual calibration mark pattern which is a cross-shaped mark pattern.

[0022] Optionally, obtaining the correction calibration position of the material picking and placing structure based on the obtained plurality of visual recognition calibration images of the material picking and placing structure includes:

[0023] Acquiring multiple marker center positions of the visual calibration marker structure according to the obtained multiple visual recognition calibration images;

[0024] According to the preset calibration center position and the obtained multiple identification center positions, the correction calibration position of the material taking and placing structure is obtained.

[0025] Optionally, obtaining multiple marker center positions of the visual calibration marker structure according to the obtained multiple visual recognition calibration images includes:

[0026] According to the obtained visual recognition calibration image in which the visual calibration mark pattern is set as a cross-shaped mark pattern, obtaining the mark center position of the cross-shaped mark pattern on the visual recognition calibration image by a nine-point calibration algorithm;

[0027] Repeat the above steps to obtain the center position of each cross-shaped marking pattern in the plurality of visual recognition calibration images.

[0028] Optionally, obtaining the deviation correction calibration position of the material taking and placing structure according to the preset calibration center position and the obtained multiple identification center positions includes:

[0029] Comparing the preset calibration center position and the obtained multiple identification center positions with the preset calibration center position to obtain multiple center position deviations;

[0030] According to the preset calibration center position and the obtained multiple center position deviations, the calibrated correction calibration position of the material taking and placing structure is obtained.

[0031] Optionally, when it is confirmed that the material picking and placing structure is offset, setting the deviation correction calibration position as a new coordinate origin position of the material picking and placing structure includes:

[0032] Verifying the multiple marker center positions using a CPK verification method according to the preset calibration center position, the multiple marker center positions, and the obtained multiple center position deviations;

[0033] When it is detected that the CPK values ​​of the center positions of the multiple markers do not meet the preset accuracy requirements, it is determined that the material taking and placing structure is offset;

[0034] When it is confirmed that the material taking and placing structure is offset, the deviation correction calibration position is set as the new coordinate origin position of the material taking and placing structure.

[0035] In addition, the present invention also proposes a system for guiding a robot to correct deviation using a visual method, which is applied to a robot device; the robot device includes a robot truss, a robot drive mechanism provided on the robot truss, a material picking and placing structure provided on the robot drive mechanism, a camera vision calibration mechanism provided on the robot truss, and a vision calibration marking structure provided on the material picking and placing structure and correspondingly located above the camera vision calibration mechanism;

[0036] The system comprises:

[0037] A deviation correction starting module is used to issue a deviation correction calibration instruction to the material picking and unloading structure when the real-time material picking and unloading times of the material picking and unloading structure reach a preset number of times, or / and the real-time working time reaches a preset time;

[0038] a deflection correction and calibration identification module, configured to control the manipulator drive mechanism to drive the pick-up and put-down structure to repeatedly move to directly above the camera vision calibration mechanism in response to the deflection correction and calibration instruction, and control the camera vision calibration mechanism to perform visual recognition on the pick-up and put-down structure having the vision calibration identification structure, thereby acquiring a plurality of vision recognition calibration images;

[0039] a deflection correction and calibration processing module, configured to obtain a deflection correction and calibration position of the material picking and placing structure based on the obtained visual recognition calibration images of the plurality of material picking and placing structures;

[0040] The correction setting module is used to set the correction calibration position as the new coordinate origin position of the material picking and placing structure when it is confirmed that the material picking and placing structure is offset, so as to complete the correction calibration operation of the material picking and placing structure.

[0041] In addition, the present invention also provides a manipulator device, comprising:

[0042] Manipulator truss;

[0043] The manipulator body comprises a manipulator drive mechanism provided on the manipulator truss, and a material taking and placing structure provided on the manipulator drive mechanism;

[0044] A correction and calibration mechanism, comprising a camera vision calibration mechanism provided on the manipulator truss, and a vision calibration marking structure provided on the material taking and placing structure and correspondingly located above the camera vision calibration mechanism; and

[0045] A controller connected to the manipulator body and the deviation correction and calibration mechanism;

[0046] Wherein, the controller is used to implement the above-mentioned method of using visual means to guide the robot to correct the deviation.

[0047] The beneficial effects brought about by the technical solution provided by the present invention include:

[0048] When the operation of the manipulator device reaches the preset calibration conditions (such as the number of real-time pick-up and put-out times reaches the preset number of pick-up and put-out times, or the real-time working time reaches the preset time), the automatic correction calibration of the manipulator device can be started. When calibrating the manipulator device, the main thing is to calibrate the movement repeatability accuracy of the pick-up and put-out structure, that is, whether the working origin position of the pick-up and put-out structure has shifted. Therefore, the pick-up and put-out structure can be repeatedly driven by the manipulator drive mechanism to move to the top of the camera vision calibration mechanism, and the visual calibration mark structure on the pick-up and put-out structure can be repeatedly visually identified and calibrated by the camera vision calibration mechanism to obtain the correction calibration position of the pick-up and put-out structure, and when the working origin position of the pick-up and put-out structure shifts, the correction calibration position is set as the new coordinate origin position of the pick-up and put-out structure.

[0049] In this way, after the robot equipment has been in a long-term cyclic action, the visual correction calibration mechanism composed of the camera vision calibration mechanism and the visual calibration identification structure set by itself can automatically detect the point offset of the picking and placing structure of the robot equipment, and automatically calibrate and correct the picking and placing structure when a point offset is detected, thereby ensuring the accuracy of the loading and unloading of the picking and placing structure in the subsequent work process, ensuring product quality and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 This is a schematic diagram of the steps of a method for guiding a robot to correct deviation using a visual method according to an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the three-dimensional structure of the manipulator device according to an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of the three-dimensional structure of the manipulator drive mechanism and the material handling structure of the manipulator device according to an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the three-dimensional structure of a manipulator device according to an embodiment of the present invention when a visual calibration marking structure is provided on a material picking and placing adsorption plate of the material picking and placing structure;

[0055] Figure 5 This is a simplified block diagram showing the structure of a system for guiding a robot to correct deviation using a visual method according to an embodiment of the present invention;

[0056] Figure 6 This is a simplified block diagram showing the structure of the manipulator device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0058] like Figure 1 As shown, the invention provides a method for guiding a robot to correct deviation using a visual method, which is applied to a robot device 100. Figures 2 to 4 As shown, the manipulator device 100 may include a manipulator truss 110, a manipulator drive mechanism 120 disposed on the manipulator truss 110, a material handling structure 130 disposed on the manipulator drive mechanism 120, a camera vision calibration mechanism 140 disposed on the manipulator truss 110, and a vision calibration mark structure 150 disposed on the material handling structure 130 and correspondingly located above the camera vision calibration mechanism 140. The manipulator drive mechanism 120 is used to drive the material handling structure 130 to move in the horizontal and vertical directions, so that the material handling structure 130 can grasp, move, and place materials; and the camera vision calibration mechanism 140 can photograph and detect the bottom surface of the material handling structure 130 and the vision calibration mark structure 150 located thereon from below.

[0059] Specifically, if Figure 1 As shown, the method of using vision to guide the robot to correct the deviation may include the following steps:

[0060] S100, when the real-time number of material picking and placing of the material picking and placing structure 130 of the manipulator device 100 reaches a preset number of material picking and placing, or / and the real-time working time reaches a preset time, issuing a correction calibration instruction for the material picking and placing structure 130;

[0061] S200, in response to the correction calibration instruction, controlling the manipulator drive mechanism 120 to drive the pick-up and place structure 130 to repeatedly move to directly above the camera vision calibration mechanism 140, and controlling the camera vision calibration mechanism 140 to visually identify the pick-up and place structure 130 having the vision calibration mark structure 150, and obtaining a plurality of visual recognition calibration images;

[0062] S300, obtaining a correction calibration position of the material picking and placing structure 130 based on the obtained visual recognition calibration images of the plurality of material picking and placing structures 130;

[0063] S400 , when it is confirmed that the material taking and placing structure 130 is offset, setting the deviation correction calibration position as the new coordinate origin position of the material taking and placing structure 130 , and completing the deviation correction calibration operation of the material taking and placing structure 130 .

[0064] When the operation of the manipulator device 100 reaches a preset calibration condition (such as the number of real-time material picking and unloading reaches a preset number of material picking and unloading, or the real-time working time reaches a preset time), the automatic correction calibration of the manipulator device 100 can be started. When calibrating the manipulator device 100, the main thing is to calibrate the movement repeatability accuracy of the material picking and unloading structure 130, that is, whether the working origin position of the material picking and unloading structure 130 has shifted. Therefore, the manipulator drive mechanism 120 can repeatedly drive the material picking and unloading structure 130 to move to the top of the camera vision calibration mechanism 140, and the camera vision calibration mechanism 140 can repeatedly perform visual recognition calibration on the visual calibration mark structure 150 on the material picking and unloading structure 130 to obtain the correction calibration position of the material picking and unloading structure 130, and when the working origin position of the material picking and unloading structure 130 shifts, the correction calibration position is set as the new coordinate origin position of the material picking and unloading structure 130.

[0065] In this way, after the robot device 100 has been in a long-term cyclic action, the visual correction calibration mechanism composed of the camera vision calibration mechanism 140 and the visual calibration identification structure 150 set by itself can automatically detect the point offset of the picking and placing structure 130 of the robot device 100, and automatically calibrate and correct the picking and placing structure 130 when the point offset is detected, thereby ensuring the accuracy of the loading and unloading of the picking and placing structure 130 in the subsequent work process, ensuring product quality, and improving production efficiency.

[0066] Furthermore, in step S100, the following steps may be further included:

[0067] S110 , pre-setting a device timing device to record the working time of the material picking and placing structure 130 of the manipulator device 100 , and obtaining the real-time working time of the material picking and placing structure 130 .

[0068] A device timing device can be set on the robot device 100 or at a location far away from the robot device 100. During the operation of the robot device 100, the working time of the robot device 100 from the first startup is counted, and the real-time working time of the robot device 100 and its material picking and placing structure 130 is recorded.

[0069] S120 , or / and a pre-set material taking and placing monitoring device records the material taking and placing times of the material taking and placing structure 130 of the manipulator device 100 , and obtains the real-time material taking and placing times of the material taking and placing structure 130 .

[0070] That is, a device timing device can also be installed on the robot device 100 or at a location away from the robot device 100 to record the picking and placing actions of the picking and placing structure 130 of the robot device 100. During the operation of the robot device 100, each picking and placing action of the picking and placing structure 130 of the robot device 100 is counted since the robot device 100 is first turned on, and the real-time picking and placing times of the picking and placing structure 130 of the robot device 100 are recorded.

[0071] S130 , when the real-time material picking and placing times of the material picking and placing structure 130 of the manipulator device 100 reaches a preset material picking and placing times, or / and the real-time working time reaches a preset time, a correction calibration instruction is issued to the material picking and placing structure 130 .

[0072] When it is detected that the operation of the robot arm device 100 reaches the preset calibration conditions (such as the real-time picking and placing times reach the preset picking and placing times, or / and the real-time working time reaches the preset time), a correction calibration instruction for the picking and placing structure 130 can be issued to the robot arm device 100 to start the correction calibration operation of the picking and placing structure 130.

[0073] Furthermore, in step S200, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to repeatedly move to directly above the camera vision calibration mechanism 140, and controlling the camera vision calibration mechanism 140 to visually recognize the pick-up and place structure 130 having the vision calibration mark structure 150, and obtaining a plurality of vision recognition calibration images, may further include the following steps:

[0074] S210, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to move to the top of the camera vision calibration mechanism 140 in the X-axis direction, the Y-axis direction, the Z-axis direction, the XY-axis direction, and the XYZ-axis direction for a preset number of repetitions;

[0075] S220 , each time the material picking and placing structure 130 moves to the position directly above the camera vision calibration mechanism 140 , the camera vision calibration mechanism 140 is controlled to perform visual recognition on the material picking and placing structure 130 having the vision calibration identification structure 150 , and a plurality of vision recognition calibration images are obtained.

[0076] When testing the movement repeatability accuracy of the material picking and placing structure 130 of the robot device 100, it is necessary to test the single-axis (such as X-axis, Y-axis, Z-axis) movement repeatability accuracy of the material picking and placing structure 130, and it is also necessary to test the multi-axis (such as XY axis, XYZ axis) movement repeatability accuracy of the material picking and placing structure 130.

[0077] Specifically, the above step S200 may further include:

[0078] S2001, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to move a first preset distance from a first preset starting position in the X-axis direction to a position directly above the camera vision calibration mechanism 140;

[0079] S2002: When the material picking and placing structure 130 reaches directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to visually photograph the bottom surface of the material picking and placing structure 130 and the vision calibration mark structure 150 thereon, thereby obtaining a first visual recognition calibration image having the vision calibration mark structure 150, and the manipulator driving mechanism 120 is controlled to drive the material picking and placing structure 130 to return to the first preset starting position.

[0080] S2003 , performing the above steps (ie, steps S2001 and S2002 ) for a first preset number of repetitions to obtain a plurality of first visual recognition calibration images.

[0081] This means that visual recognition calibration images can be captured from the X-axis of the pick-up / placement mechanism 130 of the robot device 100. This recognition operation can be repeated multiple times to obtain multiple first visual recognition calibration images, facilitating subsequent testing of the repeatability of the pick-up / placement mechanism 130's movement along the X-axis. In this embodiment, the first preset number of repetitions can be 20 or more, corresponding to 20 or more first visual recognition calibration images.

[0082] Similarly, the above step S200 may further include:

[0083] S2004, controlling the manipulator driving mechanism 120 to drive the material picking and placing structure 130 to move a second preset distance from a second preset starting position in the Y-axis direction to a position directly above the camera vision calibration mechanism 140;

[0084] S2005: When the material picking and placing structure 130 reaches directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to visually photograph the bottom surface of the material picking and placing structure 130 and the vision calibration mark structure 150 thereon, thereby obtaining a second visual recognition calibration image having the vision calibration mark structure 150, and the manipulator driving mechanism 120 is controlled to drive the material picking and placing structure 130 to return to the second preset starting position.

[0085] S2006 , performing the above steps (ie, steps S2004 and S2005 ) for a second preset number of repetitions to obtain a plurality of second visual recognition calibration images.

[0086] Similarly, visual recognition calibration images can be acquired from the Y-axis of the pick-up / placement mechanism 130 of the robot device 100. This recognition action can be repeated multiple times to acquire multiple second visual recognition calibration images, facilitating subsequent testing of the repeatability of the pick-up / placement mechanism 130's movement along the Y-axis. In this embodiment, the second predetermined number of repetitions can be 20 or more, corresponding to 20 or more acquired second visual recognition calibration images.

[0087] Similarly, the above step S200 may further include:

[0088] S2007, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to move a third preset distance from a third preset starting position in the Z-axis direction to directly above the camera vision calibration mechanism 140;

[0089] S2008: When the material picking and placing structure 130 reaches directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to visually photograph the bottom surface of the material picking and placing structure 130 and the vision calibration mark structure 150 thereon to obtain a third vision recognition calibration image having the vision calibration mark structure 150, and the manipulator driving mechanism 120 is controlled to drive the material picking and placing structure 130 to return to the third preset starting position.

[0090] S2009 , performing the above steps (ie, steps S2007 and S2008 ) for a third preset number of repetitions to obtain a plurality of third visual recognition calibration images.

[0091] Similarly, visual recognition calibration images can be acquired from the Z-axis of the pick-up / placement mechanism 130 of the robot device 100. This recognition operation can be repeated multiple times to acquire multiple third visual recognition calibration images, facilitating subsequent testing of the repeatability of the pick-up / placement mechanism 130's movement along the Z-axis. In this embodiment, the third preset number of repetitions can be 20 or more, corresponding to 20 or more third visual recognition calibration images.

[0092] Similarly, the above step S200 may further include:

[0093] S2010, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to move a fourth preset distance from a fourth preset starting position in the XY axis direction to a position directly above the camera vision calibration mechanism 140;

[0094] S2011, when the material picking and placing structure 130 reaches directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to visually photograph the bottom surface of the material picking and placing structure 130 and the vision calibration mark structure 150 thereon to obtain a fourth vision recognition calibration image having the vision calibration mark structure 150, and the manipulator driving mechanism 120 is controlled to drive the material picking and placing structure 130 to return to the fourth preset starting position;

[0095] S2012 , performing the above steps (ie, steps S2010 and S2012 ) for a fourth preset number of repetitions to obtain a plurality of fourth visual recognition calibration images.

[0096] Similarly, visual recognition calibration images can be acquired from the pick-up and place mechanism 130 of the robot device 100 along the X and Y axes. This recognition operation can be repeated multiple times to acquire multiple fourth visual recognition calibration images, facilitating subsequent testing of the repeatability of the pick-up and place mechanism 130 along the X and Y axes. In this embodiment, the fourth predetermined number of repetitions can be 20 or more, corresponding to 20 or more acquired fourth visual recognition calibration images.

[0097] Similarly, the above step S200 may further include:

[0098] S2013, controlling the manipulator driving mechanism 120 to drive the pick-up and place structure 130 to move a fifth preset distance from a fifth preset starting position in the XYZ axis direction to directly above the camera vision calibration mechanism 140;

[0099] S2014: When the material picking and placing structure 130 reaches directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to visually photograph the bottom surface of the material picking and placing structure 130 and the vision calibration mark structure 150 thereon to obtain a fifth vision recognition calibration image having the vision calibration mark structure 150, and the manipulator driving mechanism 120 is controlled to drive the material picking and placing structure 130 to return to the fifth preset starting position.

[0100] S2015 , performing the above steps (ie, steps S2013 and S2014 ) for a fifth preset number of repetitions to obtain a plurality of fifth visual recognition calibration images.

[0101] Similarly, visual recognition calibration images can be acquired from the pick-up / placement mechanism 130 of the robot device 100 along the X, Y, and Z axes. This recognition operation can be repeated multiple times to acquire multiple fifth visual recognition calibration images, facilitating subsequent testing of the repeatability of the pick-up / placement mechanism 130 along the X, Y, and Z axes. In this embodiment, the fifth predetermined number of repetitions can be 20 or more, corresponding to 20 or more acquired fifth visual recognition calibration images.

[0102] Moreover, if Figures 2 to 4 As shown, the material picking and placing structure 130 of the manipulator device 100 may include a material picking and placing mounting base provided on the manipulator drive mechanism 120, a material picking and placing suction plate 132 provided on the material picking and placing mounting base, and a plurality of vacuum suction nozzles 134 protruding from the bottom of the material picking and placing suction plate 132. The plurality of vacuum suction nozzles 134 protruding from the bottom of the material picking and placing suction plate 132 can be used to suck, grasp, or release materials.

[0103] Furthermore, the visual calibration mark structure 150 may include a visual calibration mark block 152 disposed at the bottom of the pick-up and unplacing material suction plate 132. The bottom surface of the visual calibration mark block 152 may be provided with a visual calibration mark pattern 154. The visual calibration mark pattern 154 corresponds to and cooperates with the camera visual calibration mechanism 140. The camera visual calibration mechanism 140 may include a visual camera module disposed at the bottom of the manipulator truss 110. The visual camera module may be connected to a control mechanism (e.g., controller 160) of the manipulator device 100. Furthermore, the visual camera module is located below the pick-up and unplacing structure 130 and may be used to photograph the visual calibration mark block 152 and the visual calibration mark pattern 154 on the bottom surface of the pick-up and unplacing material suction plate 132 of the pick-up and unplacing structure 130, thereby obtaining a visual recognition calibration image of the pick-up and unplacing structure 130.

[0104] Therefore, in step S220, controlling the camera vision calibration mechanism 140 to perform visual recognition on the material picking and placing structure 130 having the vision calibration identification structure 150 to obtain a plurality of vision recognition calibration images may further include the following steps:

[0105] S222. When the material picking and placing structure 130 moves to directly above the camera vision calibration mechanism 140, the camera vision calibration mechanism 140 is controlled to take visual photos of the bottom surface of the material picking and placing adsorption plate 132 of the material picking and placing structure 130 and the visual calibration identification pattern 154 on the bottom surface of the visual calibration identification structure 150 thereon, and obtain a visual recognition calibration image of the visual calibration identification pattern 154 having the visual calibration identification structure 150.

[0106] That is, the material picking and placing structure 130 moves a preset distance (such as the first preset distance, the second preset distance, the third preset distance, the fourth preset distance, the fifth preset distance, etc.) along the X-axis, Y-axis, Z-axis, XY-axis, and XYZ-axis directions respectively from a preset starting position (such as the first preset starting position, the second preset starting position, the third preset starting position, the fourth preset starting position, the fifth preset starting position, etc.). When it reaches directly above the visual camera module of the camera vision calibration mechanism 140, the control mechanism can control the visual camera module to take a picture of the material picking and placing adsorption plate 132 of the visual calibration identification structure 150 having a visual calibration identification pattern 154 on the bottom surface directly above it from below, and obtain a visual recognition calibration image with the visual calibration identification pattern 154.

[0107] S224 , repeat the above steps multiple times to obtain multiple visual recognition calibration images of the visual calibration mark pattern 154 having the visual calibration mark structure 150 .

[0108] That is, when the movement is repeated multiple times in the X-axis direction, the corresponding multiple first visual recognition calibration images are obtained; when the movement is repeated multiple times in the Y-axis direction, the corresponding multiple second visual recognition calibration images are obtained; when the movement is repeated multiple times in the Z-axis direction, the corresponding multiple third visual recognition calibration images are obtained; when the movement is repeated multiple times in the XY-axis direction, the corresponding multiple fourth visual recognition calibration images are obtained; and when the movement is repeated multiple times in the XYZ-axis direction, the corresponding multiple fifth visual recognition calibration images are obtained.

[0109] Furthermore, at least one visual calibration mark block 152 may be provided at the bottom of the pick-up / drop-out suction plate 132. This may be a single visual calibration mark block 152 with a visual calibration mark pattern 154, or multiple visual calibration marks 152 with visual calibration marks 154 may be provided at the bottom of the pick-up / drop-out suction plate 132. During the inspection and calibration process, one visual calibration mark block 152 may be used as the primary inspection and calibration block, while the other visual calibration marks 152 serve as redundant backups. Alternatively, multiple visual calibration marks 152 may be inspected simultaneously.

[0110] In this embodiment, two visual calibration markers 152 can be positioned side by side at the bottom of the pick-up / unpick-up plate 132. These two visual calibration markers 152 can be positioned outside the plurality of vacuum nozzles 134 at the bottom of the pick-up / unpick-up plate 132. Furthermore, the pick-up / unpick-up plate 132 can be a rectangular plate, and the two visual calibration markers 152 can be symmetrically arranged about the width centerline or length centerline of the rectangular bottom surface of the rectangular plate. Furthermore, the two visual calibration markers 152 can be positioned at the edges of the rectangular bottom surface of the rectangular plate, while the plurality of vacuum nozzles 134 can be positioned in the center of the rectangular bottom surface of the rectangular plate.

[0111] Furthermore, the visual calibration mark block 152 can be embedded in the bottom of the material handling suction plate 132, with the bottom surface of the visual calibration mark block 152 flush with the bottom surface of the material handling suction plate 132. This allows the visual calibration mark block 152 to be embedded in the bottom of the material handling suction plate 132, facilitating assembly and disassembly. Furthermore, the bottom surface of the visual calibration mark block 152 can be flush with the bottom surface of the material handling suction plate 132, without interfering with the camera's ability to identify the visual calibration mark pattern 154 on the bottom surface of the visual calibration mark block 152. Furthermore, the visual calibration mark structure 150 can also include a mark block base embedded in the bottom of the material handling suction plate 132. The visual calibration mark block 152 can be embedded in the mark block base, with the bottom surfaces of the visual calibration mark block 152 and the mark block base both flush with the bottom surface of the material handling suction plate 132. That is, not only can the visual calibration identification block 152 be directly embedded in the bottom of the material picking and placing adsorption plate 132, but the visual calibration identification block 152 can also be embedded on the identification block base block, and then the identification block base block is embedded on the material picking and placing adsorption plate 132.

[0112] Furthermore, in this embodiment, the visual calibration mark block 152 can be configured as a rectangular block, and the visual calibration mark pattern 154 can be configured as a cross-shaped mark pattern. Furthermore, the bottom surface of the rectangular block can be configured as a black or white bottom surface, and the cross-shaped mark pattern can be configured as a white or black pattern. When the bottom surface of the visual calibration mark block 152 is configured as a black bottom surface, the visual calibration mark pattern 154 can be configured as a white pattern; and when the bottom surface of the visual calibration mark block 152 is configured as a white bottom surface, the visual calibration mark pattern 154 can be configured as a black pattern. This allows the color of the visual calibration mark pattern 154 to form a sharp contrast with the color of the bottom surface of the visual calibration mark block 152, allowing the two to be clearly distinguished, further facilitating the visual camera module of the camera visual calibration mechanism 140 to take photos and accurately identify the visual calibration mark pattern 154.

[0113] Moreover, in this embodiment, the visual calibration mark block 152 can be set as an aluminum alloy block, or other metal block; in addition, a groove-shaped graphic pattern can be engraved and milled on the bottom surface of the visual calibration mark block 152 by a CNC machine tool or other processing equipment to form a recessed visual calibration mark pattern 154. Moreover, the bottom surface of the visual calibration mark block 152 can be processed by black anodizing to form the bottom surface of the visual calibration mark block 152 black; and the visual calibration mark pattern 154 can be directly engraved and milled into a white pattern on the bottom surface of the aluminum alloy block, or other methods (such as spray painting) can be used to form a white pattern. By setting the bottom surface of the visual calibration mark block 152 to black and the visual calibration mark pattern 154 to white (such as silver white), the boundary of the visual calibration mark pattern 154 can be clearly distinguished between black and white.

[0114] Furthermore, in step S222 , obtaining a visual recognition calibration image of the visual calibration mark structure 150 having the visual calibration mark pattern 154 may further include:

[0115] Acquire a visual recognition calibration image of the bottom surface of the material loading and unloading plate of the material loading and unloading structure 130;

[0116] The visual recognition calibration image has a visual calibration mark structure 150 that is a rectangular block and is provided on the bottom surface of the pick-up and unloading plate, and the bottom surface of the visual calibration mark structure 150 has a visual calibration mark pattern 154 that is a cross-shaped mark pattern.

[0117] In addition, in step S300, obtaining the correction calibration position of the material picking and placing structure 130 based on the obtained visual recognition calibration images of the plurality of material picking and placing structures 130 may further include the following steps:

[0118] S310 : Acquire multiple mark center positions of the visual calibration mark structure 150 according to the obtained multiple visual recognition calibration images.

[0119] That is, based on the multiple first visual recognition calibration images obtained from the X-axis verification of the material picking and placing structure 130, the multiple first identification center positions of the visual calibration identification structure 150 can be obtained; similarly, based on the multiple second visual recognition calibration images obtained from the Y-axis verification, the multiple second identification center positions of the visual calibration identification structure 150 can be obtained; similarly, based on the multiple third visual recognition calibration images obtained from the Z-axis verification, the multiple third identification center positions of the visual calibration identification structure 150 can be obtained; similarly, based on the multiple fourth visual recognition calibration images obtained from the XY-axis verification, the multiple fourth identification center positions of the visual calibration identification structure 150 can be obtained; similarly, based on the multiple fifth visual recognition calibration images obtained from the XYX-axis verification, the multiple fifth identification center positions of the visual calibration identification structure 150 can be obtained.

[0120] S320 , obtaining a deviation correction calibration position of the material taking and placing structure 130 according to the preset calibration center position and the obtained multiple identification center positions.

[0121] That is, the first deviation correction calibration position of the material picking and placing structure 130 in the X-axis direction can be obtained according to the first preset calibration center position of the material picking and placing structure 130 in the X-axis direction and the obtained multiple first identification center positions; similarly, the second deviation correction calibration position of the material picking and placing structure 130 in the Y-axis direction can be obtained according to the second preset calibration center position of the material picking and placing structure 130 in the Y-axis direction and the obtained multiple second identification center positions; similarly, the third deviation correction calibration position of the material picking and placing structure 130 in the Z-axis direction can be obtained according to the third preset calibration center position of the material picking and placing structure 130 in the Z-axis direction and the obtained multiple third identification center positions; similarly, the fourth deviation correction calibration position of the material picking and placing structure 130 in the XY-axis direction can be obtained according to the fourth preset calibration center position of the material picking and placing structure 130 in the XY-axis direction and the obtained multiple fourth identification center positions; similarly, the fifth deviation correction calibration position of the material picking and placing structure 130 in the XYZ-axis direction can be obtained according to the fifth preset calibration center position of the material picking and placing structure 130 in the XYZ-axis direction and the obtained multiple fifth identification center positions.

[0122] Furthermore, in step S310, obtaining multiple identification center positions of the visual calibration identification structure 150 according to the obtained multiple visual recognition calibration images may further include the following steps:

[0123] S312 , setting the obtained visual calibration mark pattern 154 as a visual recognition calibration image of a cross-shaped mark pattern, and obtaining the mark center position of the cross-shaped mark pattern on the visual recognition calibration image by a nine-point calibration algorithm.

[0124] The nine-point calibration algorithm is mainly used to establish the conversion relationship between the camera pixel coordinates and the manipulator world coordinates. The conversion matrix is ​​directly calculated through the known image coordinate values ​​of the nine points and the corresponding mechanical coordinate values. The core principle is least squares fitting, which is applicable to affine transformations on 2D planes. Specifically, nine intersection points can be selected on the cross-shaped identification pattern of the visual recognition calibration image (usually by identifying the intersection points on the edge line of the black and white boundary between the cross-shaped identification pattern and the visual calibration identification block 152), and the corresponding mechanical coordinate values ​​are recorded. Using the collected image coordinate values ​​and mechanical coordinate values, a 3*3 conversion matrix is ​​calculated by the least squares method. After obtaining the conversion matrix, it can be used to replace the image coordinate values ​​with new mechanical coordinate values. Through the new mechanical coordinate values, the identification center position of the cross-shaped identification pattern can be obtained (which can also be regarded as the identification center position of the material taking and discharging structure 130).

[0125] S314 , repeat the above steps (ie, step S312 ) to obtain the center position of each cross-shaped marking pattern in the plurality of visual recognition calibration images.

[0126] Furthermore, in the above step S320, the correction calibration position of the material taking and unloading structure 130 is obtained according to the preset calibration center position and the obtained multiple identification center positions, which may further include the following steps:

[0127] S322: Compare the preset calibration center position and the obtained multiple identification center positions with the preset calibration center position to obtain multiple center position deviations.

[0128] The deviation value between each marked center position and the preset calibration center position of the material taking and placing structure 130 in the X-axis direction, the Y-axis direction, the Z-axis direction, the XY-axis direction, or the XYZ-axis direction can be obtained.

[0129] S324 , obtaining a deviation correction calibration position of the calibrated material taking and placing structure 130 according to the preset calibration center position and the obtained multiple center position deviations.

[0130] That is, based on multiple center position deviations, the corresponding deviation average of the material picking and placing structure 130 in the X-axis direction, or the Y-axis direction, or the Z-axis direction, or the XY-axis direction, or the XYZ-axis direction can be obtained, and the deviation average can be added to the preset calibration center position to obtain the correction calibration position of the material picking and placing structure 130 in the X-axis direction, or the Y-axis direction, or the Z-axis direction, or the XY-axis direction, or the XYZ-axis direction.

[0131] In addition, in step S400, when it is confirmed that the material picking and placing structure 130 is offset, setting the correction calibration position as the new coordinate origin position of the material picking and placing structure 130 may further include the following steps:

[0132] S410 , verifying the center positions of the multiple markers using a CPK verification method according to a preset calibration center position, the center positions of the multiple markers, and the obtained center position deviations.

[0133] That is, after obtaining the deviation value between each marked center position of the material handling structure 130 in the X-axis, Y-axis, Z-axis, XY-axis, or XYZ-axis and the preset calibration center position, each deviation value can be verified using the CPK (Complex Process Capability Index) verification method to determine whether each deviation value is within the accuracy requirement (preset accuracy requirement). If so, the deviation value is determined to meet the requirement; if not, the deviation value is determined to not meet the requirement.

[0134] S420: When it is detected that the CPK values ​​of the center positions of the multiple markings do not meet the preset accuracy requirements, it is determined that the material taking and placing structure 130 is offset.

[0135] That is, when it is judged that the deviation value between the center position of the mark and the preset calibration center position meets the requirements, it can be proved that the material taking and putting structure 130 has not shifted and no position correction is required; and when it is judged that the deviation value between the center position of the mark and the preset calibration center position does not meet the requirements, it can be proved that the material taking and putting structure 130 has shifted and position correction is required.

[0136] S430 , when it is confirmed that the material taking and placing structure 130 is offset, setting the deviation correction calibration position as the new coordinate origin position of the material taking and placing structure 130 .

[0137] The proposed method of using vision to guide robot deviation correction improves production efficiency. Automatic visual correction reduces issues such as product scratches and vacuum alarms caused by axis precision deviation, significantly increasing production speed and reducing operation time, thereby increasing output. It also reduces human involvement: by running automated programs, manual intervention is reduced, ensuring a more continuous and stable production process and avoiding production interruptions caused by human factors. It also improves equipment production quality: automated correction reduces human error, thereby improving production quality. Furthermore, regular automatic detection and correction functions provide real-time monitoring of equipment status, enabling timely detection and correction of problems.

[0138] In addition, in other embodiments, Figure 5 As shown, the present invention also proposes a system 1000 that uses a visual method to guide a robot to correct deviation, which is applied to a robot device 100. Figures 2 to 4 As shown, the manipulator device 100 may include a manipulator truss 110, a manipulator drive mechanism 120 disposed on the manipulator truss 110, a material handling structure 130 disposed on the manipulator drive mechanism 120, a camera vision calibration mechanism 140 disposed on the manipulator truss 110, and a vision calibration mark structure 150 disposed on the material handling structure 130 and correspondingly located above the camera vision calibration mechanism 140. The manipulator drive mechanism 120 is used to drive the material handling structure 130 to move in the horizontal and vertical directions, so that the material handling structure 130 can grasp, move, and place materials; and the camera vision calibration mechanism 140 can photograph and detect the bottom surface of the material handling structure 130 and the vision calibration mark structure 150 located thereon from below.

[0139] Specifically, if Figure 5 As shown, the system 1000 for guiding the robot to correct deviation using a visual method may specifically include:

[0140] The correction starting module 1002 is used to issue a correction calibration instruction to the material picking and placing structure 130 when the real-time material picking and placing times of the material picking and placing structure 130 reaches a preset number of times or / and the real-time working time reaches a preset time;

[0141] The deflection correction and calibration identification module 1004 is used to control the manipulator drive mechanism 120 to drive the pick-up and place structure 130 to repeatedly move to directly above the camera vision calibration mechanism 140 in response to the deflection correction and calibration instruction, and control the camera vision calibration mechanism 140 to visually identify the pick-up and place structure 130 having the vision calibration mark structure 150, thereby obtaining multiple vision recognition calibration images;

[0142] The correction and calibration processing module 1006 is used to obtain the correction and calibration position of the material picking and placing structure 130 based on the visual recognition calibration images of the plurality of material picking and placing structures 130;

[0143] The correction setting module 1008 is used to set the correction calibration position as the new coordinate origin position of the picking and placing structure 130 when it is confirmed that the picking and placing structure 130 is offset, thereby completing the correction calibration operation of the picking and placing structure 130.

[0144] The system 1000 for guiding the robot to correct deviation by using a visual method described in this embodiment corresponds to the method for guiding the robot to correct deviation by using a visual method described above. The functions of each module in the system 1000 for guiding the robot to correct deviation by using a visual method in this embodiment are described in detail in the corresponding method embodiment and will not be repeated here. Figures 2 to 4 As shown, the specific structure of the robot device 100 is also described in detail in the above-mentioned method of using visual means to guide the robot to correct the deviation, and will not be repeated here.

[0145] In addition, in other embodiments, Figure 6 As shown, the present invention further proposes a manipulator device 100, comprising a manipulator truss 110, a manipulator body disposed on the manipulator truss 110, a correction and calibration mechanism disposed on the manipulator truss 110 and the manipulator body, and a controller 160 connected to the manipulator body and the correction and calibration mechanism. The manipulator body may include a manipulator drive mechanism 120 disposed on the manipulator truss 110, and a material pick-up and placement structure 130 disposed on the manipulator drive mechanism 120; the correction and calibration mechanism may include a camera vision calibration mechanism 140 disposed on the manipulator truss 110, and a visual calibration marking structure 150 disposed on the material pick-up and placement structure 130 and correspondingly located above the camera vision calibration mechanism 140; the controller 160 is connected to the manipulator drive mechanism 120, the material pick-up and placement structure 130, and the camera vision calibration mechanism 140.

[0146] Furthermore, the controller 160 of the manipulator device 100 can be used to implement the above-mentioned method of using visual means to guide the manipulator to correct its deviation. Similarly, in this embodiment, the controller 160 can be used to implement each step in the above-mentioned method of using visual means to guide the manipulator to correct its deviation. The specific implementation method can refer to the specific content of the above-mentioned method of using visual means to guide the manipulator to correct its deviation, which will not be repeated here. Moreover, if Figures 2 to 4 As shown, the specific structure of the robot device 100 is also described in detail in the above-mentioned method of using visual means to guide the robot to correct the deviation, and will not be repeated here.

[0147] In addition, in other embodiments, the present invention proposes a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement all or part of the method steps of the method of using visual means to guide the robot to correct the deviation as described above.

[0148] The present invention implements all or part of the process in the above method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0149] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program running on the processor, and when the processor executes the computer program, all or part of the method steps in the above method are implemented.

[0150] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of a computer device, connecting all parts of the entire computer device using various interfaces and lines.

[0151] The memory can be used to store computer programs and / or models. The processor implements various functions of the computer device by running or executing the computer programs and / or models stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, video data, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0152] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, servers, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage) containing computer-usable program code.

[0153] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), servers, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0156] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for guiding a robot to correct deviation using a visual method, applied to a robot device; the robot device comprises a robot truss, a robot drive mechanism disposed on the robot truss, a material handling structure disposed on the robot drive mechanism, a camera vision calibration mechanism disposed on the robot truss, and a vision calibration mark structure disposed on the material handling structure and correspondingly located above the camera vision calibration mechanism; It is characterized by: The method comprises: When the real-time material taking and putting times of the material taking and putting structure reaches a preset number of material taking and putting times, or / and the real-time working time reaches a preset time, a correction calibration instruction for the material taking and putting structure is issued; In response to the deflection correction and calibration instruction, the robot drive mechanism is controlled to drive the material picking and placing structure to repeatedly move to directly above the camera vision calibration mechanism, and the camera vision calibration mechanism is controlled to perform visual recognition on the material picking and placing structure having the vision calibration mark structure to obtain a plurality of visual recognition calibration images; Obtaining a deviation correction calibration position of the material picking and placing structure based on the obtained plurality of visual recognition calibration images of the material picking and placing structure; When it is confirmed that the material picking and placing structure has deviated, the deviation correction calibration position is set as the new coordinate origin position of the material picking and placing structure to complete the deviation correction calibration operation of the material picking and placing structure.

2. The method for guiding a robot to correct deviation using a visual method according to claim 1, characterized in that: The control manipulator driving mechanism drives the material picking and placing structure to repeatedly move to the top of the camera vision calibration mechanism, and controls the camera vision calibration mechanism to perform visual recognition on the material picking and placing structure having the vision calibration identification structure to obtain multiple vision recognition calibration images, including: Controlling the manipulator drive mechanism to drive the material picking and placing structure to move to the top of the camera vision calibration mechanism in the X-axis direction, Y-axis direction, Z-axis direction, XY-axis direction, and XYZ-axis direction for a preset number of repetitions; Each time the material taking and placing structure moves to the position directly above the camera vision calibration mechanism, the camera vision calibration mechanism is controlled to perform visual recognition on the material taking and placing structure having the vision calibration identification structure, and a plurality of vision recognition calibration images are acquired.

3. The method for guiding a robot to correct deviation using a visual method according to claim 2, characterized in that: The material picking and placing structure includes a material picking and placing mounting seat provided on the manipulator drive mechanism, a material picking and placing adsorption plate provided on the material picking and placing mounting seat, and a plurality of vacuum suction nozzles protruding from the bottom of the material picking and placing adsorption plate; The visual calibration mark structure includes a visual calibration mark block provided at the bottom of the material pickup and placement adsorption plate, the bottom surface of the visual calibration mark block is provided with a visual calibration mark pattern, and the visual calibration mark pattern corresponds to the camera visual calibration mechanism in the upper and lower directions; The controlling the camera vision calibration mechanism to perform visual recognition on the material picking and placing structure having the vision calibration identification structure to obtain a plurality of vision recognition calibration images includes: When the material picking and placing structure moves to the position directly above the camera vision calibration mechanism, the camera vision calibration mechanism is controlled to visually photograph the bottom surface of the material picking and placing adsorption plate of the material picking and placing structure and the visual calibration mark pattern on the bottom surface of the visual calibration mark structure thereon, to obtain a visual recognition calibration image of the visual calibration mark structure having the visual calibration mark pattern; The above steps are repeated multiple times to obtain multiple vision recognition calibration images of the vision calibration mark pattern having the vision calibration mark structure.

4. The method for guiding a robot to correct deviation using a visual method according to claim 3, characterized in that: The visual calibration mark block is set as a rectangular block, and the visual calibration mark pattern is set as a cross-shaped mark pattern; and the bottom surface of the rectangular block is set as a black bottom surface or a white bottom surface, and the cross-shaped mark pattern is set as a white pattern or a black pattern accordingly; The acquiring of the visual recognition calibration image having the visual calibration mark pattern of the visual calibration mark structure includes: Acquire a visual recognition calibration image of the bottom surface of the material picking and placing plate of the material picking and placing structure; The visual recognition calibration image has the visual calibration mark structure which is a rectangular block provided on the bottom surface of the pick-up and unloading plate, and the bottom surface of the visual calibration mark structure has the visual calibration mark pattern which is a cross-shaped mark pattern.

5. The method for guiding a robot to correct deviation using a visual method according to claim 4, characterized in that: The step of obtaining the correction calibration position of the material picking and placing structure based on the obtained visual recognition calibration images of the plurality of material picking and placing structures comprises: Acquiring multiple marker center positions of the visual calibration marker structure according to the obtained multiple visual recognition calibration images; According to the preset calibration center position and the obtained multiple identification center positions, the correction calibration position of the material taking and placing structure is obtained.

6. The method for guiding a robot to correct deviation using a visual method according to claim 5, characterized in that: The step of obtaining a plurality of marker center positions of the visual calibration marker structure according to the obtained plurality of visual recognition calibration images includes: According to the obtained visual recognition calibration image in which the visual calibration mark pattern is set as a cross-shaped mark pattern, obtaining the mark center position of the cross-shaped mark pattern on the visual recognition calibration image by a nine-point calibration algorithm; Repeat the above steps to obtain the center position of each cross-shaped marking pattern in the plurality of visual recognition calibration images.

7. The method for guiding a robot to correct deviation using a visual method according to claim 6, characterized in that: Obtaining the correction calibration position of the material taking and placing structure according to the preset calibration center position and the obtained multiple identification center positions includes: Comparing the preset calibration center position and the obtained multiple identification center positions with the preset calibration center position to obtain multiple center position deviations; According to the preset calibration center position and the obtained multiple center position deviations, the calibrated correction calibration position of the material taking and placing structure is obtained.

8. The method for guiding a robot to correct deviation using a visual method according to claim 7, characterized in that: When it is confirmed that the material picking and placing structure is offset, setting the correction calibration position as the new coordinate origin position of the material picking and placing structure includes: Verifying the multiple marker center positions using a CPK verification method according to the preset calibration center position, the multiple marker center positions, and the obtained multiple center position deviations; When it is detected that the CPK values ​​of the center positions of the multiple markers do not meet the preset accuracy requirements, it is determined that the material taking and placing structure is offset; When it is confirmed that the material taking and placing structure is offset, the deviation correction calibration position is set as the new coordinate origin position of the material taking and placing structure.

9. A system for visually guiding a robot to correct deviation, applied to a robot device; the robot device comprises a robot truss, a robot drive mechanism mounted on the robot truss, a material handling structure mounted on the robot drive mechanism, a camera vision calibration mechanism mounted on the robot truss, and a vision calibration marker structure mounted on the material handling structure and correspondingly located above the camera vision calibration mechanism. It is characterized by: The system comprises: A deviation correction starting module is used to issue a deviation correction calibration instruction to the material picking and unloading structure when the real-time material picking and unloading times of the material picking and unloading structure reach a preset number of times, or / and the real-time working time reaches a preset time; a deflection correction and calibration identification module, configured to control the manipulator drive mechanism to drive the pick-up and put-down structure to repeatedly move to directly above the camera vision calibration mechanism in response to the deflection correction and calibration instruction, and control the camera vision calibration mechanism to perform visual recognition on the pick-up and put-down structure having the vision calibration identification structure, thereby acquiring a plurality of vision recognition calibration images; a deflection correction and calibration processing module, configured to obtain a deflection correction and calibration position of the material picking and placing structure based on the obtained visual recognition calibration images of the plurality of material picking and placing structures; The correction setting module is used to set the correction calibration position as the new coordinate origin position of the material picking and placing structure when it is confirmed that the material picking and placing structure is offset, so as to complete the correction calibration operation of the material picking and placing structure.

10. A manipulator device, characterized in that: include: Manipulator truss; The manipulator body comprises a manipulator drive mechanism provided on the manipulator truss, and a material taking and placing structure provided on the manipulator drive mechanism; A correction and calibration mechanism, comprising a camera vision calibration mechanism provided on the manipulator truss, and a vision calibration marking structure provided on the material taking and placing structure and correspondingly located above the camera vision calibration mechanism; and A controller connected to the manipulator body and the deviation correction and calibration mechanism; Wherein, the controller is used to implement the method of using visual means to guide the robot to correct the deviation as described in any one of claims 1-8.

Images