Robot target grabbing method and system

Abstract

The embodiment of the invention discloses a robot target grabbing method and system, and relates to the technical field of automation, and the method comprises the steps: photographing each target through a camera to obtain a plurality of images, taking the images corresponding to different postures as template images, constructing a conversion matrix between a robot base and the camera according to the images, and taking the conversion matrix as the template image; a plurality of edge point direction vectors are obtained according to the template images and the conversion matrix, a plurality of target coordinates are obtained through calculation according to the edge point direction vectors, the grabbing position of a mechanical arm of the robot is obtained, and the rotation offset and the plane offset of the mechanical arm are obtained through calculation according to the conversion matrix, the target coordinates and the grabbing position; and multiple template instructions corresponding to the template images are constructed according to the rotation offset and the plane offset of the mechanical arm, and the targets are grabbed according to the template instructions. By combining a visual system and mechanical arm control, efficient and accurate target grabbing of the robot is achieved, and the problem that in the prior art, the recognition degree and the grabbing efficiency are low is solved.

Description

Technical Field

[0001] The present invention relates to the field of automation technology, and in particular to a robot target grasping method and system. Background Art

[0002] With the advent of the Industrial 4.0 era, industrial robots and machine vision technology have developed rapidly and are playing an increasingly important role in the field of industrial inspection and automation. This technology simulates the human visual system to achieve accurate inspection of product size and surface defects, significantly improving production efficiency and flexibility. Especially in the 3C and lithium battery industries, robot vision technology has become an indispensable part.

[0003] However, traditional robotics technology is unable to cope with the technical requirements of the lithium battery industry, such as dexterous operation, autonomous navigation, environmental perception, human-machine interaction and safety. In particular, during the cell loading process, the accuracy of recognition of soft-pack batteries is low due to possible deviations in the placement of soft-pack batteries or in the case of stacking, resulting in low production efficiency and increased costs.

[0004] At present, although machine vision technology has been applied to automatic sorting systems and the guidance and control of robotic arms, it still faces many deficiencies and difficulties in the specific application scenario of battery cell loading. For example, how to quickly and accurately identify and grab stacked or position-deviated soft-pack batteries, how to achieve precise control of the gripping posture of the robotic arm, and how to improve the intelligence and reliability of the entire system.

[0005] Therefore, there is an urgent need for a robot target grasping method that can accurately obtain the target position and posture information, thereby guiding the multi-axis robot to grasp the target accurately and efficiently, and can adapt to different production environments and grasping targets. Summary of the invention

[0006] Each embodiment of the present invention provides a robot target grasping method to solve the problem that the prior art has low recognition and grasping efficiency and cannot adapt to different production environments and grasping targets. The technical solution is as follows:

[0007] According to one aspect of the present invention, a robot target grasping method includes: taking pictures of each target with a camera to obtain multiple images, using the images corresponding to different postures as template images, and constructing a transformation matrix between a robot base and a camera according to the images; obtaining multiple edge point direction vectors according to each of the template images and the transformation matrix, and calculating multiple target coordinates according to the edge point direction vectors; obtaining a grasping position of a robot manipulator, and calculating a manipulator arm rotation offset and a plane offset according to the transformation matrix, the target coordinates and the grasping position; constructing multiple template instructions corresponding to each of the template images according to the manipulator arm rotation offset and the plane offset, and grasping each of the targets according to the template instructions.

[0008] In one embodiment, constructing a transformation matrix between the robot base and the camera according to the image is achieved by the following steps: obtaining a first transformation matrix between the calibration plate and the camera by taking an image of the calibration plate, and reading a second transformation matrix between the robotic arm and the robot base through the robot's teaching pendant; obtaining a third transformation matrix between the calibration plate and the robot base, and a fourth transformation matrix between the camera and the robotic arm by calculating the first transformation matrix and the second transformation matrix of each of the images; and obtaining a transformation matrix between the robot base and the camera according to the second transformation matrix and the fourth transformation matrix.

[0009] In one embodiment, obtaining the transformation matrix between the robot base and the camera according to the second transformation matrix and the fourth transformation matrix is ​​achieved by the following steps: multiplying the negative first power of the second transformation matrix by the fourth transformation matrix to obtain the transformation matrix between the robot base and the camera.

[0010] In one of the embodiments, grasping each of the targets according to the template instructions also includes the following steps: if the target has no corresponding template instructions, acquiring an image of the target in real time, calculating corresponding grasping position information according to the transformation matrix, and controlling the robot to grasp the target according to the grasping position information.

[0011] According to one aspect of the present invention, a robot target grasping system includes a visual system, a robot, a control system and a monitoring system; the robot is used to grasp the target according to the control instructions of the control system; the visual system is used to identify the position of the target; the control system is used to generate control instructions according to the position information of the visual system; the monitoring system is used to monitor the status of the system in real time and adjust parameters.

[0012] In one embodiment, the robot includes a four-axis robotic arm equipped with a grasping mechanism for grasping a target.

[0013] In one of the embodiments, the visual system includes a camera module; the camera module is located above the transmission device and is used to obtain a posture image of the target and an image of a calibration plate.

[0014] In one of the embodiments, the control system is used to calculate the rotation offset and plane offset of the robotic arm and perform posture compensation on the robotic arm.

[0015] In one embodiment, the visual system includes a computing module; the computing module calculates the pose information of the target according to the pose image through image processing and pattern recognition algorithms, and calculates the transformation matrix according to the calibration plate image.

[0016] In one of the embodiments, the monitoring system comprises a visualization module and a touch screen; the visualization module is used to view and transmit the system's operating status, operation records and fault alarm information in real time; the touch screen is used to adjust parameters.

[0017] The beneficial effects brought by the technical solution provided by the present invention are:

[0018] In the above technical scheme, the present invention first obtains multiple images by photographing each target through a camera, uses images corresponding to different postures as template images, constructs a transformation matrix between a robot base and a camera according to the images, obtains multiple edge point direction vectors according to each template image and the transformation matrix, calculates multiple target coordinates according to the edge point direction vectors, obtains the grasping position of the robot's mechanical arm, calculates the mechanical arm rotation offset and plane offset according to the transformation matrix, the target coordinates and the grasping position, constructs multiple template instructions corresponding to each template image according to the mechanical arm rotation offset and the plane offset, grasps each target according to the template instructions, and does not It not only improves the accuracy of target recognition, but also significantly enhances the adaptability and flexibility of the system, enabling it to cope with complex and changeable working environments. Template instructions are constructed based on the calculated rotational offset and plane offset to guide the robotic arm to perform precise grasping. It not only realizes fully automated operation, but also greatly simplifies the operation process and reduces dependence on manual labor. By combining the visual system, precise calculation and robotic arm control, the robot can grasp the target object efficiently and accurately, significantly improving production efficiency and product quality, thereby effectively solving the problems of low recognition and grasping efficiency of existing technologies and inability to adapt to different production environments and grasping targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a robot target grasping method according to an exemplary embodiment;

[0021] Figure 2 It is a block diagram of a robot target grasping system according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0023] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0024] The present invention provides a robot target grasping method, which realizes efficient and accurate grasping of target objects by the robot by combining a visual system, precise calculation and mechanical arm control, significantly improves production efficiency and product quality, solves the problems of low recognition and grasping efficiency in the prior art, and inability to adapt to different production environments and grasping targets. The robot target grasping method is suitable for a robot target grasping device, which can be an electronic device. The robot target grasping method in the embodiment of the present invention can be applied to a variety of scenarios, such as robot target grasping, etc.

[0025] See also Figure 1 , an embodiment of the present invention provides a robot target grasping method, which is applicable to electronic equipment.

[0026] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0027] like Figure 1As shown, the method may include the following steps:

[0028] Step 110, taking photos of each target by a camera to obtain multiple images, using the images corresponding to different postures as template images, and constructing a transformation matrix between the robot base and the camera based on the images.

[0029] In one possible implementation, the first transformation matrix between the calibration plate and the camera is obtained by taking an image of the calibration plate, the second transformation matrix between the robotic arm and the robot base is read out through the robot's teaching pendant, the third transformation matrix between the calibration plate and the robot base and the fourth transformation matrix between the camera and the robotic arm are obtained by calculating the first transformation matrix and the second transformation matrix of each image, and the transformation matrix between the robot base and the camera is obtained based on the second transformation matrix and the fourth transformation matrix.

[0030] In a possible implementation, the transformation matrix between the robot base and the camera is obtained by multiplying the negative first power of the second transformation matrix by the fourth transformation matrix.

[0031] Step 130 , obtaining a plurality of edge point direction vectors according to each template image and the transformation matrix, and obtaining a plurality of target coordinates according to the edge point direction vectors.

[0032] Step 150, obtaining the grasping position of the robot arm, and calculating the rotation offset and plane offset of the robot arm according to the transformation matrix, the target coordinates and the grasping position.

[0033] Step 170 , construct multiple template instructions corresponding to each template image according to the robot arm rotation offset and the plane offset, and grab each target according to the template instructions.

[0034] In a possible implementation, if the target does not have a corresponding template instruction, the image of the target is acquired in real time, the grasping position information corresponding to the target is calculated according to the transformation matrix, and the robot is controlled to grasp the target according to the grasping position information.

[0035] Through the above process, the present invention first obtains multiple images by photographing each target through a camera, uses images corresponding to different postures as template images, constructs a transformation matrix between a robot base and a camera according to the images, obtains multiple edge point direction vectors according to each template image and the transformation matrix, calculates multiple target coordinates according to the edge point direction vectors, obtains the grasping position of the robot manipulator, calculates the manipulator rotation offset and plane offset according to the transformation matrix, the target coordinates and the grasping position, constructs multiple template instructions corresponding to each template image according to the manipulator rotation offset and the plane offset, grasps each target according to the template instructions, and does not It not only improves the accuracy of target recognition, but also significantly enhances the adaptability and flexibility of the system, enabling it to cope with complex and changeable working environments. Template instructions are constructed based on the calculated rotational offset and plane offset to guide the robotic arm to perform precise grasping. It not only realizes fully automated operation, but also greatly simplifies the operation process and reduces dependence on manual labor. By combining the visual system, precise calculation and robotic arm control, the robot can grasp the target object efficiently and accurately, significantly improving production efficiency and product quality, thereby effectively solving the problems of low recognition and grasping efficiency of existing technologies and inability to adapt to different production environments and grasping targets.

[0036] In one application scenario, the target of the grab is a soft-pack battery. The system consists of two parts: the software system and the hardware system. The hardware system consists of a mechanical structure and various components, and the software system includes a control system, a visual system, a robot and a touch screen. The soft-pack battery is gripped by an industrial robot, and the control system composed of a PLC controller combines with a visual system to identify the soft-pack battery, and is operated and monitored in real time through the touch screen.

[0037] On the soft-pack battery production line, the placement position on the pull belt of the battery cell feeder is deviated or the accurate recognition of the soft-pack battery is low when stacked. Therefore, the embodiment of the present invention uses a deep learning algorithm to extract the various postures of the soft-pack battery to be grasped, and then performs visual analysis calculations through the visual system to quickly find the position of the soft-pack battery and use the camera to perform three-dimensional reconstruction to obtain the three-dimensional coordinates and posture data of the soft-pack battery on the pull belt, thereby guiding the industrial robot to grasp it.

[0038] In the above process, the embodiment of the present invention ensures the speed and accuracy of soft-pack battery search on the one hand, and uses a four-axis robot to replace manual labor on the other hand, thereby improving the efficiency of soft-pack battery identification and grasping, and meeting the requirements of future industrial robot production lines for intelligence, high efficiency and reliability.

[0039] Specifically, a camera is selected to take pictures of the soft-pack batteries, and the visual system processes different images to create templates. Then, according to the templates, instructions are sent to the robotic arm for the next step for the soft-pack batteries in different postures. After receiving the information instructions, the robot grabs the soft-pack batteries in different postures on the drawstring, completing the entire pick-and-place process.

[0040] Among them, a four-axis robotic arm is used as a grasping mechanism, and a visual system is installed above the drawstring to obtain the posture information of the soft-pack battery. The computing device is used to calculate the offset of the soft-pack battery to perform posture compensation on the grasping action of the robotic arm, so that the grasping mechanism can accurately grasp the soft-pack battery.

[0041] Specifically, the standard coordinate value (X, Y, θ1) of the battery is obtained, the actual coordinate value (X1, Y1, θ2) of the soft-pack battery is calculated according to the soft-pack battery image, the standard value (X0, Y0) corresponding to the standard grasping position where the rotation center of the robot arm is located is obtained, and the robot arm center rotation offset β = θ2-θ1 and the plane offset are calculated.

[0042] The offset is calculated as follows:

[0043] Offset_X = (X1-X0)cosβ-(Y1-Y0)sinβ+X0-X;

[0044] Offset_Y=(X1-X0)sinβ-(Y1-Y0)cosβ+Y0-Y.

[0045] Furthermore, the rotation offset and translation offset are sent to the visual system for processing. Specifically, four transformation matrices are obtained: the transformation matrix from the four-axis robot base coordinate system to the calibration plate coordinate system Transformation matrix from calibration plate coordinate system to camera coordinate system The transformation matrix from the four-axis robot end coordinate system to the four-axis industrial robot base coordinate system And the transformation matrix from the camera coordinate system to the base coordinates of the four-axis industrial robot

[0046] The calculation equation is applied to each image captured by the camera:

[0047] in, Directly solve by shooting the calibration plate image, which is composed of the camera's internal parameters; Directly read out through the teaching pendant of the four-axis industrial robot: is an unknown quantity, but the calibration plate does not move. are all the same, is a fixed value. The above calculation equation is applied to each image, and then the calculation results of multiple images can be used to solve and The value of .

[0048] Furthermore, through the equation Transformation matrix from the base coordinate system of a four-axis industrial robot to the camera coordinate system

[0049] Through the above process, the embodiment of the present invention realizes full automation of the battery cell loading production process through the three-in-one PLC, vision system and robot. The system is flexible in operation, and the system settings can be completed on the human-machine interface. The operator can monitor the entire operating status very well. The installation position has high accuracy. The camera plane pixels are converted into workpiece coordinates. It is only necessary to calculate the offset and move and position the robot arm to the grasping position according to the offset soft-pack battery coordinate system. There is no need to calculate the absolute coordinate value of the robot movement, thereby simplifying the calculation process, extracting the soft-pack battery position point from the image, and solving the problem of large error and low grasping efficiency during traditional robot plane grasping.

[0050] In another application scenario, Figure 2 A robotic object grasping system 800 is shown.

[0051] The robot target grasping system 800 includes but is not limited to: a vision system 810 , a robot 820 , a control system 830 and a monitoring system 840 .

[0052] Specifically, first start all components of the robot target grasping system, including the visual system, robot, control system, and monitoring system. Check and ensure that all hardware devices are connected properly and the software system is running stably. On the monitoring system interface, set and initialize system parameters, including robot movement speed, visual system recognition accuracy, etc.

[0053] Furthermore, the target object is photographed in high definition through the camera module in the visual system to obtain image information of the target object, and the photographed image is preprocessed, including denoising, contrast enhancement and other operations to improve image quality, providing a clear and accurate image basis for subsequent position recognition. The computing module in the visual system is used to extract and match features of the preprocessed image to determine the exact position of the target object in the image.

[0054] Furthermore, through specific algorithms and mathematical models, the position information in the image is converted into three-dimensional coordinates in the robot coordinate system. The control system generates corresponding control instructions based on the position information provided by the visual system. These instructions include parameters such as the robot's motion trajectory and grasping force. After receiving the control instructions, the robot begins to move toward the target object according to the specified trajectory and force. When the robot reaches the target position, it starts the grasping mechanism to grasp the target object.

[0055] During the entire grasping process, the monitoring system monitors the system status and the location information of the target object in real time. If an abnormal situation is found or the parameters need to be adjusted, the monitoring system will issue an alarm or prompt in time, and the operator can make corresponding adjustments according to the actual situation.

[0056] Finally, after the robot completes the grasping task, the control system will issue a reset command to restore the robot and the visual system to their initial state in order to carry out the next grasping task. According to the needs of the actual application scenario, the grasped target objects are subsequently processed, such as classification, storage, transportation, etc.

[0057] Through the above steps, the application scenario implementation process of the robot target grasping system was successfully completed. The system not only improves the accuracy and efficiency of the grasping task, but also realizes real-time monitoring and adjustment of the entire grasping process, ensuring the stability and reliability of the system.

[0058] It should be noted that the robot target grasping provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the robot target grasping system will be divided into different functional modules to complete all or part of the functions described above.

[0059] In addition, the robot target grasping system and the robot target grasping method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0060] Compared with the related art, the present invention has the following beneficial effects:

[0061] 1. The present invention firstly obtains a plurality of images by photographing each target through a camera, uses the images corresponding to different postures as template images, constructs a transformation matrix between a robot base and a camera according to the images, obtains a plurality of edge point direction vectors according to each template image and the transformation matrix, calculates a plurality of target coordinates according to the edge point direction vectors, obtains the grasping position of the robot manipulator, calculates the rotation angle and the plane offset of the manipulator according to the transformation matrix, the target coordinates and the grasping position, constructs a plurality of template instructions corresponding to each template image according to the rotation angle and the plane offset of the manipulator, and grasps each target according to the template instructions, which not only improves It not only improves the accuracy of target recognition, but also significantly enhances the adaptability and flexibility of the system, enabling it to cope with complex and changeable working environments. It builds template instructions based on the calculated rotation angle and plane offset to guide the robotic arm to perform precise grasping. It not only realizes fully automated operation, but also greatly simplifies the operation process and reduces dependence on manual labor. By combining the visual system, precise calculation and robotic arm control, the robot can grasp the target object efficiently and accurately, significantly improving production efficiency and product quality, thereby effectively solving the problems of low recognition and grasping efficiency of existing technologies and inability to adapt to different production environments and grasping targets.

[0062] 2. The present invention can realize fully automated production: by applying PLC, visual system and robot three-in-one, the battery core feeding production process is fully automated. The system is flexible and easy to operate. Both manual and automatic system settings can be completed on the human-machine interface, and the operator can monitor the entire operating status well.

[0063] 3. The present invention can achieve high installation position accuracy: the system uses a visual system to achieve precise grasping, with high installation position accuracy, which improves production efficiency and product quality. The camera plane pixels are converted into coordinates in the workpiece coordinate system, and the calibration method is simple and convenient. This technical solution only needs to calculate the offset, and then move and position the robot arm to the grasping position of the standard soft-pack battery according to the offset soft-pack battery coordinate system. There is no need to calculate the absolute coordinate value of the robot movement, thus simplifying the calculation process.

[0064] 4. The present invention improves the efficiency and accuracy of grasping: through image processing, pattern recognition and other algorithms, the position points of soft-pack batteries are extracted from the image, which solves the problems of large errors and low grasping efficiency when the traditional robot is grasping on the plane. This system meets the requirements of intelligent, efficient and reliable industrial robot production lines in the future, and improves the overall performance of the production line. The operator can monitor the entire operating status in real time through the human-machine interface and modify relevant parameters, which enhances the controllability and stability of the system.

[0065] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0066] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A robot target grasping method, characterized in that: The method comprises: A plurality of images are obtained by taking photos of each target through a camera, and the images corresponding to different postures are used as template images, and a conversion matrix between the robot base and the camera is constructed according to the images; Obtain a plurality of edge point direction vectors according to each of the template images and the transformation matrix, and obtain a plurality of target coordinates according to the edge point direction vectors; Obtaining a grasping position of a robot arm, and calculating a rotation offset and a plane offset of the robot arm according to the transformation matrix, the target coordinates and the grasping position; A plurality of template instructions corresponding to each of the template images are constructed according to the rotation offset and the plane offset of the robot arm, and each of the targets is captured according to the template instructions.

2. The robot target grasping method according to claim 1, characterized in that: The step of constructing a transformation matrix between the robot base and the camera according to the image comprises: A first conversion matrix between the calibration plate and the camera is obtained by shooting an image of the calibration plate, and a second conversion matrix between the robot arm and the robot base is read out through the robot's teaching pendant; Obtaining a third transformation matrix between the calibration plate and the robot base, and a fourth transformation matrix between the camera and the robotic arm by calculating the first transformation matrix and the second transformation matrix of each of the images; A transformation matrix between the robot base and the camera is obtained according to the second transformation matrix and the fourth transformation matrix.

3. The robot target grasping method according to claim 2, characterized in that: The step of obtaining a conversion matrix between the robot base and the camera according to the second conversion matrix and the fourth conversion matrix includes: The negative first power of the second transformation matrix is ​​multiplied by the fourth transformation matrix to obtain a transformation matrix between the robot base and the camera.

4. The robot target grasping method according to claim 1, characterized in that: The capturing of each of the targets according to the template instruction further includes: If the target has no corresponding template instruction, the image of the target is acquired in real time, the corresponding grasping position information is calculated according to the conversion matrix, and the robot is controlled to grasp the target according to the grasping position information.

5. A robot target grasping system, comprising a visual system, a robot, a control system and a monitoring system; the robot is used to grasp the target according to the control instructions of the control system; the visual system is used to identify the position of the target; the control system is used to generate control instructions according to the position information of the visual system; the monitoring system is used to monitor the system status in real time and adjust parameters.

6. The robot target grasping system according to claim 5, characterized in that: The robot comprises a four-axis mechanical arm, and the four-axis mechanical arm is equipped with a grasping mechanism to grasp a target.

7. The robot target grasping system according to claim 5, characterized in that: The visual system includes a camera module; the camera module is located above the transmission device and is used to obtain a position image and a calibration plate image of the target.

8. The robot target grasping system according to claim 5, characterized in that: The control system is used to calculate the rotation offset and plane offset of the robotic arm and perform posture compensation on the robotic arm.

9. The robot target grasping system according to claim 5, characterized in that: The visual system includes a calculation module; the calculation module calculates the pose information of the target according to the pose image through image processing and pattern recognition algorithm, and calculates the transformation matrix according to the calibration plate image.

10. The robot target grasping system according to claim 5, characterized in that: The monitoring system comprises a visualization module and a touch screen; the visualization module is used to view and transmit the system's operating status, operation records and fault alarm information in real time; the touch screen is used to adjust parameters.

Images