3D visual guidance grabbing method and device applied to silencer product cover

Abstract

The invention discloses a 3D visual guidance grabbing method and device applied to a silencer product cover, and the method comprises the following steps: an upper computer triggers a 3D camera to carry out the image collection of a silencer target, and transmits the image to the upper computer; the upper computer converts the received target image into a target point cloud and carries out denoising processing to obtain a denoised target image; carrying out extraction processing on the target pose of the silencer in the target image after noise reduction to obtain target pose information of the silencer; the target pose information of the silencer and the physical space relation information stored in the upper computer are used for mathematical algorithm conversion processing, and corresponding position information of the robot is obtained; the upper computer generates a target grabbing path planning instruction based on the position information and the silencer target pose information; and the robot executes a grabbing task of the silencer target in the out-of-order based on the out-of-order grabbing planning instruction. The target grabbing reliability can be improved, the production takt is effectively improved, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of robot grasping production technology, and in particular to a 3D vision-guided grasping method and device for a muffler product cover. Background Art

[0002] Existing silencer product covers all use fishing structures or mechanical vibration disks to lift the silencer product covers to the conveyor belt. The fishing mechanism is implemented by using a magnetic suction mechanism to control the pulley by winding the line. The fishing structure has the following disadvantages: the reliability of material collection is not high. When the material is collected by the fishing mechanism, the number of silencer covers collected each time is uncertain; because of the magnetic suction method, the posture of the material absorbed by the magnetic mechanism is uncertain, and there is a certain probability that the silencer cover is absorbed in a vertical posture, resulting in the material falling back to the material collection point during the rising process due to unstable suction; because the posture of the material is uncertain, the difficulty of subsequent material processing is increased; the material collection operation efficiency of the fishing structure method is not high; it requires coordinated processing by multiple mechanisms, and it is easy for materials to be stuck during the production process due to the lack of coordination of various mechanisms; each time a different material is changed for production, the mechanism must be adjusted, delaying production efficiency; it is easy to misjudge the positive and negative.

[0003] The mechanical vibration plate method is to separate the materials through the vibration plate, and control the materials to be finally transferred to the feeding belt one by one. However, the use of mechanical vibration plates has the following disadvantages: due to the large number of product models and different specifications, the equipment needs to frequently switch products during the daily production process, but this feeding method requires employees to adjust the feeding mechanism based on experience and memory to adjust the product, resulting in great instability in production and a high utilization rate, which seriously affects production. Therefore, a 3D vision-guided grasping method for the cover of the muffler product is designed to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a 3D vision-guided grasping method and device for silencer product covers in response to the deficiencies in the prior art. The 3D vision-guided grasping method and device can improve the reliability of target grasping, effectively increase production rhythm, and improve production efficiency.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions.

[0006] A 3D vision-guided grasping method for a muffler product cover includes the following steps:

[0007] Step 1: The host computer triggers the 3D camera to collect images of the silencer target, and transmits the collected target image to the host computer;

[0008] Step 2: The host computer converts the received target image into a 3D point cloud and performs denoising to obtain a denoised target image;

[0009] Step 3, extracting and processing the silencer target position and posture of the silencer target in the denoised target image in the pixel coordinate system to obtain the silencer target position and posture information;

[0010] Step 4, using the target posture information of the muffler and the physical space relationship information stored on the host computer to perform mathematical algorithm conversion processing to obtain the corresponding position information of the robot;

[0011] Step 5, the host computer generates a random grabbing action planning instruction for the robot based on the position information corresponding to the robot and the target posture information of the muffler;

[0012] Step six, the robot executes the task of grasping the silencer target in disorder based on the disordered grasping action planning instruction.

[0013] Among them, the host computer triggers the 3D camera to capture images of the silencer target, and transmits the captured target image to the host computer, the host computer generates a trigger instruction, and sends the trigger instruction to the 3D camera based on the TCP network protocol; the 3D camera triggers image capture of the silencer target based on the trigger instruction; the 3D camera transmits the captured target image to the host computer based on the TCP network protocol.

[0014] Among them, the host computer converts the received target image into a target 3D point cloud, and performs denoising processing to obtain a denoised target image; the host computer converts the target image into a target 3D point cloud, and performs filtering and denoising processing on the target 3D point cloud to form a denoised target 3D point cloud; the denoised target 3D point cloud is reconverted into an image to obtain a denoised target image.

[0015] Among them, the silencer target posture in the silencer target image coordinate system in the denoised target image is extracted and processed to obtain the silencer target posture information, the silencer target in the denoised target image is matched with the posture template stored in the host computer, and the posture information of the silencer target is obtained based on the matching result; the silencer target in the denoised target image is extracted and processed based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target; the silencer target posture information is obtained based on the posture information of the silencer target and the pixel coordinates of the silencer target.

[0016] Among them, the silencer target in the denoised target image is matched with the posture template stored in the host computer, the silencer target is extracted from the denoised target image to obtain the silencer target; and the silencer target is matched with the posture template stored in the host computer.

[0017] The physical space relationship information stored on the host computer is obtained by algorithm conversion based on preset target image posture information and preset robot position information.

[0018] Among them, the method for obtaining the preset target image posture information is to place the calibration plate within the field of view of the 3D camera; the host computer triggers the 3D camera to take a photo of the calibration plate to obtain the calibration plate image; extract the posture information of the target position of the calibration plate in the pixel coordinate system, and record the extracted posture information on the host computer as the preset target image posture information.

[0019] Among them, the method for obtaining the preset robot position information is to move the robot so that the center of the robot in the tool coordinate system is aligned with the target position of the calibration plate; when aligning with the target position of the calibration plate, the position information of the robot is obtained, and the position information of the robot is transmitted to the host computer for recording; and the position information of the robot is set as the preset robot position information in the host computer.

[0020] Among them, the host computer generates the robot's out-of-order grasping action planning instructions based on the robot's corresponding position information and the silencer target posture information; the host computer plans and generates the robot's out-of-order grasping path based on the robot's corresponding position information and the silencer target posture information; and generates the robot's out-of-order grasping action planning instructions based on the robot's out-of-order grasping path.

[0021] The present invention also provides a 3D random order capture device for a muffler based on image recognition, which includes an image acquisition module, a denoising module, an extraction module, a conversion calculation module, an instruction generation module and an instruction execution module;

[0022] The image acquisition module is used for the host computer to trigger the 3D camera to acquire images of the silencer target, and transmit the acquired target images to the host computer;

[0023] The denoising module is used by the host computer to convert the received target image into a 3D point cloud and perform denoising processing to obtain a denoised target image;

[0024] The extraction module is used to extract the position and posture of the muffler target in the denoised target image in the pixel coordinate system of the calibration plate to obtain the position and posture information of the muffler target;

[0025] The conversion calculation module is used to perform mathematical algorithm conversion processing using the target posture information of the muffler and the physical space relationship information stored on the host computer to obtain the corresponding position information of the robot;

[0026] The instruction generation module is used for the host computer to generate the out-of-order grasping action planning instructions of the robot based on the position information corresponding to the robot and the target posture information of the muffler;

[0027] The instruction execution module is used for the robot to execute the task of grasping the silencer target in disorder based on the disorderly grasping action planning instruction.

[0028] The beneficial effects of the present invention are as follows: the 3D vision-guided grasping method and device for the silencer product cover described in the present invention acquires images through a 3D camera and recognizes the images on a host computer; the host computer controls the robot to perform random grasping tasks; the reliability of material picking is greatly improved; at the same time, after the robot performs grasping, it can be directly transferred, which will greatly improve the production rhythm and improve production efficiency; at the same time, through image recognition, the front and back sides of the target can be directly identified, and no special hardware is required for front and back detection and a flipping mechanism to realize the flipping function. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention is further described below with the aid of the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention.

[0030] Figure 1 3D random grabbing method of a muffler based on image recognition in an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the structural composition of a 3D out-of-order capture device for a silencer based on image recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described below in conjunction with specific implementation methods.

[0033] like Figure 1 As shown, a 3D vision-guided grasping method for a silencer product cover includes the following steps:

[0034] Step 1 S11, the host computer triggers the 3D camera to collect images of the silencer target, and transmits the collected target image to the host computer;

[0035] Step 2 S12, the host computer converts the received target image into a 3D point cloud and performs denoising to obtain a denoised target image;

[0036] Step three S13, extracting the position and posture of the muffler target in the denoised target image in the pixel coordinate system to obtain the position and posture information of the muffler target;

[0037] Step 4 S14, using the muffler target posture information and the physical space relationship information stored in the host computer to perform mathematical algorithm conversion processing to obtain the corresponding position information of the robot;

[0038] Step 5 S15, the host computer generates a random grabbing action planning instruction of the robot based on the position information corresponding to the robot and the target posture information of the muffler;

[0039] Step six S16, the robot executes the task of grasping the silencer target in a disorderly order based on the disorderly grasping action planning instruction.

[0040] Furthermore, the host computer triggers the 3D camera to capture images of the silencer target and transmits the captured target image to the host computer; the host computer generates a trigger instruction and sends the trigger instruction to the 3D camera based on the TCP network protocol; the 3D camera triggers image capture of the silencer target based on the trigger instruction; the 3D camera transmits the captured target image to the host computer based on the TCP network protocol.

[0041] Furthermore, the host computer converts the received target image into a target 3D point cloud, and performs denoising processing to obtain a denoised target image; the host computer converts the target image into a target 3D point cloud, and performs filtering and denoising processing on the target 3D point cloud to form a denoised target 3D point cloud; the denoised target 3D point cloud is reconverted into an image to obtain a denoised target image.

[0042] Furthermore, the pose of the silencer target in the denoised target image in the coordinate system of the silencer target image is extracted and processed to obtain pose information of the silencer target, the silencer target in the denoised target image is matched with the pose template stored in the host computer, and the pose information of the silencer target is obtained based on the matching result; the pixel coordinates of the silencer target in the denoised target image are extracted and processed based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target; the pose information of the silencer target is obtained based on the pose information of the silencer target and the pixel coordinates of the silencer target.

[0043] Furthermore, the silencer target in the denoised target image is matched with the posture template stored in the host computer, and the silencer target is extracted from the denoised target image to obtain the silencer target; and the silencer target is matched with the posture template stored in the host computer.

[0044] Furthermore, the physical space relationship information stored in the host computer is obtained by algorithm conversion based on preset target image posture information and preset robot position information.

[0045] Furthermore, the preset target image posture information is obtained by placing the calibration plate within the field of view of the 3D camera; the host computer triggers the 3D camera to take a photo of the calibration plate to obtain an image of the calibration plate; extracting the posture information of the target position of the calibration plate in the pixel coordinate system, recording the extracted posture information on the host computer, and using it as the preset target image posture information.

[0046] Furthermore, the method for obtaining the preset robot position information is to move the robot so that the center of the robot in the tool coordinate system is aligned with the target position of the calibration plate; when aligning with the target position of the calibration plate, the position information of the robot is obtained, and the position information of the robot is transmitted to the host computer for recording; and the position information of the robot is set as the preset robot position information in the host computer.

[0047] Furthermore, the host computer generates out-of-order grasping action planning instructions for the robot based on the position information corresponding to the robot and the target posture information of the silencer; the host computer plans and generates an out-of-order grasping path for the robot based on the position information corresponding to the robot and the target posture information of the silencer; and generates an out-of-order grasping action planning instruction for the robot based on the out-of-order grasping path of the robot.

[0048] In the specific implementation process of the present invention, in step 1 S11, the host computer triggers the 3D camera to capture images of the silencer target, and transmits the captured target image to the host computer, including: the host computer generates a trigger instruction, and sends the trigger instruction to the 3D camera based on the TCP network protocol; the 3D camera triggers image capture of the silencer target based on the trigger instruction; the 3D camera transmits the captured target image to the host computer based on the TCP network protocol.

[0049] Specifically, in the present invention, the execution of the 3D out-of-order grabbing method for the silencer needs to include the following hardware structure, specifically a binocular camera or a laser (3D camera), which can realize wide-area image capture; the robot's mechanical arm can be controlled to operate the 3D camera to take pictures of the object, and perform grabbing after identification; the mechanical arm is a six-turn mechanical arm; the host computer can extract the image information obtained by the 3D camera, and identify and match the silencer model to see which one meets the assembly requirements, and generate posture information to generate control information, and control the robot's mechanical arm through the control information to realize the out-of-order grabbing and assembly of the silencer target.

[0050] That is, firstly, a trigger instruction for triggering the 3D camera is generated on the host computer, and then the trigger instruction is sent to the 3D camera through the TCP network protocol. After receiving the trigger instruction, the 3D camera triggers the 3D camera to collect images of the silencer target through the trigger instruction; finally, the target image collected by the 3D camera is transmitted to the host computer through the TCP network protocol, and the host computer will subsequently process the received target image.

[0051] In the specific implementation process of the present invention, in step two S12, the host computer converts the received target image into a target point cloud, and performs denoising processing to obtain a denoised target image, including: after receiving the target image, the host computer converts the target image into a target point cloud based on a method of converting the image coordinate system into a world coordinate system; performs filtering and denoising processing on the target point cloud to form a denoised target point cloud; and converts the denoised target point cloud back into an image to obtain a denoised target image.

[0052] Specifically, after receiving the target image, the host computer needs to convert the target image into a target point cloud, that is, convert the target image into a target point cloud by converting the image coordinate system into the world coordinate system, and then perform relevant filtering processing on the target point cloud to reduce noise, so as to form a denoised target point cloud; the denoised target point cloud is converted back into an image to obtain a denoised target image.

[0053] In the specific implementation process of the present invention, in step three S13, the silencer target posture of the silencer target in the denoised target image in the pixel coordinate system of the calibration plate is extracted and processed to obtain the silencer target posture information, including: matching the silencer target in the denoised target image with the posture template retained in the host computer, and obtaining the posture information of the silencer target based on the matching result; extracting the pixel coordinates of the silencer target in the denoised target image with the pixel coordinate system of the calibration plate as a reference to obtain the pixel coordinates of the silencer target; obtaining the silencer target posture information based on the posture information of the silencer target and the pixel coordinates of the silencer target.

[0054] Specifically, first, the silencer target in the denoised target image is matched with the retained posture template retained in the host computer to obtain a matching result, and the posture information of the silencer target is obtained based on the matching result; the silencer target in the denoised target image is subjected to pixel coordinate extraction processing based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target, and finally, the silencer target posture information P1 (x, y, z, a, b, c) is obtained based on the posture information of the silencer target and the pixel coordinates of the silencer target.

[0055] When performing matching processing, the denoised target image is subjected to silencer target extraction processing. Specifically, the target edge can be extracted by binarization, and then the target is segmented according to the extracted target edge to obtain the silencer target; then the silencer target is matched with the posture template retained in the host computer.

[0056] In the specific implementation process of the present invention, in step four S14, the physical space relationship information stored in the host computer is obtained by algorithm conversion based on the preset target image posture information and the preset robot position information.

[0057] The physical spatial relationship between the specific silencer target and the robot is obtained through calibration. Calibration, also known as hand-eye calibration, has the goal and function of establishing a physical spatial relationship between the 3D camera and the robot. That is, after the host computer acquires the image, it obtains image pixel information, and the obtained posture information is based on the coordinate system of the image pixels, which is inconsistent with the coordinate system used by the robot, so calibration is required to establish a conversion relationship between the two.

[0058] That is, a calibration plate is placed within the field of view of the 3D camera; the 3D camera is triggered by the host computer to take pictures and process, and a 3D image on the calibration plate is obtained; the host computer extracts the pose information P(x, y, z, a, b, c) of the target position of the calibration plate based on the pixel coordinate system through the 3D image (3), and the host computer records it; the robot is moved so that the center of the robot tool coordinate system is aligned with the target position of the calibration plate, and the position R(x, y, z, a, b, c) of the robot is sent to the host computer for recording; (5) through the conversion algorithm, according to the image pose information P(x, y, z, a, b, c) and the robot position information R(x, y, z, a, b, c), a physical space relationship V(x, y, z, a, b, c) between the two is established; and then the physical space relationship V(x, y, z, a, b, c) is stored on the host computer.

[0059] After obtaining the target position information of the muffler, the target position information of the muffler is converted and calculated with the physical space relationship information stored in the host computer to obtain the corresponding position information of the robot. The host computer then generates corresponding instructions based on the corresponding position information of the robot to control the robot to execute the movement; thus, the calibration only needs to be performed once, and then it can be used for a long time, and it will not be affected even if the material product is changed.

[0060] During the specific implementation of the present invention, in step five S15, the host computer generates out-of-order grasping planning instructions for the robot based on the position information corresponding to the robot and the target posture information of the silencer, including: the host computer plans and generates an out-of-order grasping path for the robot based on the position information corresponding to the robot and the target posture information of the silencer; and generates an out-of-order grasping planning instruction for the robot based on the out-of-order grasping path of the robot.

[0061] Specifically, the robot's out-of-order grasping path is planned and generated on the host computer according to the robot's corresponding position information and the silencer target posture information; after the out-of-order grasping path is generated, the robot's out-of-order grasping planning instructions are generated through the robot's out-of-order grasping path generation; and then the out-of-order grasping planning instructions are sent to the robot through the TCP network protocol.

[0062] In the specific implementation process of the present invention, in step six S16, after receiving the out-of-order grasping planning instruction through the TCP network, the robot parses the instruction and executes the grasping task of the silencer target in the out-of-order according to the parsed out-of-order grasping planning instruction.

[0063] To sum up, in the embodiments of the present invention, image acquisition is performed through a 3D camera, and the image is recognized on the host computer; the host computer controls the robot to perform out-of-order grasping tasks; the reliability of material picking is greatly improved; at the same time, after the robot performs grasping, it can be directly transferred, which will greatly improve the production rhythm and improve production efficiency; at the same time, through image recognition, the front and back sides of the target can be directly identified, and there is no need for special hardware to perform front and back detection and flipping mechanism to realize the flipping function.

[0064] The present invention also provides a 3D random order capture device for a muffler based on image recognition, comprising an image acquisition module 21, a denoising module 22, an extraction module 23, a conversion calculation module 24, an instruction generation module 25 and an instruction execution module 26;

[0065] The image acquisition module 21 is used by the host computer to trigger the 3D camera to acquire images of the silencer target, and transmit the acquired target image to the host computer. In the specific implementation process of the present invention, the host computer triggers the 3D camera to acquire images of the silencer target, and transmits the acquired target image to the host computer, including: the host computer generates a trigger instruction, and sends the trigger instruction to the 3D camera based on the TCP network; the 3D camera triggers image acquisition of the silencer target based on the trigger instruction; the 3D camera transmits the acquired target image to the host computer based on the TCP network.

[0066] Specifically, in the present invention, the execution of the 3D out-of-order grabbing method for the silencer needs to include the following hardware structure, specifically a binocular camera or a laser (3D camera), which can realize wide-area image capture; the robot's mechanical arm can be controlled to operate the 3D camera to take pictures of the object, and perform grabbing after identification; the mechanical arm is a six-axis mechanical arm; the host computer can extract the image information obtained by the 3D camera, and identify and match the silencer model, select the assembly requirements that meet the optimal target, and generate posture information to generate control information, and control the robot's mechanical arm through the control information to realize the out-of-order grabbing and assembly of the silencer target.

[0067] That is, firstly, a trigger instruction for triggering the 3D camera is generated on the host computer, and then the trigger instruction is sent to the 3D camera through the TCP network protocol. After receiving the trigger instruction, the 3D camera triggers the 3D camera to collect images of the silencer target through the trigger instruction; finally, the target image collected by the 3D camera is transmitted to the host computer through the TCP network protocol, and the host computer will subsequently process the received target image;

[0068] The denoising module 22 is used for the host computer to convert the received target image into a 3D point cloud, and perform denoising processing to obtain the denoised target image. In the specific implementation process of the present invention, the host computer converts the received target image into a target point cloud, and performs denoising processing to obtain the denoised target image, including: after receiving the target image, the host computer converts the target image into a target point cloud based on the image coordinate system into the world coordinate system; performs filtering and denoising processing on the target point cloud to form a denoised target point cloud; and converts the denoised target point cloud back into an image to obtain a denoised target image.

[0069] Specifically, after receiving the target image, the host computer needs to convert the target image into a target point cloud, that is, convert the target image into a target point cloud by converting the image coordinate system into the world coordinate system, and then perform relevant filtering processing on the target point cloud to reduce noise, so as to form a denoised target point cloud; the denoised target point cloud is converted back into an image to obtain a denoised target image;

[0070] The extraction module 23 is used to extract and process the silencer target posture of the silencer target in the denoised target image in the pixel coordinate system of the calibration plate to obtain the silencer target posture information. In the specific implementation process of the present invention, the silencer target posture of the silencer target in the denoised target image in the pixel coordinate system of the calibration plate is extracted and processed to obtain the silencer target posture information, including: matching the silencer target in the denoised target image with the posture template retained in the host computer, and obtaining the posture information of the silencer target based on the matching result; extracting and processing the pixel coordinates of the silencer target in the denoised target image based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target; obtaining the silencer target posture information based on the posture information of the silencer target and the pixel coordinates of the silencer target.

[0071] Furthermore, the process of matching the silencer target in the denoised target image with the posture template stored in the host computer includes: performing silencer target extraction processing on the denoised target image to obtain the silencer target; and matching the silencer target with the posture template stored in the host computer.

[0072] Specifically, first, the silencer target in the denoised target image is matched with the retained posture template retained in the host computer to obtain a matching result, and the posture information of the silencer target is obtained based on the matching result; the silencer target in the denoised target image is subjected to pixel coordinate extraction processing based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target, and finally, the silencer target posture information P1 (x, y, z, a, b, c) is obtained based on the posture information of the silencer target and the pixel coordinates of the silencer target.

[0073] When performing matching processing, the target image after noise reduction is subjected to muffler target extraction processing. Specifically, the target edge can be extracted by binarization, and then the target is segmented according to the extracted target edge to obtain the muffler target; and then the muffler target is matched with the posture template retained in the upper computer;

[0074] The conversion calculation module 24 is used to use the silencer target posture information and the physical space relationship information stored on the host computer to perform mathematical algorithm conversion processing to obtain the corresponding position information of the robot. In the specific implementation process of the present invention, the physical space relationship information stored on the host computer is obtained by algorithm conversion based on the preset target image posture information and the preset robot position information.

[0075] Furthermore, the method for obtaining the preset target image posture information includes: placing the calibration plate within the field of view of the 3D camera; the host computer triggers the 3D camera to take a photo of the calibration plate to obtain an image of the calibration plate; extracting the posture information of the target position of the calibration plate in the pixel coordinate system, recording the extracted posture information on the host computer, and using it as the preset target image posture information.

[0076] Furthermore, the method for obtaining the preset robot position information includes: moving the robot so that the center of the robot in the tool coordinate system is aligned with the target position of the calibration plate; when aligning with the target position of the calibration plate, obtaining the position information of the robot, and transmitting the position information of the robot to the host computer for recording; setting the position information of the robot in the host computer as the preset robot position information.

[0077] The physical spatial relationship between the specific silencer target and the robot is obtained through calibration. Calibration, also known as hand-eye calibration, has the goal and function of establishing a physical spatial relationship between the 3D camera and the robot. That is, after the host computer acquires the image, it obtains image pixel information, and the obtained posture information is based on the coordinate system of the image pixels, which is inconsistent with the coordinate system used by the robot, so calibration is required to establish a conversion relationship between the two.

[0078] That is, a calibration plate is placed within the field of view of the 3D camera; the 3D camera is triggered by the host computer to take pictures and process, and a 3D image on the calibration plate is obtained; the host computer extracts the pose information P(x, y, z, a, b, c) of the target position of the calibration plate based on the pixel coordinate system through the 3D image (3), and the host computer records it; the robot is moved so that the center of the robot tool coordinate system is aligned with the target position of the calibration plate, and the position R(x, y, z, a, b, c) of the robot is sent to the host computer for recording; (5) through the conversion algorithm, according to the image pose information P(x, y, z, a, b, c) and the robot position information R(x, y, z, a, b, c), a physical space relationship V(x, y, z, a, b, c) between the two is established; and then the physical space relationship V(x, y, z, a, b, c) is stored on the host computer.

[0079] After obtaining the target position information of the muffler, the target position information of the muffler is converted and calculated with the physical space relationship information stored in the host computer to obtain the corresponding position information of the robot. The host computer then generates corresponding instructions based on the corresponding position information of the robot to control the robot to execute the movement; thus, the calibration only needs to be performed once, and then it can be used for a long time, and it will not be affected even if the material product is changed;

[0080] The instruction generation module 25 is used for the host computer to generate the out-of-order grasping action planning instructions of the robot based on the position information corresponding to the robot and the target posture information of the silencer. In the specific implementation process of the present invention, the host computer generates the out-of-order grasping planning instructions of the robot based on the position information corresponding to the robot and the target posture information of the silencer, including: the host computer plans and generates the out-of-order grasping path of the robot based on the position information corresponding to the robot and the target posture information of the silencer; and generates the out-of-order grasping planning instructions of the robot based on the out-of-order grasping path of the robot.

[0081] Specifically, the host computer plans and generates the robot's random grasping path according to the corresponding position information of the robot and the target posture information of the muffler; after the random grasping path is generated, the robot's random grasping planning instructions are generated through the robot's random grasping path generation; and then the random grasping planning instructions are sent to the robot through the TCP network protocol;

[0082] The instruction execution module 26 is used for the robot to execute the grabbing task of the silencer target in disorder based on the disordered grabbing action planning instruction. In the specific implementation process of the present invention, after receiving the disordered grabbing planning instruction through the TCP network protocol, the robot parses the instruction, extracts the instruction information, and executes the grabbing task of the silencer target in disorder according to the parsed disordered grabbing planning instruction.

[0083] In the embodiment of the present invention, image acquisition is performed through a 3D camera, and the image is recognized on the host computer; the host computer controls the robot to perform out-of-order grasping tasks; the reliability of material picking is greatly improved; at the same time, after the robot performs grasping, it can be directly transferred, which will greatly improve the production rhythm and improve production efficiency; at the same time, through image recognition, the front and back sides of the target can be directly identified, and there is no need for special hardware to perform front and back detection and flipping mechanism to realize the flipping function.

[0084] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0085] The above contents are only preferred embodiments of the present invention. For ordinary technicians in this field, according to the concept of the present invention, there will be changes in the specific implementation methods and application scopes. The content of this specification should not be understood as limiting the present invention.

Claims

1. A 3D vision-guided grasping method for a muffler product cover, characterized in that: The following steps are included: Step 1: The host computer triggers the 3D camera to collect images of the silencer target, and transmits the collected target image to the host computer; Step 2: The host computer converts the received target image into a 3D point cloud and performs denoising to obtain a denoised target image; Step 3, extracting and processing the silencer target position and posture of the silencer target in the denoised target image in the pixel coordinate system to obtain the silencer target position and posture information; Step 4, using the target posture information of the muffler and the physical space relationship information stored on the host computer to perform mathematical algorithm conversion processing to obtain the corresponding position information of the robot; Step 5, the host computer generates a random grabbing action planning instruction for the robot based on the position information corresponding to the robot and the target posture information of the muffler; Step six, the robot executes the task of grasping the silencer target in disorder based on the disordered grasping action planning instruction.

2. The muffler 3D random grabbing method according to claim 1 is characterized by: The host computer triggers the 3D camera to capture images of the silencer target and transmits the captured target image to the host computer; the host computer generates a trigger instruction and sends the trigger instruction to the 3D camera based on the TCP network protocol; the 3D camera triggers image capture of the silencer target based on the trigger instruction; the 3D camera transmits the captured target image to the host computer based on the TCP network protocol.

3. The muffler 3D random grabbing method according to claim 1 is characterized in that: The host computer converts the received target image into a target 3D point cloud, and performs denoising processing to obtain a denoised target image. The host computer converts the target image into a target 3D point cloud, performs filtering and denoising processing on the target 3D point cloud to form a denoised target 3D point cloud; the denoised target 3D point cloud is reconverted into an image to obtain a denoised target image.

4. The muffler 3D random grabbing method according to claim 1 is characterized in that: The process of extracting and processing the position and posture of the silencer target in the denoised target image in the coordinate system of the silencer target image to obtain the position and posture information of the silencer target, matching the silencer target in the denoised target image with the posture template stored in the host computer, and obtaining the posture information of the silencer target based on the matching result; extracting and processing the pixel coordinates of the silencer target in the denoised target image based on the pixel coordinate system of the calibration plate to obtain the pixel coordinates of the silencer target; The position information of the silencer target is obtained based on the posture information of the silencer target and the pixel coordinates of the silencer target.

5. The muffler 3D random grabbing method according to claim 4 is characterized in that: The muffler target in the denoised target image is matched with the posture template stored in the host computer, and the muffler target is extracted from the denoised target image to obtain the muffler target; and the muffler target is matched with the posture template stored in the host computer.

6. The muffler 3D random grabbing method according to claim 1 is characterized by: The physical space relationship information stored in the host computer is obtained by algorithm conversion based on preset target image posture information and preset robot position information.

7. The muffler 3D random grabbing method according to claim 6 is characterized by: The method for obtaining the preset target image posture information is to place the calibration plate in the field of view of the 3D camera; the host computer triggers the 3D camera to take a picture of the calibration plate to obtain an image of the calibration plate; The pose information of the target position of the calibration plate in the pixel coordinate system is extracted, and the extracted pose information is recorded on the host computer as preset target image pose information.

8. The method for capturing 3D random images of a muffler according to claim 6, characterized in that: The method for obtaining the preset robot position information is to move the robot so that the center of the robot in the tool coordinate system is aligned with the target position of the calibration plate; when aligning with the target position of the calibration plate, the position information of the robot is obtained, and the position information of the robot is transmitted to the host computer for recording; and the position information of the robot is set as the preset robot position information in the host computer.

9. The muffler 3D random grabbing method according to claim 1, characterized in that: The host computer generates a random grasping action planning instruction for the robot based on the position information corresponding to the robot and the target posture information of the silencer; the host computer plans and generates a random grasping path for the robot based on the position information corresponding to the robot and the target posture information of the silencer; and generates a random grasping action planning instruction for the robot based on the random grasping path of the robot.

10. A 3D random grabbing device for silencers based on image recognition, characterized in that: It includes an image acquisition module, a denoising module, an extraction module, a conversion calculation module, an instruction generation module and an instruction execution module; The image acquisition module is used for the host computer to trigger the 3D camera to acquire images of the silencer target, and transmit the acquired target images to the host computer; The denoising module is used by the host computer to convert the received target image into a 3D point cloud and perform denoising processing to obtain a denoised target image; The extraction module is used to extract the position and posture of the muffler target in the denoised target image in the pixel coordinate system of the calibration plate to obtain the position and posture information of the muffler target; The conversion calculation module is used to perform mathematical algorithm conversion processing using the target posture information of the muffler and the physical space relationship information stored on the host computer to obtain the corresponding position information of the robot; The instruction generation module is used for the host computer to generate the out-of-order grasping action planning instructions of the robot based on the position information corresponding to the robot and the target posture information of the muffler; The instruction execution module is used for the robot to execute the task of grasping the silencer target in disorder based on the disorderly grasping action planning instruction.

Images