A robot vision recognition and positioning method for manhole cover and bolt features of tank trucks

Abstract

The present invention discloses a robot visual recognition and positioning method for the features of a manhole cover and bolts of a tank truck, which specifically comprises the following steps: Step 1: Ensure that the tank truck is in a stationary state, the robot carries a visual device, and after the visual device is initialized, the laser sensor gives the rough position of the manhole cover, a wide-field camera images the manhole cover, and successively collects images of the manhole cover body; Step 2: After image processing, the target position is calculated and transmitted to the robot end, and the robot is guided. The field of view presented by the first large field of view is 1.2 times the diameter of the manhole cover, and a template matching algorithm is used to match the target position; the present invention realizes the accurate recognition of the shape and position of the manhole cover of the tank truck and the bolts around it under the conditions of complex outdoor light changes and large changes in the consistency of the outer dimensions of the operation target; the present invention realizes the unmanned operation process of the visually guided robot, with a high degree of automation, high efficiency, and improved operation safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of identification of closed quantitative loading systems for railway and highway gasoline, diesel and aviation kerosene, and in particular relates to a robot visual identification and positioning method for manhole covers and bolt features of tank trucks. Background Art

[0002] At present, when gasoline, diesel, aviation kerosene, petroleum benzene, xylene and other media are filled by the medium injection loading system, the manhole cover of the tank truck requires the staff to manually unscrew the bolts around the top of the tank truck and open the manhole cover before filling. Many petrochemical refineries are located in the northeast region. In winter, ice and snow are easy to form on the tank body. When manual operations are carried out, people need to frequently get on and off the tank truck, and the risk of personnel falling is high. The manual loading process has many procedures, large labor volume, high labor intensity, and a harsh working environment.

[0003] Currently, a method is needed to realize the shape recognition and precise positioning of the manhole cover of a tank truck and the bolts around the locking manhole cover, so as to guide the automated equipment or robot to automatically open and close the manhole cover. Due to the large error in the straight-line distance of the parking position in the direction of travel of the tank truck, the poor product consistency of the manhole cover and the bolts around it, and the influence of rain, snow, dust and sand in the outdoor environment, the shape recognition and precise positioning of the manhole cover of the tank truck and the bolts around it are relatively difficult. Summary of the invention

[0004] The purpose of the present invention is to provide a robot visual recognition and positioning method for the manhole cover and bolt features of a tank truck, which is mainly used in the railway and highway fields, so as to solve the problems proposed in the above-mentioned background technology that the manhole cover of the tank truck requires workers to go to the top of the tank truck to manually unscrew the bolts around it, and when manual work is performed, people need to frequently get on and off the tank truck, and the risk of people falling is high; the manual loading process is adopted, with numerous procedures, large amount of labor, high labor intensity, and poor working environment.

[0005] To achieve the above object, the present invention provides the following technical solution: a robot visual recognition and positioning method for manhole cover and bolt features of a tank truck, specifically comprising the following steps:

[0006] Step 1: Ensure that the tank truck is stationary. The robot carries a visual device. After the visual device is initialized, the laser sensor gives the rough position of the manhole cover. The wide-field camera images the manhole cover and collects images of the manhole cover body in sequence.

[0007] Step 2: After image processing, the target position is calculated and transmitted to the robot end to guide the robot. The first large field of view presents a field of view of 1.2 times the diameter of the manhole cover, and the template matching algorithm is used to match the target position;

[0008] Step 3: If the manhole cover cannot be identified, readjust the matching parameters and return to step 1 for re-identification;

[0009] Step 4: After obtaining the target position, estimate the approximate position of the bolt, process the bolt image, obtain the image coordinate position of the bolt, and combine it with the visual calibration result to calculate the actual position of the bolt; for a tank truck manhole cover that is already in a stationary state, the robot needs to carry a visual device to calculate the orientation and position of the manhole cover on the two-dimensional plane. The result obtained in this step guides the robot to carry the end effector to approach the manhole cover. The robot end effector needs to locate the cover opening handle on the manhole cover to open the manhole cover, and the positioning accuracy is required to be within 1cm; after calculating the overall position of the manhole cover, in order to guide the automatic screw machine to disassemble the four long bolts on the manhole cover, the robot vision must give the position of the bolt in the two-dimensional plane. For disassembling the bolts, the visual positioning accuracy is required to be within 0.5mm;

[0010] Step 5: Transmit the coordinates to the robot.

[0011] Preferably, it also includes calibration paper established on the X and Y axes, the calibration paper is placed orthogonally to the coordinate system of the bolt in the coordinate system, and after the camera takes a picture, the marking points in the X and Y directions are extracted and the pixel difference is calculated.

[0012] Preferably, in the step 2, when imaging in a large field of view, it is necessary to image the manhole cover in a cluttered background. In order to eliminate the influence of ambient light, a light shield is installed around the manhole cover and a circular angle light source is used to achieve small field of view imaging. The light source focuses on the upper surface of the bolt, so that the surface of the bolt presents a circular contour.

[0013] Preferably, in step 2, in order to achieve target matching, the target recognition method based on contour features includes a feature library of object templates and an image of the object to be detected, and determines whether the image to be detected contains the detection target by calculating the difference between the object template in the feature library and the contour shape of the image to be detected, specifically including the following steps:

[0014] A. Binarize the object templates and the images to be detected in the feature library;

[0015] B. Perform contour detection on the object templates in the feature library and the image to be detected;

[0016] C. Calculate the centroid of the object template in the feature library and the centroid of the image to be detected;

[0017] D. Align the centroid of the image to be detected with the centroid of the object template in the feature library;

[0018] E. Determine whether the image to be detected contains the detection target by calculating the difference between the object template in the feature library and the contour shape of the image to be detected.

[0019] Preferably, the robot vision system also includes identifying a MARK point on the scale line of the calibration tooling table coordinate system, the MARK point is located on the scale line of the robot user coordinate system, and the image coordinate system of the camera is established using the MARK point.

[0020] Preferably, in step 4, the approximate position of the bolt is the field of view of the subsequent four small field of view imagings, and the visual imaging field of view is 15 cm×15 cm.

[0021] Preferably, the bolt positions on the tank cover are identified by a visual system, and their position coordinates are converted into the user coordinate system of the robot, with a total of 4 sets of position coordinates;

[0022] Since the screw machine is symmetrically distributed, the robot TCP point can be moved to the midpoint of the symmetrical bolt connection line, and then the screw machine sleeve can be moved above the bolt by rotating it around the Z axis of the user coordinate system for a certain angle, where:

[0023] The host computer sends the TCP point coordinates to the robot as and

[0024] The host computer sends the TCP point to the robot, and the rotation angle around the Z axis of the user coordinate system is and

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The present invention realizes accurate identification of the shape and position of the manhole cover of the tank truck and the surrounding bolts in the case of complex outdoor light environment and large changes in the consistency of the outer dimensions of the operating target;

[0027] 2. The present invention realizes the unmanned operation process of the vision-guided robot, with high automation, high efficiency and improved operation safety;

[0028] 3. The combined application of laser scanner technology and visual technology realizes the recognition and positioning of the shape and spatial position size of the work target, with fast speed, low cost and high reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the visual workflow of the present invention;

[0030] Figure 2 Establishing a diagram for the coordinate system in the actual space of the present invention;

[0031] Figure 3 This is a schematic diagram of the calibration paper for calibrating the robot of the present invention;

[0032] Figure 4 This is a schematic diagram of tank truck cover identification data of the present invention;

[0033] Figure 5 Schematic diagram of the robot visual coordinate system of the present invention;

[0034] Figure 6 Schematic diagram of the mathematical model of the robot of the present invention;

[0035] Figure 7 It is a schematic diagram of the mathematical model of the robot end effector of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] See also Figures 1 to 7 The present invention provides a technical solution: a robot visual recognition and positioning method for manhole covers and bolt features of tank trucks, which specifically includes the following steps:

[0038] Step 1: Ensure that the tank truck is stationary. The robot carries a visual device. After the visual device is initialized, the laser sensor gives the rough position of the manhole cover. The wide-field camera images the manhole cover and collects images of the manhole cover body in sequence.

[0039] Step 2: After image processing, the target position is calculated and transmitted to the robot end to guide the robot. The first large field of view presents a field of view of 1.2 times the diameter of the manhole cover, and the template matching algorithm is used to match the target position;

[0040] Step 3: If the manhole cover cannot be identified, readjust the matching parameters and return to step 1 for re-identification;

[0041] Step 4: After obtaining the target position, estimate the approximate position of the bolt, process the bolt image, obtain the image coordinate position of the bolt, and combine it with the visual calibration result to calculate the actual position of the bolt;

[0042] Step 5: Transmit the coordinates to the robot.

[0043] In this embodiment, calibration paper is also included on the X and Y axes. The purpose of camera calibration is to obtain the representation of the actual distance in pixels. Figure 3 The calibration paper shown in the figure is in the same coordinate system as Figure 2The coordinate system shown is placed orthogonally. After the camera takes a picture, the X and Y marking points are extracted and the pixel difference is calculated. During the production of the calibration paper, the distance between the two marking points is known. The calibration paper is placed on a plane at the same height as the bolt surface, and the pixel difference is calculated. The distance between the two points is known. The corresponding proportional relationship can be obtained.

[0044] In the present embodiment, preferably, in the step 2, when imaging in a large field of view, it is necessary to image the manhole cover in a cluttered background. In order to eliminate the influence of ambient light, a light shield is installed around the manhole cover and a circular angle light source is used to achieve small field of view imaging. The light source focuses on irradiating the upper surface of the bolt, so that the surface of the bolt presents a circular contour.

[0045] In this embodiment, in the step 2, in order to achieve target matching, the target recognition method based on contour features includes a feature library of object templates and an image of the object to be detected, and determines whether the image to be detected contains the detection target by calculating the difference between the object template in the feature library and the contour shape of the image to be detected, which specifically includes the following steps:

[0046] A. Binarize the object templates and the images to be detected in the feature library;

[0047] B. Perform contour detection on the object templates in the feature library and the image to be detected;

[0048] C. Calculate the centroid of the object template in the feature library and the centroid of the image to be detected;

[0049] D. Align the centroid of the image to be detected with the centroid of the object template in the feature library;

[0050] E. Determine whether the image of the object to be detected contains the detection target by calculating the difference between the object template in the feature library and the contour shape of the image to be detected; For this system, the goal of the hand-eye calibration technology is to make the position of the object in the image obtained by the camera be mapped to the actual space. The opening of the handle of the manhole cover and the disassembly of the bolts can be considered as a point on the two-dimensional plane. The calibration of visual work can be simplified to the proportional relationship between the camera pixel and the actual distance, such as Figure 2 As shown, a coordinate system is established in the actual space. The coordinate system circle selects a target that can be accurately aligned in the actual space. When the robot aligns the handle and the bolt, it only needs to ensure that the position is aligned on the two-dimensional plane. The up and down stroke of the Z axis is obtained by other sensors. Assuming that the position of the upper right red bolt in the actual space is to be known, the position of the origin in the actual space is obtained, and the coordinates of the bolt in the image are obtained through image processing. To obtain the actual position of the bolt, the pixels at two points in time are required to represent the distance in the actual space.

[0051] In this embodiment, the robot vision system identifies the MARK point on the scale line of the calibration workbench coordinate system. The MARK point is located on the scale line of the robot user coordinate system, and the image coordinate system of the camera is established using the MARK point. In this way, the robot's user coordinate system and the camera's image coordinate system both coincide with the scale line coordinate system on the calibration workbench. Using the hand-eye calibration method of the robot and vision system, the transformation relationship between the camera coordinate system and the robot coordinate system can be obtained.

[0052] In this embodiment, in step 4, the approximate position of the bolt is the field of view of the subsequent four small field of view imagings, and the visual imaging field of view is 15 cm×15 cm.

[0053] In this embodiment, the bolt positions on the tank cover are identified by the visual system, and their position coordinates are converted into the user coordinate system of the robot. There are 4 sets of position coordinates in total;

[0054] Since the screw machine is symmetrically distributed, the robot TCP point can be moved to the midpoint of the symmetrical bolt connection line, and then the screw machine sleeve can be moved above the bolt by rotating it around the Z axis of the user coordinate system for a certain angle, where:

[0055] The host computer sends the TCP point coordinates to the robot as and

[0056] The host computer sends the TCP point to the robot, and the rotation angle around the Z axis of the user coordinate system is and Robot TCP point calibration method: The robot's TCP point position is as follows: Figure 5 As shown in the figure, a height-adjustable needle is inserted in the center of the calibration fixture frame. The tip of the needle is the TCP point of the lid opening robot actuator. The vertical height of the TCP point is the same as the vertical height of the bolt sleeve. The relative position of the TCP point relative to the rotation center of the A-axis and the flange surface is fixed. It can be calibrated by the robot's TCP calibration method, or the offset size can be determined by a three-dimensional drawing.

[0057] The robot's user coordinate system is Figure 5 As shown, the user coordinate system is located on the calibration tooling table, and the robot calibrates the user coordinate system by teaching the position of the engraved lines on the tooling table.

[0058] exist Figure 6 In the figure, the X-axis, Y-axis, and Z-axis are the translation axes of the rectangular coordinate robot, and A is the rotation axis of the robot end. Figure 7 In the figure, E1, E2, and E3 are axes executed by the robot end, and are set as external axes in the robot model.

[0059] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A robot visual recognition and positioning method for manhole covers and bolt features of tank trucks, characterized in that: Specifically, the following steps are included: Step 1: Ensure that the tank truck is stationary, the robot carries a visual device, and after the visual device is initialized, the laser sensor gives the rough position of the manhole cover, the wide-field camera images the manhole cover, and collects the manhole cover body image in succession; Step 2: After image processing, the target position is calculated and transmitted to the robot end to guide the robot. The first wide-field field presents a field of view of 1.2 times the diameter of the manhole cover, and the template matching algorithm is used to match the target position; Step 3: If the manhole cover cannot be identified, readjust the matching parameters and return to step 1 for re-identification; Step 4: After obtaining the target position, estimate the approximate position of the bolt, process the bolt image, and obtain the image coordinate position of the bolt, which is combined with the visual calibration result to calculate the actual position of the bolt; Step 5: The coordinate position is transmitted to the robot; It also includes calibration paper established on the X and Y axes, the calibration paper is placed orthogonally to the coordinate system of the bolt in the coordinate system, and after the camera takes a picture, the marking points in the X and Y directions are extracted and the pixel difference is calculated; In the step 2, when imaging in a large field of view, it is necessary to image the manhole cover in a cluttered background. In order to eliminate the influence of ambient light, a light shield is installed around the manhole cover and a circular angle light source is used to achieve imaging in a small field of view. The light source focuses on the upper surface of the bolt, so that the bolt surface presents a circular contour; In the step 2, in order to achieve target matching, the target recognition method based on contour features includes a feature library of object templates and an image of the object to be detected, and determines whether the image to be detected contains the detection target by calculating the difference between the contour shape of the object template in the feature library and the image to be detected, which specifically includes the following steps: A. Binarization of the object template in the feature library and the image to be detected; B. Contour detection of the object template in the feature library and the image to be detected; C. Calculating the centroid of the object template in the feature library and the centroid of the image to be detected; D. Aligning the centroid of the image to be detected with the centroid of the object template in the feature library; E. Determining whether the image of the object to be detected contains the detection target by calculating the difference between the contour shape of the object template in the feature library and the image to be detected; It also includes the robot vision system identifying and calibrating the MARK point on the engraved line of the tooling table coordinate system. The MARK point is located on the engraved line of the robot user coordinate system, and the image coordinate system of the camera is established using the MARK point; In step 4, the approximate position of the bolt is the field of view of the subsequent four small field of view imaging, and the visual imaging field of view is 15 cm×15 cm; The bolt position on the can cover is identified by the visual system, and its position coordinates are converted to the robot's user coordinate system. There are 4 sets of position coordinates in total. Since the screw machine is symmetrically distributed, the robot TCP point can be moved to the midpoint of the symmetrical bolt connection line, and then the screw machine sleeve can be moved to the bolt by rotating it around the user coordinate system Z axis at a certain angle. The host computer sends the TCP point coordinates to the robot as follows: and ; The host computer sends the TCP point to the robot. The rotation angle around the Z axis of the user coordinate system is and ; Among them, X1, X2, X3, X4, Y1, Y2, Y3, and Y4 are the horizontal and vertical coordinate positions of the bolts on the tank cover.

Images