Steel spline boxing positioning method and system

Abstract

The invention discloses a steel spline boxing positioning method and system, and the method comprises the steps: recognizing and determining a boxing type corresponding to a small steel spline based on an original image of the small steel spline at a preset position; according to the boxing type, obtaining three-point coordinates of a spline box corresponding to the boxing type through a positioning algorithm; and on the basis of the three-point coordinates, the small steel splines are stacked into the corresponding boxing type sample boxes according to a preset rule through a mechanical arm of the small sample robot. According to the invention, the robot automatic sampling assembly line system in the steel production and manufacturing process can realize intelligent steel sample strip boxing and positioning, so that the working efficiency is improved, and the labor cost is saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent manufacturing, and particularly relates to a method and system for positioning steel splines in a packing box. Background Art

[0002] In the actual steel production process, the existing intelligent degree of the steel spline packing link is low, resulting in low work efficiency and being prone to errors. Summary of the Invention

[0003] The purpose of the present invention is to overcome the defects of the prior art. The present invention provides a method and system for positioning steel splines in a packing box. After the splines are randomly placed in a specified area for packing, the manipulator automatically obtains the three-point coordinates of the steel spline packing box through automatic positioning, so as to realize automatic packing of the splines.

[0004] In order to achieve the expected effect, the present invention adopts the following technical solutions:

[0005] The present invention discloses a method for positioning steel splines in a packing box, including:

[0006] Based on the original image of the steel small spline at a preset position, identifying and determining the packing category corresponding to the steel small spline;

[0007] According to the packing category, obtaining the three-point coordinates of the spline box corresponding to the packing category through a positioning algorithm;

[0008] Based on the three-point coordinates, placing the steel small splines into the spline box corresponding to the packing category in a preset rule through the robotic arm of the small spline robot.

[0009] Further, the packing categories include: sample delivery box, sample retention box, slow cooling box.

[0010] Further, the obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm includes: fixing the positions of each spline box in a manner that the box foot angle steel corresponds to the ground slot, and obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm based on the coordinates of the ground slot.

[0011] Further, the obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm includes: the electromagnetic chuck ejector on the robotic arm of the small spline robot collides through the positioning algorithm to obtain the three-point coordinates of the spline box corresponding to the packing category.

[0012] Further, the positioning algorithm is the amcl algorithm.

[0013] Further, the three - point coordinates include a first - point coordinate, a second - point coordinate, and a third - point coordinate. Among them, the first - point coordinate and the second - point coordinate are on the first plane of the spline box, and the third - point coordinate is on the second plane of the spline box. The first plane is perpendicular to the second plane.

[0014] Further, the path planning and navigation of the manipulator of the small spline robot are completed using the move_base framework.

[0015] Further, the process of placing the steel small spline into the corresponding packing - category spline box according to the preset rules by the manipulator of the small spline robot includes:

[0016] Taking the line connecting the first point and the second point as the X - axis, and the perpendicular line from the third point to the X - axis as the Y - axis. The intersection point of the X - axis and the Y - axis is the origin.

[0017] Calculating the origin coordinates through a matrix equation;

[0018] Based on the origin coordinates, calculating the attitude of the manipulator of the small spline robot at the origin through a rotation matrix and an Euler - angle conversion function;

[0019] Based on the attitude of the manipulator of the small spline robot at the origin, converting the coordinates of the spline placement according to the size of the small spline grid.

[0020] Further, the preset rule is to place the small splines in a "field" shape from bottom to top in the spline box.

[0021] The present invention discloses a steel - spline packing and positioning system, including:

[0022] An acquisition module, configured to acquire the original image of the steel small spline at a preset position;

[0023] A packing and positioning module, configured to perform steel - spline packing and positioning according to the method described in any one of the above.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a steel - spline packing and positioning method and system. The present invention can enable the robot automatic sampling pipeline system in the steel production and manufacturing process to realize intelligent packing and positioning of steel splines, improve work efficiency, and save labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following - described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] Figure 1 It is a flowchart of a method for positioning steel splines in a packing box provided by an embodiment of the present invention.

[0027] Figure 2 It is a schematic diagram of three - point coordinates provided by an embodiment of the present invention.

[0028] Figure 3 It is a schematic diagram of an application scenario of a method for positioning steel splines in a packing box provided by an embodiment of the present invention. Detailed implementation manners

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0030] See Figures 1 to 3 , the present invention discloses a method for positioning steel splines in a packing box, including:

[0031] Step 1: Based on the original image of the steel small spline at a preset position, identify and determine the packing category corresponding to the steel small spline;

[0032] Specifically, when the sensor installed on the conveyor belt senses that the steel small spline reaches the preset position of the conveyor belt, in response to the induction of the sensor, the camera installed on the robotic arm of the small - spline robot obtains the original image of the steel small spline at the preset position in real - time. Based on the original image of the steel small spline at the preset position, identify and determine the packing category corresponding to the steel small spline.

[0033] In a preferred embodiment, the packing categories include: sample - sending box, sample - retaining box, slow - cooling box. Different small splines are placed in different spline boxes.

[0034] Step 2: According to the packing category, obtain the three - point coordinates of the spline box corresponding to the packing category through a positioning algorithm;

[0035] Furthermore, obtaining the three - point coordinates of the spline box corresponding to the packing category through the positioning algorithm includes: fixing the positions of each spline box in a way that the box - foot angle steel corresponds to the ground slot. Based on the coordinates of the ground slot, obtain the three - point coordinates of the spline box corresponding to the packing category through the positioning algorithm. At the same time, ensure that the spline box is within the grasping range of the robotic arm of the small - spline robot.

[0036] Further, the three - point coordinates of the spline box corresponding to the packing category obtained through the positioning algorithm include: the electromagnetic chuck thimble on the robotic arm of the small - spline robot collides through the positioning algorithm to obtain the three - point coordinates of the spline box corresponding to the packing category.

[0037] Further, the positioning algorithm is the amcl algorithm.

[0038] Further, the three - point coordinates include the first - point coordinate, the second - point coordinate, and the third - point coordinate. Among them, the first - point coordinate and the second - point coordinate are on the first plane of the spline box, and the third - point coordinate is on the second plane of the spline box. The first plane is perpendicular to the second plane, as Figure 2 shown.

[0039] Further, the path planning and navigation of the robotic arm of the small - spline robot are completed using the move_base framework.

[0040] Step 3: Based on the three - point coordinates, the steel small splines are placed into the spline box corresponding to the packing category by the robotic arm of the small - spline robot according to a preset rule.

[0041] Further, the process of placing the steel small splines into the spline box corresponding to the packing category by the robotic arm of the small - spline robot based on the three - point coordinates includes:

[0042] Connect the first point P1 and the second point P2 as the X - axis, and draw a perpendicular line from the third point P3 to the X - axis as the Y - axis. The intersection of the X - axis and the Y - axis is the origin;

[0043] Calculate the origin coordinates through the matrix equation;

[0044] Based on the origin coordinates, calculate the attitude of the robotic arm of the small - spline robot at the origin through the rotation matrix and the Euler - angle conversion function;

[0045] Based on the attitude of the robotic arm of the small - spline robot at the origin, calculate the spline placement coordinates according to the small - spline quadrilateral size.

[0046] Exemplarily, calculate the origin P4 through the matrix equation Then calculate the attitude of the origin P4 through the rotation matrix [P4P1, P4P3, P4Z] and the OrientZYX Euler - angle conversion function, and finally calculate the spline placement coordinates according to the small - spline quadrilateral size.

[0047] Preferably, the robotic arm of the small - spline robot has 6 motion axes, namely the rotary axis, the vertical - arm axis, the cross - arm axis, the wrist axis, the wrist - swing axis, and the wrist - transmission axis. Among them,

[0048] The motion range of the rotary axis is [+170°, - 170°], and the maximum speed is 100° / s;

[0049] The movement range of the vertical arm axis is [+85°, -65°], and the maximum speed is 90° / s;

[0050] The movement range of the cross arm axis is [+70°, -180°], and the maximum speed is 90° / s;

[0051] The movement range of the wrist axis is [+300°, -300°], and the maximum speed is 170° / s;

[0052] The movement range of the wrist swing axis is [+130°, -130°], and the maximum speed is 120° / s;

[0053] The movement range of the wrist transmission axis is [+360°, -360°], and the maximum speed is 190° / s.

[0054] Preferably, the automatic sequence of the small spline conveyor line includes:

[0055] Sequence 0: Stopped state;

[0056] Sequence 1: Initialization;

[0057] Sequence 2: The blocking cylinder drops to block the material;

[0058] Sequence 3: Standby state;

[0059] Sequence 4: Scrap release;

[0060] Sequence 5: Spline positioning.

[0061] Preferably, the automatic sequence of the small spline robot includes:

[0062] Sequence 0: Stopped state;

[0063] Sequence 1: The robot returns to the zero position;

[0064] Sequence 2: Hopper positioning;

[0065] Sequence 3: Wait for material grasping;

[0066] Sequence 4: Surface engraving;

[0067] Sequence 5: Side engraving;

[0068] Sequence 6: Spline packing.

[0069] Further, the preset rule is to place the small splines in the spline box in a cross shape from bottom to top.

[0070] It should be noted that the splines in the spline box may be regularly stacked or irregularly stacked.

[0071] Specifically, for the regularly stacked ones, the splines are placed in a cross shape in the spline box and laid flat from bottom to top.

[0072] Exemplarily, the slow-cooling splines are randomly placed in the slow-cooling spline hopper. After the placement reaches a certain value, the entire slow-cooling spline hopper is lifted and transported into the slow-cooling box.

[0073] The present invention is based on a robot automatic sampling pipeline system used in the steel production and manufacturing process. The system includes a large spline robot and a small spline robot. The robotic arm of the large spline robot uses a steel spline grasping method of the present invention to grab the large splines on the spline conveyor in the spline pit to the large spline conveyor line through an electromagnet. The robotic arm of the small spline robot places the cut small splines into the corresponding spline boxes according to certain rules by using a steel spline packing and positioning method and system. The robotic arms of the large spline robot and the small spline robot are both operated under the guidance of machine vision.

[0074] In addition, the moved large splines need to be transported forward by a feeding mechanism to the laser marking station for sample number marking. The laser marking position is in the square area (marking area 100*100mm) near one end on the front or bottom surface of the spline. The marking form is white primer and black font, and a QR code is marked at the same time. The marked content is data sent by the superior system. And the marked content is stored. After the marking is completed, the feeding mechanism continues to feed to the hydraulic shear station and feeds according to the size data sent by the automatic sampling pipeline system. The feeding mechanism is linked with the hydraulic shear to cut the small splines according to the agreed size. After the cut small splines come out of the hydraulic shear, when the sensor on the conveyor belt detects that the small splines reach the preset position, the small spline robot takes out the small splines from the small spline conveyor belt and puts them into the corresponding spline boxes. After the cutting of the entire large spline is completed in sequence and cyclically, in the case of longer waste materials, the hydraulic shear also cuts the waste materials so that the length of the waste materials is less than or equal to the preset value (for example, 1.2m). The remaining waste materials are transported by the conveyor belt to the waste box located at the head of the conveyor belt and automatically fall into the waste box according to gravity. The waste box is a pit below the ground, and the waste box is placed inside. The distance from the bottom of the waste box to the elevation of the conveyor belt should be greater than the length of the waste materials to ensure that the waste materials can be completely put into the waste box.

[0075] The present invention discloses a steel spline packing and positioning system, including:

[0076] An acquisition module for acquiring the original image of the steel small splines at the preset position;

[0077] A packing and positioning module for performing steel spline packing and positioning according to the method described in any one of the above.

[0078] The system embodiments can be implemented corresponding to the foregoing method embodiments one by one, and will not be elaborated herein.

[0079] In a preferred embodiment, the present invention discloses a method for grasping a steel spline, including:

[0080] Step 1: After performing color space conversion on a specified area of the original image of the steel spline, output a grayscale image of the specified color channel;

[0081] In a preferred embodiment, the original image of the steel spline is taken from any one of a camera memory card, a local memory, and a software development kit.

[0082] Exemplarily, a camera is installed on the robotic arm of a large spline grasping robot to perform real-time image capture to obtain the original image of the steel spline.

[0083] In a preferred embodiment, the content of the color space conversion is at least one of grayscale, HSV, HSI, and YUV spaces.

[0084] HSV (Hue, Saturation, Value) is a color space created based on the intuitive characteristics of colors, also known as the Hexcone Model. The HSV color model refers to a visible photon set in the three-dimensional color space of H, S, and V, which contains all colors in a certain color domain.

[0085] The HSI [Hue-Saturation-Intensity (Lightness), HSI or HSL] color model describes the color characteristics with three parameters of H, S, and I. Among them, H defines the frequency of the color, called the hue; S represents the depth of the color, called the saturation; I represents the intensity or brightness. In the double hexagonal pyramid representation of the HSI color model, I is the intensity axis, and the angular range of the hue H is [0, 2π]. Among them, the angle of pure red is 0, the angle of pure green is 2π / 3, and the angle of pure blue is 4π / 3.

[0086] YUV is a type of true-color color space. Proper nouns such as Y’UV, YUV, YCbCr, and YPbPr can all be called YUV, and there is overlap among them. "Y" represents luminance, and "U" and "V" represent chrominance and concentration.

[0087] Step 2: Use a target detection algorithm to find the target position in the grayscale image of the specified color channel and perform target classification;

[0088] Preferably, the target detection algorithm is any one of the YOLO series and the RCNN series.

[0089] Specifically, the object detection algorithms can be as follows: 1. Traditional object detection algorithms: VJ, HOG, DPM Detector; 2. Deep learning two-stage object detection algorithms: RCNN, SPPNet, Fast RCNN, Faster RCNN; 3. Algorithms for object detection tricks: FPN, Cascade RCNN; 4. Deep learning one-stage object detection algorithms: Yolo v1, v2, v3, v4, v5, X, SSD, RetinaNet; 5. Deep learning anchor-free object detection algorithms: CornerNet, CenterNet, FCOS; 6. Transformer-based object detection algorithms: DETR.

[0090] Step 3: Use an image segmentation model to detect and segment the result of object classification to obtain an image of a steel spline.

[0091] Further, the image segmentation model is any one of SegNet, DeepLab, Mask R-CNN, U-Net, and Gated SCNN.

[0092] Specifically, image segmentation refers to dividing an image into several non-overlapping regions according to features such as grayscale, color, spatial texture, and geometric shape, so that these features show consistency or similarity within the same region and obvious differences between different regions. Simply put, it is to separate the object from the background in an image. In the present invention, a pre-trained image segmentation model is used, and then parameters are configured to detect foreground objects.

[0093] SegNet is an architecture based on a deep encoder and decoder, also known as semantic pixel segmentation. It includes performing low-dimensional encoding on the input image, and then using the direction invariance ability in the decoder to restore the image. A segmented image is then generated at the decoder end.

[0094] DeepLab performs image segmentation while helping to control the extraction of image signals to reduce the number of samples and the amount of data that the network has to process, enabling multi-scale context feature learning to aggregate features from images at different scales. DeepLab uses ImageNet pre-trained ResNet for feature extraction. In addition, DeepLab uses dilated convolutions instead of regular convolutions, and different dilation rates for each convolution enable the ResNet block to capture multi-scale context information.

[0095] Mask R-CNN mainly consists of three parts: the input layer, the convolutional layer, and the pooling layer. The input layer converts the image into vector form, the convolutional layer processes the vector, and the pooling layer is used to extract the features of the convolutional layer. In the pooling layer, the mask represents the weight of each channel, while the logits represent the weighted average of all channels.

[0096] U-net includes: the input layer, the core layer, and the output layer. The input layer converts the image into vector form, the core layer converts the vector into an instance, and the output layer classifies the instance according to predefined labels.

[0097] Gated SCNN uses two streams of image classification and contour prediction in the network, and uses the idea of gated, which greatly improves the efficiency and accuracy of image segmentation.

[0098] Furthermore, the detection and segmentation of the foreground target by using the image segmentation model for the result of target classification includes: using a position correction tool to correct the target motion offset and assist in precise positioning for the region of interest in the target classification result, and then using the image segmentation model to detect and segment the region of interest to obtain the foreground target.

[0099] Region of Interest: In machine vision and image processing, the area that needs to be processed outlined in the form of a rectangle, circle, ellipse, irregular polygon, etc. from the processed image is called the Region of Interest ROI. Various operators (Operator) and functions are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the Region of Interest ROI and perform the next step of image processing.

[0100] Specifically, a benchmark for position offset is established according to the matching points and the angle of the matching box in the template matching result, and then the coordinate rotation offset of the ROI detection box is realized according to the relative position offset between the running point and the reference point in the feature matching result, that is, to make the ROI area able to follow the changes of the image angle and pixels.

[0101] Step Four: Perform BLOB analysis on the steel spline image to obtain the camera coordinates of the steel spline;

[0102] Specifically, Blob analysis in computer vision is to perform connected component extraction and labeling on the binary image after foreground or background separation. Each labeled Blob represents a foreground target, and then some related features of the Blob can be calculated. Its advantage is that through Blob extraction, information about the relevant area can be obtained.

[0103] Step Five: Calibrate and convert the camera coordinates of the steel spline to obtain the robotic arm world coordinates of the large spline gripper;

[0104] Further, the calibration conversion of the steel spline camera coordinates to obtain the manipulator world coordinates of the large spline gripper robot includes: selecting N points on the steel spline image for calibration to obtain the steel spline camera coordinates, and converting the camera coordinates of the N points to the manipulator world coordinates of the large spline gripper robot, where N is a positive integer greater than or equal to 4.

[0105] In a preferred embodiment, the present invention adopts a nine-point calibration method, that is, nine points are selected for calibration. After calculation, such a grasping accuracy is the highest.

[0106] Step six: Grasp the steel spline according to the manipulator world coordinates.

[0107] Further, the grasping of the steel spline according to the manipulator world coordinate system includes: analyzing and calculating the center point coordinates and rotation angle of the steel spline according to the manipulator world coordinate system, and then grasping the steel spline according to the center point coordinates and rotation angle of the steel spline.

[0108] Preferably, the manipulator has six motion axes including a rotary axis, a vertical arm axis, a cross arm axis, a wrist axis, a wrist swing axis, and a wrist transmission axis. Among them,

[0109] The motion range of the rotary axis is [+180°, -180°], and the maximum speed is 75° / s;

[0110] The motion range of the vertical arm axis is [+85°, -60°], and the maximum speed is 50° / s;

[0111] The motion range of the cross arm axis is [+60°, -180°], and the maximum speed is 55° / s;

[0112] The motion range of the wrist axis is [+300°, -300°], and the maximum speed is 100° / s;

[0113] The motion range of the wrist swing axis is [+100°, -100°], and the maximum speed is 100° / s;

[0114] The motion range of the wrist transmission axis is [+360°, -360°], and the maximum speed is 160° / s.

[0115] Preferably, the automatic sequence of the large spline gripper robot includes:

[0116] Sequence 0: Shutdown state;

[0117] Sequence 1: The robot returns to the zero point;

[0118] Sequence 2: Standby state;

[0119] Sequence 3: The ground rail extends;

[0120] Step 4: Take a photo of the sample tail 1;

[0121] Step 5: Take a photo of the sample tail 2;

[0122] Step 6: Pick up the sample bar;

[0123] Step 7: Retract the ground rail;

[0124] Step 8: Buffer the sample bar;

[0125] Step 9: Feed the material to the shearing line.

[0126] Preferably, the present invention further includes: after completing the automatic steps of the large sample bar grabbing robot, entering the automatic steps of the large sample bar conveying line, and the automatic steps of the large sample bar conveying line include:

[0127] Step 0: Shutdown state;

[0128] Step 1: Initialize;

[0129] Step 2: Start the hydraulic shear;

[0130] Step 3: Send the feeding servo to the standby position;

[0131] Step 4: Standby state;

[0132] Step 5: The servo goes to the material taking position;

[0133] Step 6: The electromagnet drops to suck the material;

[0134] Step 7: Quick positioning;

[0135] Step 8: Slow positioning;

[0136] Step 9: Go to the position of the sample bar head;

[0137] Step 10: Cut the sample bar head;

[0138] Step 11: The servo searches for the end of the plate;

[0139] Step 12: Remaining length judgment;

[0140] Step 13: Cut the standard sample bar;

[0141] Step 14: Cut the waste.

[0142] The present invention can enable the robot automatic sampling pipeline system in the steel production and manufacturing process to realize intelligent grabbing of steel sample bars, improve work efficiency, and save labor costs.

[0143] In one embodiment, the present invention may include the following steps:

[0144] Step S102: Identify the target grasping object from the monitoring information, where the above-mentioned monitoring information is obtained by using a depth camera installed on the grasping robot to acquire the real-time video of the target scene.

[0145] Step S104: Determine the position coordinates of the above-mentioned target grasping object in the first space coordinate system of the above-mentioned grasping robot, where the above-mentioned first space coordinate system is a coordinate system established with the base of the robotic arm of the above-mentioned grasping robot as the origin.

[0146] Step S106: Based on the above-mentioned position coordinates, determine the target navigation point in the fourth space coordinate system, where the above-mentioned fourth space coordinate system is a coordinate system established according to the above-mentioned target scene.

[0147] Step S108: Control the above-mentioned grasping robot to move to the above-mentioned target navigation point for grasping.

[0148] In the embodiment of the present invention, the execution subject of the autonomous movement method of the grasping robot provided in the above steps S102 to S108 is the above-mentioned grasping robot. The above-mentioned grasping robot continuously shoots the target scene by using a depth camera in the autonomous cruising state. The data information captured is the above-mentioned monitoring information, and the target grasping object is identified from the monitoring information, and the position coordinates in the camera coordinate system are determined. Through continuous conversion of multiple preset conversion matrices, the position coordinates of the target grasping object in the target scene coordinate system are finally obtained, and the target navigation point is calculated according to the above-mentioned position coordinates, so as to control the grasping robot to move to the above-mentioned target navigation point for grasping operations.

[0149] It should be noted that the above-mentioned monitoring information is obtained by using a depth camera installed on the grasping robot to shoot the target scene; the above-mentioned coordinate systems include: the robotic arm base coordinate system (the first space coordinate system), the camera coordinate system (the second space coordinate system), the robot coordinate system (the third space coordinate system), and the map coordinate system (the fourth space coordinate system). The above-mentioned first space coordinate system is a coordinate system established with the base of the robotic arm of the above-mentioned grasping robot as the origin; the above-mentioned second space coordinate system is a coordinate system established with the above-mentioned depth camera as the origin; the above-mentioned third space coordinate system is a coordinate system established with the central position of the above-mentioned grasping robot as the origin; the above-mentioned fourth space coordinate system is a coordinate system established according to the above-mentioned target scene.

[0150] In an embodiment of the present invention, after the camera recognition is completed, the position of the grasping point in the camera coordinate system can be obtained. The grasping coordinate point in the camera coordinate system can be converted to the robotic arm base coordinate system through the camera and robotic arm calibration parameters. The robotic arm is fixed on the robot, and the coordinates in the robotic arm base coordinate system can be converted to the robot coordinate system through the position parameters. The robot coordinate system also has position coordinates on the map, and finally the grasping point recognized by the camera can be converted to the map coordinate system; the navigation process is based on the map coordinate system, and the grasping process is based on the robotic arm base coordinate system.

[0151] It should also be noted that when the position coordinates are converted between the above-mentioned multiple spatial coordinate systems, the position coordinates in the response coordinate system can be calculated based on the preset conversion matrix. The above-mentioned conversion matrix is determined based on the origin positions of the above-mentioned multiple coordinate systems and the installation positions of the various components of the grasping robot; by continuously converting the positions of the target grasping objects in the above-mentioned multiple spatial coordinates and determining the specific positions in different spatial coordinate systems, the autonomous movement and grasping accuracy of the grasping robot can be improved.

[0152] As an optional embodiment, the grasping robot moves and grasps the overall process diagram, the robot enters the autonomous cruising state, and continuously monitors the surrounding environment information through the depth camera during the movement; if the robot does not recognize the target object, it continues to cruise autonomously; if the target object is recognized, it stops cruising, and calculates the spatial position of the target object grasping point in the robot arm base coordinate system based on the camera recognition result. If the position is within the robot's graspable range, it grasps it, otherwise it further calculates the position coordinates of the target object grasping point in the robot coordinate system and the position coordinates in the target scene coordinate system, and uses the target object as the target. The position coordinates in the scene coordinate system are the center of the circle, and the arm span is the radius to determine the grasping range. Multiple pose points (i.e., initial grasping points) are evenly generated within the grasping range. At the same time, the non-navigable grasping points in the initial grasping points are eliminated according to the obstacle information obtained by the depth camera, and the grasping point with the shortest path is selected from the remaining grasping points as the target navigation point, thereby generating a navigation trajectory and moving autonomously to reach the vicinity of the target object for grasping. During the entire grasping process, the robot can determine whether it has received a stop signal. If it has received it, the robot will stop. If it has not received a stop signal, it will continue to cruise or grasp.

[0153] It should be noted that the positioning of the robot in the map mentioned above uses the amcl (adaptive Monte Carlo Localization) algorithm, and the path planning and robot navigation are completed using the move_base framework.

[0154] Through the embodiments of the present invention, without human participation, after the robot is started, it will enter the patrol state. Once the target object is recognized, the robot will calculate the relative position between itself and the target object, and autonomously plan a path to approach the target object, and perform grasping near the target object. A series of navigation and positioning algorithms such as move_base and amcl can be combined to accurately control the robot to move near the target object and realize the recognition, positioning and grasping functions. This method requires no human intervention, has a fast response speed and high control accuracy.

[0155] In an alternative embodiment, before recognizing the target grasping object from the monitoring information, the method further includes: controlling the grasping robot to perform cruise monitoring processing on the target scene; receiving the monitoring information obtained during the cruise monitoring processing.

[0156] As an alternative embodiment, the monitoring information is obtained in real time through a depth camera installed on the grasping robot. The depth camera can be installed at a position convenient for monitoring operations according to the actual situation, and a second space coordinate system is established at the installation position.

[0157] In an alternative embodiment, determining the position coordinates of the target grasping object in the first space coordinate system of the grasping robot includes: obtaining the second space coordinates of the target grasping object in the second space coordinate system of the depth camera, where the second space coordinate system is a coordinate system established with the depth camera as the origin; using a first preset transformation relation formula to convert the second space coordinates in the second space coordinate system into the first space coordinates in the first space coordinate system; taking the first space coordinates as the position coordinates.

[0158] In the embodiments of the present invention, the spatial position of the target grasping object in the base coordinate system of the robotic arm (the first space coordinate system) is calculated according to the camera recognition result, that is, the position information in the camera coordinate system (the second space coordinate system) is converted into the spatial position in the first space coordinate system. The first preset transformation relation (transformation formula) is as follows:

[0159] PA = TAC PC;

[0160] Wherein, PA represents the coordinates of the grasping point in the base coordinate system of the robotic arm, PC represents the coordinates of the grasping point in the camera coordinate system, and the coordinates are obtained by measurement of the depth camera. TAC represents the transformation matrix from the camera coordinate system to the base coordinate system of the robotic arm, and this matrix can be obtained by means of hand-eye calibration.

[0161] Optionally, PA is used as the position coordinates for subsequent calculation and processing.

[0162] In an alternative embodiment, determining the target navigation point in the fourth spatial coordinate system based on the above position coordinates includes: obtaining a grasping distance based on the above position coordinates, where the grasping distance is the distance between the above position coordinates and the origin of the above first spatial coordinate system; determining the magnitude relationship between the above grasping distance and the arm span length of the manipulator of the above grasping robot; if the above grasping distance is less than the arm span length, using the current position of the above grasping robot as the above target navigation point; if the above grasping distance is greater than the arm span length, determining the third spatial coordinate of the above position coordinates in the third spatial coordinate system and the fourth spatial coordinate of the above position coordinates in the fourth spatial coordinate system, and determining the above target navigation point based on the above third spatial coordinate and the above fourth spatial coordinate.

[0163] As an alternative embodiment, a grasping distance is obtained based on the above position coordinates. If the above grasping distance is less than the arm span length, the current position of the above grasping robot is used as the above target navigation point for grasping operations; if the above grasping distance is greater than the arm span length, the third spatial coordinate of the above position coordinates in the third spatial coordinate system and the fourth spatial coordinate of the above position coordinates in the fourth spatial coordinate system are determined, and the above target navigation point is determined based on the above third spatial coordinate and the above fourth spatial coordinate

[0164] It should be noted that the above grasping distance is the distance between the above position coordinates and the origin of the above first spatial coordinate system, and the determination method is as follows:

[0165] |PA| ≤ thr1;

[0166] where |PA| represents the distance from the grasping point to the origin of the manipulator base coordinate system, and thr1 is the manipulator grasping threshold, which can be determined according to the arm span of the manipulator. For example, the grasping threshold is set to 80% of the arm span.

[0167] In an alternative embodiment, determining the third - space coordinates of the position coordinates in the third - space coordinate system and determining the target navigation point in the fourth - space coordinate system based on the third - space coordinates includes: converting the first - space coordinates in the first - space coordinate system into the third - space coordinates in the third - space coordinate system by using a second - preset transformation relation, where the third - space coordinate system is a coordinate system established with the central position of the grasping robot as the origin; converting the third - space coordinates in the third - space coordinate system into the fourth - space coordinates in the fourth - space coordinate system by using a third - preset transformation relation, where the fourth - space coordinate system is a coordinate system established according to the target scenario; determining the grasping range according to the fourth - space coordinates and the arm - span length; generating a plurality of first initial navigation pose points within the grasping range, and determining the target navigation point from the plurality of first initial navigation pose points.

[0168] In an embodiment of the present invention, calculating the third - space coordinates PB of the grasping point of the target object in the robot coordinate system (the third - space coordinate system) and the fourth - space coordinates PW in the map coordinate system (the fourth - space coordinate system), the second - preset transformation relation and the third - preset transformation relation are as follows:

[0169] Wherein, TBA is the transformation matrix of the base coordinate system of the robotic arm in the robot coordinate system, which is determined by the designed position of the robotic arm in the robot and obtained during the structural design. TWB is the transformation matrix of the robot coordinate system relative to the map coordinate system, which is calculated by the robot odometer. Since the robot moves in a plane, the z - axis coordinate value of the fourth - space coordinate PW is transformed to 0 after transformation, which is equivalent to the projection of the grasping point on the map plane.

[0170] Optionally, after obtaining the fourth - space coordinates PW through transformation, determining the grasping range according to the fourth - space coordinates and the arm - span length; generating a plurality of first initial navigation pose points within the grasping range, and determining the target navigation point from the plurality of first initial navigation pose points.

[0171] As an alternative embodiment, n first initial navigation pose points (pose points) relative to the map coordinate system are uniformly generated within a circular ring with the obtained coordinate PW as the center, the value of thr1 as the outer - circle radius, and the value of thr2 as the inner - circle radius.

[0172] It should be noted that the value of thr2 is less than the value of thr1, and it can be set according to the actual situation. The number n of pose points is a set value, and in actual application, it can be set according to the working - environment situation. If the environment is relatively complex, then n can be set larger (n≥30); if the environment is not complex, then n can be set smaller (15≤n<30). The number of actually generated pose points is normally distributed around n.

[0173] As an alternative embodiment, the pose point (the first initial navigation pose point) sampling method includes three methods: offset sampling, arc sampling, and star sampling.

[0174] Optionally, the offset sampling method includes: generating a sampling point array centered on PW, with the same offset Δ in the x and y directions of the array. The offset is related to thr1 and the number of sampling points n, and the calculation formula is as follows:

[0175] Where the symbol "[]" is the integer part symbol. To ensure the number of samples, a is the number of sampling rows and columns. After sampling, the sampling points outside the circular ring are removed to obtain the above-mentioned first initial navigation pose points.

[0176] Optionally, the arc sampling method includes: generating any number of concentric circles within a circular ring centered on PW. The radii of the concentric circles form an arithmetic progression with thr1 and thr2. Sampling is performed at equal arcs on all concentric circles. Taking the generation of 3 concentric circles as an example, the arc relationship formula is derived as follows:

[0177] Where L is the total arc length and rad is the radian value between two sampling points on the circle.

[0178] Optionally, the star sampling method includes: sending rays outward with PW as the endpoint and sampling on the rays within the circular ring range. The distance d between any two adjacent points on the same ray is equal, and the angle between any two adjacent rays is equal and divisible by 180. Setting the angle as θ, the relationship formula for the sampling point distance d is as follows:

[0179] Where N is the number of rays and M is the number of sampling points on each ray.

[0180] It should be noted that pose is a general term for the position and attitude of a robot. Among them, the position is the position of the sampling point (coordinate value on the map plane), and the attitude is related to the installation angle of the camera on the robot. To make the camera face the target object, that is, the projection of the camera z-axis on the map is parallel to the vector from the camera to the grasping point.

[0181] In an alternative embodiment, determining the target navigation point from the above-mentioned multiple first initial navigation pose points includes: identifying obstacle information from the above-mentioned monitoring information; removing the initial grasping points that cannot be navigated from the above-mentioned multiple first initial navigation pose points based on the above-mentioned obstacle information to obtain multiple second initial navigation pose points; calculating the driving distances between the above-mentioned multiple second initial navigation pose points and the current position of the grasping robot; and taking the second initial navigation pose point with the minimum driving distance as the above-mentioned target navigation point.

[0182] As an alternative embodiment, obstacle information is identified from the above monitoring information. According to the obstacle information, the non-navigable pose points among the first initial navigation pose points are eliminated. If there are navigable poses, the grasping operation is continued; if there are no navigable points, it is determined that there are too many obstacles around the target object to achieve grasping, the target grasping object at the PW position is blocked, and no operation is performed on it later. At the same time, the grasping robot enters the autonomous cruising state.

[0183] Optionally, calculate the driving distances between the above-mentioned multiple second initial navigation pose points and the current position of the above-mentioned grasping robot; use the second initial navigation pose point with the minimum driving distance as the above-mentioned target navigation point, and use move_base to generate a navigation trajectory and perform autonomous movement to approach the target object for grasping.

[0184] In an alternative embodiment, after controlling the above-mentioned grasping robot to move to the above-mentioned target navigation point for grasping, the method further includes: determining whether a stop instruction is received; if the stop instruction is not received, continue to control the above-mentioned grasping robot to perform cruising monitoring processing on the above-mentioned target scene.

[0185] As an alternative embodiment, after the grasping is completed, the robot determines whether a stop operation signal is received. If it is received, the robot stops; if the stop signal is not received, the cruising monitoring process continues.

[0186] Through the above steps, the grasping robot can autonomously approach the grasping object based on visual recognition and position conversion, without human intervention, with a fast response speed and high control accuracy.

[0187] Based on the same inventive concept, the present invention also discloses an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus. The processor can call the logical instructions in the memory to execute a method for positioning and packing steel splines, including:

[0188] Based on the original image of the steel small spline at a preset position, identify and determine the packing category corresponding to the steel small spline;

[0189] According to the packing category, obtain the three-point coordinates of the spline box corresponding to the packing category through a positioning algorithm;

[0190] Based on the three-point coordinates, use the robotic arm of the small spline robot to place the steel small spline into the spline box corresponding to the packing category according to a preset rule code.

[0191] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0192] On the other hand, an embodiment of the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a steel spline packing and positioning method provided by each of the above method embodiments, including:

[0193] Based on the original image of the small steel spline at a preset position, identify and determine the packing category corresponding to the small steel spline;

[0194] According to the packing category, obtain the three-point coordinates of the spline box corresponding to the packing category through a positioning algorithm;

[0195] Based on the three-point coordinates, use the robotic arm of the small spline robot to place the small steel spline into the spline box corresponding to the packing category according to a preset rule.

[0196] On another aspect, an embodiment of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute a steel spline packing and positioning method provided by each of the above embodiments, including:

[0197] Based on the original image of the small steel spline at a preset position, identify and determine the packing category corresponding to the small steel spline;

[0198] According to the packing category, obtain the three-point coordinates of the spline box corresponding to the packing category through a positioning algorithm;

[0199] Based on the three-point coordinates, use the robotic arm of the small spline robot to place the small steel spline into the spline box corresponding to the packing category according to a preset rule.

[0200] It should be understood that although the steps in the flowchart of the accompanying drawings are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0201] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for positioning steel splines in a packing case, characterized in that, Including: Based on the original image of the steel small spline at a preset position, identify and determine the packing category corresponding to the steel small spline; According to the packing category, obtain the three-point coordinates of the spline box corresponding to the packing category through a positioning algorithm; Based on the three-point coordinates, use the robotic arm of the small spline robot to place the steel small spline into the spline box corresponding to the packing category according to a preset rule.

2. The method for positioning and packing steel splines as claimed in claim 1, wherein, The packing categories include: sample delivery box, sample retention box, slow cooling box.

3. The method for positioning a steel spline in a packing case according to claim 2, characterized in that, The obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm includes: fixing the positions of each spline box in a way that the box foot angle steel corresponds to the ground slot, and based on the coordinates of the ground slot, obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm.

4. The method for positioning a steel spline in a packing case according to claim 3, characterized in that, The obtaining the three-point coordinates of the spline box corresponding to the packing category through the positioning algorithm includes: the electromagnetic chuck thimble on the robotic arm of the small spline robot collides through the positioning algorithm to obtain the three-point coordinates of the spline box corresponding to the packing category.

5. The method for packing and positioning a steel spline according to claim 4, characterized in that, The positioning algorithm is the amcl algorithm.

6. The method for positioning a steel spline during boxing as described in claim 5, characterized in that, The three-point coordinates include the first point coordinate, the second point coordinate, and the third point coordinate. Among them, the first point coordinate and the second point coordinate are on the first plane of the spline box, the third point coordinate is on the second plane of the spline box, and the first plane is perpendicular to the second plane.

7. The method for positioning a steel spline in a packing case according to claim 1, wherein, The path planning and navigation of the robotic arm of the small spline robot are completed using the move_base framework.

8. The method for positioning and packing steel splines as described in claim 7, characterized in that, The based on the three-point coordinates, using the robotic arm of the small spline robot to place the steel small spline into the spline box corresponding to the packing category according to a preset rule includes: Taking the line connecting the first point and the second point as the X-axis, and making the perpendicular line from the third point to the X-axis as the Y-axis, and the intersection of the X-axis and the Y-axis is the origin; Calculating the origin coordinates through a matrix equation; Based on the origin coordinates, calculating the pose of the robotic arm of the small spline robot at the origin through a rotation matrix and an Euler angle conversion function; Based on the pose of the robotic arm of the small spline robot at the origin, calculating the spline placement coordinates according to the size of the small spline square grid.

9. The method for positioning and packing steel splines as described in claim 1, wherein The preset rule is to place the small splines in a cross shape from bottom to top in the spline box.

10. A steel spline packing and positioning system, characterized in that, Including: An acquisition module, used to acquire the original image of the steel small spline at a preset position; A packing positioning module, used to perform steel spline packing positioning according to the method described in any one of claims 1-9.

Images