A template grabbing and positioning system based on 3D vision technology

Abstract

The application relates to the technical field of steel industry sample grabbing, and discloses a sample grabbing and positioning system based on 3D vision technology, which comprises a data acquisition end, a central server and a control system. The data acquisition end comprises a first processing module, the central server comprises a second processing module, the first processing module communicates with a plurality of laser line scanning camera devices, can acquire sample surface point cloud data in a material basket in real time, and can collect sample upper surface data, the second processing module can pre-process collected point cloud data, process height data of the point cloud data, and pre-process point cloud height data. The sample grabbing and positioning system based on the 3D vision technology can accurately position the sample, reduces the accident rate, improves production efficiency and product quality, perfects the quality tracking process, simplifies the work content, can effectively improve the sample grabbing efficiency, and has very important significance for smoothly grabbing the sample.

Description

Technical Field

[0001] This invention relates to the field of sample grasping technology in the steel industry, specifically a sample grasping and positioning system based on 3D vision technology. Background Technology

[0002] During the process of the robotic arm grasping the sample, if the position is not precise enough, the electric magnet may not be able to hold the sample firmly, which may easily cause accidents such as the sample falling.

[0003] To reduce the accident rate, improve production efficiency and product quality, and perfect the quality tracking process, the production line urgently needs an intelligent template surface precision positioning device equipped with machine vision and machine learning systems. In order to achieve good positioning results for templates of common thicknesses, a template grasping and positioning system based on 3D vision technology is proposed to solve the above problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a template grasping and positioning system based on 3D vision technology. This system has advantages such as good positioning effect on templates of common thicknesses. It solves the problem that in the process of a robotic arm grasping a template, the electric magnet cannot hold the template firmly due to insufficient positioning, which can easily lead to accidents such as the template falling.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a template grasping and positioning system based on 3D vision technology, comprising a data acquisition terminal, a central server and a control system, wherein the data acquisition terminal includes a first processing module and the central server includes a second processing module.

[0006] Preferably, the first processing module communicates with multiple laser line scanning camera devices to acquire point cloud data of the sample surface in the material basket in real time, and collects data from the sample surface.

[0007] Preferably, the second processing module preprocesses the collected point cloud data, processes the height data of the point cloud data, and performs preprocessing of the point cloud height data.

[0008] Preferably, the control system sends the detection result coordinates to control the production equipment, operates the robotic arm to grab the sample, and performs coordinate and offset angle positioning.

[0009] Preferably, the specific steps of the first processing module in acquiring the point cloud data of the sample surface in the material basket in real time are as follows: S8.1 Prepare the laser camera and set it up: Install and connect the laser camera, and ensure that it communicates normally with the computer and control system; S8.2 Positioning and Adjusting the Camera: Position the laser camera appropriately above or to the side of the scanning area to ensure that the camera can fully see the surface of the steel plate to be scanned; S8,3 Emit laser and scan: Start the laser camera and trigger the laser emission and scanning process through the camera's software and control interface; S8,4 Data Processing and Point Cloud Generation: The reflection data collected by the laser camera will be processed and analyzed to generate surface point cloud data; S8.5 Export and Visualization: The generated surface point cloud data can be exported to other software and formats, such as PLY and OBJ.

[0010] Preferably, the specific methods for preprocessing the height data in the collected point cloud data by the second processing module can be the ToMat method, the Find method, and the FindAndToMat method.

[0011] Preferably, the ToMat method converts point cloud data into a binary image.

[0012] Preferably, the FindAndToMat method first calls the Find method to find the outline, and then calls the ToMat method to convert the result into a Mat object.

[0013] Preferably, the specific steps used in the Find method are as follows: S9.1 Based on the given parameters, use the GetAveData method to perform average processing on the point cloud data; S9.2 Then, the SubtractPC method is used to find the points with significant differences and store them in the Contours list; S9 and 3 finally return the results.

[0014] Preferably, the specific steps of the ToMat method to convert point cloud data into a binary image are as follows: S10,1 First, calculate the width and height of the image based on the size of the data and the frame size; S10, 2 Then create a Mat object of the appropriate size, iterate through the points in the Contours list, and set their corresponding positions in the image to white; S10 and 3 finally return the generated binary image.

[0015] Compared with the prior art, the present invention provides a template grasping and positioning system based on 3D vision technology, which has the following beneficial effects: This 3D vision-based sample grasping and positioning system can accurately locate the sample, reduce the accident rate, improve production efficiency and product quality, and improve the quality tracking process. It greatly simplifies the work and can effectively improve the efficiency of sample grasping, which is of great significance for successfully grasping the sample. Attached Figure Description

[0016] Figure 1 This is a network topology diagram of the present invention; Figure 2 This is a block diagram of the present invention; Figure 3 This is a flowchart illustrating the real-time acquisition of point cloud data of the sample surface in the material basket according to the present invention. Figure 4 This is a flowchart of the ToMat method of the present invention; Figure 5 This is a flowchart illustrating how the present invention finds contours from point cloud data and converts them into binary images. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1-5 A template grasping and positioning system based on 3D vision technology includes a data acquisition terminal, a central server, and a control system. The data acquisition terminal includes a first processing module, and the central server includes a second processing module.

[0019] Specifically, the first processing module communicates with multiple laser line scanning camera devices, sets the laser camera according to the movement speed of the robotic arm on the production line, acquires point cloud data of the sample surface in the material basket in real time, and collects data from the sample surface.

[0020] Furthermore, the laser camera detects the height along the z-axis from 980mm-400mm to 980+400mm using a laser beam. The baseline distance along the x-axis is 510mm, with a close range of 300mm and a far range of 720mm. The y-axis range is generated by the direction of the laser camera's movement, theoretically due to motion. At a default frequency of 500Hz, the operating speed is 11.25mm / s, meaning it travels 112.5mm in 10 seconds. Within this timeframe, the x, y, and z axes generated by the motion form a 3D cube, using a blue semiconductor laser that is visible to the naked eye. The camera then writes back the scanned data, saving each received data point to create a complete image of the object.

[0021] Furthermore, the data generated by the laser camera during movement and the data generated when it is stationary—even when stationary, the laser will still trigger and generate data—all need to be processed and discarded.

[0022] Furthermore, the timing of starting and stopping the camera will affect the generated data, typically resulting in a positive or negative error of 50 lines of data, caused by network circuit communication delays.

[0023] Furthermore, laser cameras are commonly used for 3D scanning and measurement, including scanning steel plates and generating surface point cloud data.

[0024] Specifically, the second processing module preprocesses the collected point cloud data, processes the height data of the point cloud data, and performs preprocessing on the point cloud height data, which includes height data and grayscale data.

[0025] Specifically, the control system sends the coordinates of the detection results to the control production equipment, operates the robotic arm to grab the template, and performs coordinate and offset angle positioning.

[0026] Specifically, the first processing module performs the following steps for real-time acquisition of point cloud data of the sample surface in the material basket: S8.1 Preparing the laser camera and setting it up: Install and connect the laser camera, and ensure that it communicates normally with the computer and control system; according to the camera model and specifications, perform the corresponding settings and calibrations, and set the scanning area, exposure time, and laser power; S8.2 Positioning and Adjusting the Camera: Place the laser camera appropriately above or to the side of the scanning area to ensure that the camera can fully see the surface of the steel plate to be scanned, and adjust the position, angle and focal length parameters of the camera as needed; S8,3 Emit and Scan Laser: Start the laser camera and trigger the laser emission and scanning process through the camera's software and control interface. The laser camera will scan the surface of the steel plate with the laser beam and record the reflection data returned after the laser interacts with the surface. S8.4 Data Processing and Point Cloud Generation: The reflection data acquired by the laser camera will be processed and analyzed to generate surface point cloud data; this involves measuring and calculating the reflection and scattering of the laser beam to determine the three-dimensional coordinates and topological information of the surface. The processing includes algorithms such as data filtering, registration, and triangulation. S8.5 Export and Visualization: The generated surface point cloud data can be exported to other software and formats, such as PLY and OBJ; and various 3D visualization software and tools can be used to view, analyze and process the point cloud data to obtain the required information and measurement results.

[0027] Specifically, the second processing module can use the ToMat method, the Find method, and the FindAndToMat method to preprocess the height data in the collected point cloud data.

[0028] Furthermore, the ToMat method converts point cloud data into a binary image Mat object.

[0029] Furthermore, the FindAndToMat method first calls the Find method to find the outline, and then calls the ToMat method to convert the result into a Mat object.

[0030] Furthermore, an XYZ class is used, which contains a `XYZ` class definition that is not shown in the provided code snippet, but it is used as a structure to store the coordinates and difference values ​​of the points; according to the logic of the code above, the goal of the FindContours class is to find contours from the point cloud data and convert the contour points into binary image Mat objects.

[0031] Furthermore, the specific steps for using the Find method are as follows: S9.1 Based on the given parameters, use the GetAveData method to perform average processing on the point cloud data; S9.2 Then, the SubtractPC method is used to find the points with significant differences and store them in the Contours list; S9 and 3 finally return the results.

[0032] Furthermore, the specific steps of the ToMat method to convert point cloud data into a binary image Mat object are as follows: S10,1 First, calculate the width and height of the image based on the size of the data and the frame size; S10, 2 Then create a Mat object of the appropriate size, iterate through the points in the Contours list, and set their corresponding positions in the image to white pixels with a value of 255; S10 and 3 finally return the generated binary image.

[0033] Specifically, when using the FindAndToMat method, you should call the Find method to find the contour, and then call the ToMat method to convert the found contour into a binary image Mat object and return it.

[0034] Furthermore, the specific steps for finding contours from point cloud data and converting them into binary images are as follows: S1. Segment the point cloud data and convert it to Mat; S2. Binarization processing greatly reduces the amount of data, making it easier to extract. S3. Find the contour and extract the contour using OpenCVSharp. S4. Find the center coordinates XY of the steel plate. The center of the contour can be obtained directly through Moments. S5. Finally, Y is obtained by using the height value recorded in the point cloud data. Since the data is stored in z corresponding to xy, we only need to swap xy to get z. Then, according to the formula, we substitute it to obtain the actual distance of z, which is 980-(Z-32768)*16 / 1000mm.

[0035] Furthermore, for the detection of tilt angles: taking 3 points on the plane, which can be the 4 corners of the upper contour, we can obtain the plane normal vector, and then calculate the angle between the normal vector and the horizontal plane; the above perfectly realizes the processing of point cloud data, which is greatly reduced to a two-dimensional plane, and the extracted xy is used to find z. Otherwise, a lot of complex calculations would be required on the point cloud data, and the response time would be completely unsuitable for generation efficiency.

[0036] In summary, this 3D vision-based sample grasping and positioning system can accurately locate the sample, reduce the accident rate, improve production efficiency and product quality, and improve the quality tracking process. It greatly simplifies the work and effectively improves the efficiency of sample grasping, which is of great significance for successfully grasping samples. It solves the problem that in the process of robotic arms grasping samples, insufficient positioning can cause the electric magnets to fail to hold the sample firmly, leading to accidents such as sample drops.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A template grasping and positioning system based on 3D vision technology, characterized in that: The system includes a data acquisition terminal, a central server, and a control system. The data acquisition terminal includes a first processing module, and the central server includes a second processing module. The second processing module preprocesses the acquired point cloud data, processes the height data of the point cloud data, and performs preprocessing of the point cloud height data. The second processing module uses the FindAndToMat method to preprocess the height data in the collected point cloud data. The FindAndToMat method first calls the Find method to find the outline, and then calls the ToMat method to convert the result into a Mat object; The specific steps for finding contours from point cloud data and converting them into binary images are as follows: S1. Segment the point cloud data and convert it to Mat; S2, Binarization; S3. Find the contour and extract the contour using OpenCVSharp. S4. Find the XY coordinates of the center of the steel plate and obtain the center of the contour using Moments. S5. Obtain the height value Z by swapping the xy coordinates of the point cloud data, and calculate the true distance along the Z axis based on the formula 980-(Z-32768)*16 / 1000mm. The specific steps used in the Find method are as follows: S9.1 Average the point cloud data according to the given parameters; S9,2 Then find the points with significant differences and store them in the Contours list; S9.3 finally returns the result; The ToMat method converts point cloud data into a binary image. The specific steps of the ToMat method to convert point cloud data into a binary image are as follows: S10,1 First, calculate the width and height of the image based on the size of the data and the frame size; S10, 2 Then create a Mat object of the appropriate size, iterate through the points in the Contours list, and set their corresponding positions in the image to white; S10 and 3 finally return the generated binary image.

2. The template grasping and positioning system based on 3D vision technology according to claim 1, characterized in that: The first processing module communicates with multiple laser line scanning camera devices to acquire point cloud data of the sample surface in the material basket in real time, and collects data from the sample surface.

3. The template grasping and positioning system based on 3D vision technology according to claim 1, characterized in that: The control system sends the coordinates of the detection results to control the production equipment, operates the robotic arm to grab the sample, and performs coordinate and offset angle positioning.

4. The template grasping and positioning system based on 3D vision technology according to claim 2, characterized in that: The specific steps of the first processing module in acquiring the point cloud data of the sample surface in the material basket in real time are as follows: S8.1 Prepare the laser camera and set it up: Install and connect the laser camera, and ensure that it communicates normally with the computer and control system; S8.2 Positioning and Adjusting the Camera: Position the laser camera appropriately above or to the side of the scanning area to ensure that the camera can fully see the surface of the steel plate to be scanned; S8,3 Emit laser and scan: Start the laser camera and trigger the laser emission and scanning process through the camera's software or control interface; S8,4 Data Processing and Point Cloud Generation: The reflection data collected by the laser camera will be processed and analyzed to generate surface point cloud data; S8.5 Export and Visualization: Export the generated surface point cloud data to other software and formats.

Images