A multi-view binocular waste metal size measurement and discrimination method, device and system

Abstract

The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, which comprises a multi-view binocular camera array formed by arranging multiple binocular cameras, a suction disc is moved to the center of the field of view of the multi-view binocular camera array after the suction disc is filled with waste metals, multiple binocular cameras simultaneously take two-dimensional images to generate images under multiple views, waste metals are identified, three-dimensional reconstruction is performed, three-dimensional point clouds are filtered to remove outlier point clouds, point clouds under different views of the same waste metal are target matched, point clouds under different views of the same waste metal are spliced, the maximum thickness of each waste metal is obtained, if the maximum thickness is greater than a threshold value, the suction disc filled with waste metals is sent to a gas cutting machine for processing and then to a gantry shearing machine for processing, and if the maximum thickness is less than the threshold value, the suction disc is directly sent to the gantry shearing machine for processing, and the application can replace manual operation, quickly measure the size of waste metals, and improve the recycling efficiency of waste metals.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of waste metal visual recognition, and particularly relates to a multi-view binocular waste metal size measurement and discrimination method, device and system. BACKGROUND

[0002] In recent years, the cost of iron ore is high; waste metal is an important material raw material of metal. At present, especially in the steelmaking industry, waste steel is used as raw material for steelmaking to meet the requirement of "precious material into the furnace", that is, the size of the waste steel into the furnace is required. The oversized raw material needs to be sorted and crushed, and a gantry shear cutting machine is generally used for crushing. However, the waste steel that is too thick will cause the gantry shear cutting machine to be stuck, greatly affecting the work efficiency, damaging the mechanical equipment and causing great loss. In the waste steel recovery link of large steel plants, most of them still use manual discrimination to grade waste steel. However, due to the cognitive differences between different employees and the influence of subjective state factors such as employee energy, the discrimination results in the waste steel recovery process will also deviate and fluctuate.

[0003] In the waste metal recovery link of large recycling plants, especially in the recovery of waste steel, it is gradually changed from the traditional manual discrimination method to the intelligent grading method based on deep learning. For example, a deep learning or machine learning method is used to extract waste steel samples, develop a discrimination algorithm, and directly predict the discrimination result after deployment. However, this method has high requirements for the shape of different types of waste steel, and the generalization ability is poor when copying between different deployment points. That is, the existing waste steel recognition scheme has low recognition accuracy for the thickness of waste steel, resulting in low waste steel recovery efficiency. SUMMARY

[0004] In view of the deficiencies of the prior art, the application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, which can replace manual work, quickly measure the size of waste metal and improve the waste metal recovery efficiency.

[0005] In order to solve the above technical problems, the technical scheme adopted by the application is as follows:

[0006] A multi-view binocular waste metal size measurement and discrimination method comprises the following steps:

[0007] Step 1, a multi-view binocular camera array composed of multiple binocular cameras is set, and waste metal is moved to the center of the field of view of the multi-view binocular camera array by a suction cup;

[0008] Step 2, when the suction cup enters the center of the field of view, multiple binocular cameras simultaneously take two-dimensional images to generate images under multiple viewing angles;

[0009] Step 3, scrap metal recognition and segmentation under multiple views, preliminary identification of over-thick scrap metal;

[0010] Step 4, three-dimensional reconstruction of the over-thick scrap metal segmented under different views;

[0011] Step 5, filtering of three-dimensional point cloud, removing outlier point cloud;

[0012] Step 6, target matching of point clouds under different views of the same scrap metal, according to the camera pose relationship calibrated when the binocular camera is installed, the point clouds under different views of the same scrap metal are spliced;

[0013] Step 7, obtaining the length and width of each scrap metal point cloud, forming a length-width plane, drawing a perpendicular line perpendicular to the length-width plane, and taking the length between the intersection points of the perpendicular line and the front and back surfaces of the scrap metal point cloud as the thickness of the scrap metal;

[0014] Step 8, obtaining the maximum thickness of each scrap metal, if the maximum thickness is greater than the threshold, the suction cup is full of scrap metal and is sent to the gas cutting machine for processing, after processing by the gas cutting machine, it is sent to the gantry shearing machine for processing, if it is less than the threshold, it is directly sent to the gantry shearing machine for processing.

[0015] Further, in step 1, the multi-view binocular camera array includes at least four binocular cameras, when four binocular cameras are used, the four binocular cameras are arranged in a square shape to form four views, and the optical axes of adjacent binocular cameras are perpendicular to each other.

[0016] Further, in step 1, the binocular camera pose relationship under different views is calibrated by two cameras of adjacent views, the conversion relationship between the coordinate systems of the binocular cameras of adjacent views is obtained, and then according to the conversion relationship between the adjacent views, the coordinate systems of the other binocular cameras are converted into the reference coordinate system through a rotation matrix and a translation matrix, and the coordinate transformation of the adjacent binocular cameras satisfies the following formula:

[0017]

[0018] The conversion is:

[0019] P B =RP A +t

[0020] Where X B , Y B , Z B are the point cloud coordinates in the reference coordinate system; X A , Y A , Z A are the point cloud coordinates in the coordinate system of the other binocular camera; r i,jt i are parameters within the rotation matrix and translation matrix, 1≤i≤3, 1≤j≤3; R, T are the rotation matrix and translation matrix; P B ,P A are the system coordinate system and the non-system coordinate system.

[0021] Further, in step 3, a large number of images are collected by the multi-view binocular camera array in step 3, an instance segmentation data set is made, a deep learning model is established, the instance segmentation data set is trained, the deep learning model is segmented, the weight of the deep learning model is obtained, and then the model freezing graph is output and deployed to the terminal. The images collected by the four views are recognized and segmented.

[0022] Further, the step 4 specifically includes:

[0023] Step 401, after arranging a plurality of binocular cameras, calibrate each binocular camera to obtain intrinsic and extrinsic parameters, and perform distortion removal and epipolar correction on the photographed pictures;

[0024] Step 402, for a binocular camera, a space point (X w ,Y w ,Z w ) is projected on the photographic plane of the left and right cameras to form P l (x l ,y l ) and P r (x r ,y r ), according to the method of triangulation, the following formula is obtained:

[0025]

[0026] After conversion, we get:

[0027]

[0028] Further conversion gives:

[0029]

[0030] For X w and Y w , we have:

[0031]

[0032]

[0033] where T is the epipolar line, representing the distance between the optical centers of the left and right cameras on the X axis; f is the focal length, d is the disparity of the left and right cameras, (u l ,vl ) is the pixel coordinate of the projection point on the left camera, c x y is the distance from the optical center to the edge of the plane, Z w w w is the world coordinate corresponding to the left camera pixel point, (x l l r r is the coordinate on the photographic plane coordinate system of the left and right cameras, and the optical center is the origin of the photographic coordinate system.

[0034] Step 403, combined with the parallax map and the calibration parameters, the three-dimensional coordinates of the waste metal under each view angle are obtained, and a three-dimensional point cloud model is obtained.

[0035] Further, in step 6, combined with the point cloud center distance and the image features, the point clouds filtered under multiple views are transformed into the same reference coordinate system, and the center distances between the point clouds are calculated, combined with the point cloud center distance and the image-based twin neural network, the target tracking and matching under multiple views are obtained, and the point clouds of different views of the same waste metal are spliced together.

[0036] Further, in step 7, the OBB oriented bounding box method is used, the three main directions of the point cloud are obtained by PCA principal component analysis method, the centroid is obtained, the covariance is calculated, the covariance matrix is obtained, the eigenvalues and eigenvectors of the covariance matrix are calculated, the eigenvectors are the main directions, the input point cloud is converted to the origin by using the main direction and the centroid, and the main direction is returned to the coordinate system direction, the bounding box of the point cloud transformed to the origin is established, and the length and width of the waste metal are directly obtained by the bounding box.

[0037] A multi-view binocular waste metal size measurement and discrimination device, comprising:

[0038] A multi-view setting module is used to set a multi-view binocular camera array composed of multiple binocular cameras, and the waste metal is moved to the center of the field of view of the multi-view binocular camera array by a suction cup.

[0039] A shooting module is used to take two-dimensional images by multiple binocular cameras when the suction cup enters the center of the field of view, and multiple images under multiple views are generated.

[0040] An identification module is used to identify and segment the waste metal under multiple views, and the over-thick waste metal is preliminarily identified.

[0041] A reconstruction module is used to perform three-dimensional reconstruction on the over-thick waste metal segmented under different views.

[0042] ​​​​​​A filtering module is configured to filter a three-dimensional point cloud to remove outlier point clouds.

[0043] A splicing module is configured to perform target matching on point clouds of the same waste metal under different viewing angles, and splice the point clouds under different viewing angles according to a camera pose relationship calibrated when the binocular camera is installed.

[0044] A thickness identification module is configured to obtain the length and width of each waste metal point cloud, form a length-width plane, draw a perpendicular line of the length-width plane, and intercept a length between intersection points of the perpendicular line and front and back surfaces of the waste metal point cloud as a thickness of the waste metal.

[0045] A determination module is configured to obtain a maximum thickness of each waste metal, and if the maximum thickness is greater than a threshold value, the waste metal is fully sucked by a suction cup and sent to a gas cutting machine for processing, and after processing by the gas cutting machine, the waste metal is sent to a gantry shearing machine for processing, and if the maximum thickness is less than the threshold value, the waste metal is directly sent to the gantry shearing machine for processing.

[0046] A multi-view binocular waste metal size measurement and discrimination system, comprising:

[0047] A suction cup is configured to adsorb the waste metal.

[0048] An image acquisition unit comprises a plurality of binocular cameras configured to form a multi-view binocular camera array.

[0049] A data processing unit comprises a processor and a memory configured to store a computer program capable of running on the processor, and the processor is configured to execute the steps of the multi-view binocular waste metal size measurement and discrimination method.

[0050] A computer storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-view binocular waste metal size measurement and discrimination method.

[0051] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0052] (1) The present application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, a multi-view binocular camera array comprising a plurality of binocular cameras is provided, and the size of the waste metal sucked by the suction cup is determined by the multi-view binocular camera array, and the entire process is highly automated, reducing manual intervention and reducing labor costs.

[0053] (2) The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, the cameras in the multi-view binocular camera array do not need to move, which can avoid the reduction of the service life of the cameras caused by the movement of the cameras in the harsh environment of the factory, avoid the situation of movement collision, and the fixation of the cameras can avoid the system error caused by the movement of the cameras and improve the measurement accuracy.

[0054] (3) The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, which is a batch measurement method, can measure all waste metals on the suction cup at one time, and greatly improves the work efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate the non-limiting exemplary embodiments of the application and serve to explain the application together with the specification. In the drawings:

[0056] Figure 1 It is a workflow diagram of the multi-view binocular waste metal size measurement and discrimination method of the application;

[0057] Figure 2 It is a measurement platform layout diagram;

[0058] Figure 3 It is a four-view camera layout diagram;

[0059] Figure 4 It is a camera layout diagram on the camera support under the same view;

[0060] Figure 5 It is a waste metal point cloud measurement diagram;

[0061] Figure 6 It is a schematic diagram of the multi-view binocular waste metal size measurement and discrimination device of the application.

[0062] Wherein: 1, waste metal yard; 2, multi-view binocular camera array; 21, RGB camera; 3, suction cup; 4, thick waste metal to be processed area; 5, gantry shearing machine. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application.

[0064] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0065] The present application provides a multi-view binocular waste metal size measurement and discrimination method, as shown in the figure, comprising the following steps: Figures 1-3

[0066] Step 1, set up a multi-view binocular camera array 2 composed of multiple binocular cameras, and move the waste metal to the center of the field of view of the multi-view binocular camera array 2 by sucking it full of waste metal with a suction cup 3 in a waste metal yard 1;

[0067] Step 2, when the suction cup 3 enters the center of the field of view, multiple binocular cameras simultaneously take two-dimensional images, generating images under multiple viewing angles;

[0068] Step 3, identify and segment the waste metal under multiple viewing angles, and preliminarily identify the over-thick waste metal;

[0069] Step 4, three-dimensional reconstruction of the over-thick waste metal segmented under different viewing angles;

[0070] Step 5, filter the three-dimensional point cloud to remove outlier point clouds;

[0071] Step 6, target matching of point clouds under different viewing angles of the same waste metal, according to the camera pose relationship calibrated when the binocular camera is installed, the point clouds under different viewing angles of the same waste metal are spliced;

[0072] Step 7, obtain the length and width of each waste metal point cloud to form a length-width plane, draw a perpendicular line perpendicular to the length-width plane, and intercept the length between the intersection points of the perpendicular line and the front and back surfaces of the waste metal point cloud as the thickness of the waste metal;

[0073] ​Step 8, the maximum thickness of each waste metal is obtained, if the maximum thickness is greater than the threshold value, the waste metal is sucked by the suction cup 3 and sent to the thick waste metal processing area 4, after being processed by the gas cutting machine, it is sent to the gantry shearing machine 5 for processing, if it is less than the threshold value, it is directly sent to the gantry shearing machine 5 for processing.

[0074] The application provides a multi-view binocular waste metal size measurement and discrimination method, which can solve the problems of low waste metal size measurement precision and low waste metal recovery efficiency in the waste metal recognition and classification scheme in the prior art.

[0075] The application provides a multi-view binocular waste metal size measurement and discrimination method, which can identify multiple waste metals at a time, has high efficiency, and can accurately identify the thickness of each position of each waste metal, if the maximum thickness is greater than the threshold value, the waste metal is sucked by the suction cup 3 and sent to the gas cutting machine for processing, after being processed by the gas cutting machine, it is sent to the gantry shearing machine 5 for processing, if it is less than the threshold value, it is directly sent to the gantry shearing machine 5 for processing.

[0076] The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, a multi-view binocular camera array 2 composed of multiple binocular cameras is arranged, the size of the thick waste metal in the waste metal sucked by the suction cup 3 is judged by the multi-view binocular camera array 2, the whole process is highly automated, manual intervention is reduced, and the labor cost is reduced.

[0077] The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, the cameras in the multi-view binocular camera array 2 do not need to move, the reduction of the service life of the cameras caused by the movement of the cameras in a poor environment of a factory can be avoided, the situation of movement collision can be avoided, meanwhile, the system error caused by the movement of the cameras can be avoided by fixing the cameras, and the measurement precision is improved.

[0078] The application provides a multi-view binocular waste metal size measurement and discrimination method, device and system, which is a batch measurement method, can measure all the waste metals on the suction cup 3 at a time, and greatly improves the work efficiency.

[0079] In the application, the waste metal is waste iron or waste metal, and the suction cup 3 is an electromagnetic suction cup, so that the waste iron or waste metal can be quickly attracted.

[0080] In step 1, the multi-view binocular camera array 2 includes at least four binocular cameras, and the four binocular cameras can achieve high identification precision, when the four binocular cameras are used, the four binocular cameras are arranged in a square shape, four views are formed, the optical axes of adjacent binocular cameras are perpendicular to each other, and the intersection center of the optical axes of the multiple binocular cameras coincides with the center of the suction cup 3.

[0081] When four binocular cameras or more are used, a plurality of binocular camera annular arrays.

[0082] The application provides a multi-view binocular waste metal size measurement and discrimination method, a plurality of binocular camera annular arrays, which can comprehensively obtain three-dimensional point clouds of waste metals and reduce the influence of waste metal shielding.

[0083] As shown in the figure, Figure 4 In the application, when the binocular cameras are installed, 2 parallel arranged RGB cameras 21 are placed on each camera fixing support to form a binocular camera or an integrated depth camera, the distance between the binocular camera and the suction cup 3 is determined by the size of the suction cup 3, and it is appropriate that the waste metal covers the field of view of the binocular camera.

[0084] In step 1, the pose relationship of the binocular cameras under different viewing angles is calibrated by two cameras of adjacent viewing angles, the conversion relationship of the binocular camera coordinate systems between adjacent viewing angles is obtained, and then according to the conversion relationship between adjacent viewing angles, the coordinate system of the other binocular camera is converted into the reference coordinate system through a rotation matrix and a translation matrix, and the coordinate transformation of the adjacent binocular cameras satisfies the following formula:

[0085]

[0086] The conversion is as follows:

[0087] P B =RP A +t

[0088] Wherein, X B , Y B , Z B are point cloud coordinates in the reference coordinate system; X A , Y A , Z A are point cloud coordinates in the coordinate system of the other binocular camera; r i,j , t i are parameters in the rotation matrix and the translation matrix, 1≤i≤3, 1≤j≤3; R, T are the rotation matrix and the translation matrix; P B , P A are the system coordinate system and the non-system coordinate system.

[0089] The rotation matrix and the translation matrix are calibrated when the camera position is installed, and when the camera position is fixed, the conversion relationship between the world coordinate systems is also determined.

[0090] In the present application, in step 3, a large number of images are collected by the multi-view binocular camera array 2, an instance segmentation data set is made, a deep learning model is established, the instance segmentation data set is used for training, the deep learning model is segmented, the weight of the deep learning model is obtained, and then the model freezing graph is output and deployed to the terminal to recognize and segment the waste metal in the images collected by the four views.

[0091] The present application combines deep learning, inputs a large number of image data sets to a deep learning model for training, makes the deep learning model learn the characteristics of these images, and achieves the purpose of being able to predict waste metal in the images.

[0092] In the present application, the waste metal in the image is recognized by the Yolact algorithm or the Mask-RCNN algorithm.

[0093] In the present application, two RGB cameras form a binocular camera, and the three-dimensional coordinates of the pixel points in the image in the world coordinate system are calculated by the binocular parallax triangulation principle.

[0094] Step 401, after arranging a plurality of binocular cameras, calibrate each binocular camera to obtain internal and external parameters, and perform distortion removal and epipolar correction on the photographed pictures;

[0095] Step 402, for the binocular camera, a space point (X w ,Y w ,Z w ) is projected on the photographic plane of the left and right two cameras to form P l (x l ,y l ) and P r (x r ,y r ), according to the method of triangulation, the following formula is obtained:

[0096]

[0097] After conversion, we get:

[0098]

[0099] Further conversion gives:

[0100]

[0101] For X w and Y w , we have:

[0102]

[0103]

[0104] wherein T is epipolar line, represents the distance of left and right camera optical center on X axis, f is focal length, d is disparity of left and right cameras, (u l ,v l ) is pixel coordinate of projection point on left camera, c x ,c y is the distance of optical center to plane edge, Z w ,X w ,Y w is the world coordinate corresponding to left camera pixel point, (x l ,y l ),(x r ,y r ) is the coordinate on the photographic plane coordinate system of left and right cameras, and the optical center is the origin of the photographic coordinate system;

[0105] In step 403, each pixel point is converted in combination with the disparity map and the calibration parameters, so that the three-dimensional coordinates of the waste metal under each view angle are calculated, and thus the three-dimensional point cloud model is obtained.

[0106] In the application, in step 5, the search radius is set to 15, the number of point clouds contained is 1000, the point cloud filtering is performed, and the clean waste metal dense point cloud is obtained.

[0107] In step 6, the waste metal point cloud splicing method combines the point cloud center of gravity distance and the image features, performs coordinate transformation on the filtered point clouds of multiple view angles, converts into the same reference coordinate system, simultaneously calculates the center of gravity distance between the point clouds, combines the point cloud center of gravity distance and the image-based twin neural network, obtains the target tracking and matching under multiple view angles, and splices the point clouds of different view angles of the same waste metal together.

[0108] In the embodiment of the application, the filtered point clouds of four view angles are subjected to coordinate transformation and converted into the same reference coordinate system.

[0109] In step 7, the OBB oriented bounding box method is adopted, the three main directions of the point cloud are obtained by using the PCA principal component analysis method, the centroid is obtained, the covariance is calculated, the covariance matrix is obtained, the eigenvalue and eigenvector of the covariance matrix are calculated, the eigenvector is the main direction, the input point cloud is converted to the origin by using the main direction and the centroid, the main direction is returned to the coordinate system direction, the bounding box of the point cloud converted to the origin is established, and the length and width of the waste metal are directly obtained through the bounding box.

[0110] Specifically, the covariance matrix is as follows:

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118] Where N is the number of point clouds, x i ,y i ,z i Let μ be the three-dimensional coordinate value of the i-th point. x ,μ y ,μ z This represents the average value of the three-dimensional coordinates of the point cloud.

[0119] The formulas for calculating the semi-axis sizes a, b, and c of the three principal axes of the OBB bounding box are as follows:

[0120]

[0121]

[0122]

[0123] Among them, P i Let P be the i-th point in the point cloud. i ·X,P i ·Y,P i • Z represents the coordinate values ​​of the i-th point in the point cloud in the X, Y, and Z directions.

[0124] like Figure 5 As shown, a perpendicular line is then drawn to the length and width of the plane, and the length between the intersection of this perpendicular line and the front and back of the scrap metal point cloud is taken as the thickness of the scrap metal.

[0125] The thickness measurement method used in this invention is based on the OBB oriented bounding box. Perpendicular lines are drawn to the length and width of the bounding box. The perpendicular lines pass through the front and back of the point cloud to form two intersection points. The distance between the two intersection points is the thickness of the scrap metal. This method has high measurement accuracy.

[0126] In step 8, the threshold is determined by the cutting capacity of the gantry shear. Generally, the width of the gantry shear blade is 1.6m, and the maximum cutting thickness is about 30mm. The threshold is set with a size that is 20% to 50% smaller than the maximum cutting thickness.

[0127] This invention also provides a multi-view binocular scrap metal size measurement and discrimination device, such as... Figure 6 As shown, it includes:

[0128] A multi-view setting module is configured to set a multi-view binocular camera array 2 composed of multiple binocular cameras, and the suction cup 3 is moved to the center of the field of view of the multi-view binocular camera array 2 by suction.

[0129] A shooting module is configured to shoot a two-dimensional image by the multiple binocular cameras when the suction cup 3 enters the center of the field of view.

[0130] An identification module is configured to identify and segment the multiple-view images to preliminarily identify the over-thick waste metal.

[0131] A reconstruction module is configured to reconstruct the over-thick waste metal segmented from different views in three dimensions.

[0132] A filtering module is configured to filter the three-dimensional point cloud to remove outlier point clouds.

[0133] A splicing module is configured to perform target matching on the point clouds of the same waste metal from different views, and splice the point clouds from different views of the same waste metal according to the camera pose relationship calibrated when the binocular camera is installed.

[0134] A thickness identification module is configured to obtain the length and width of each waste metal point cloud, form a length-width plane, draw a perpendicular line perpendicular to the length-width plane, and intercept the length between the intersection points of the perpendicular line and the front and back surfaces of the waste metal point cloud as the thickness of the waste metal.

[0135] A determination module is configured to obtain the maximum thickness of each waste metal, and if the maximum thickness is greater than a threshold value, the suction cup 3 is full of waste metal and is sent to a gas cutting machine for processing, and after processing by the gas cutting machine, the waste metal is sent to a gantry shearing machine 5 for processing, and if the maximum thickness is less than the threshold value, the waste metal is directly sent to the gantry shearing machine 5 for processing.

[0136] A multi-view binocular waste metal size measurement and discrimination device, comprising:

[0137] A suction cup 3 is configured to adsorb waste metal.

[0138] An image acquisition unit includes multiple binocular cameras, and the multiple binocular cameras are configured to form a multi-view binocular camera array 2.

[0139] A data processing unit includes a processor and a memory for storing a computer program capable of running on the processor, and the processor is configured to execute the steps of the multi-view binocular waste metal size measurement and discrimination method described in any one of the above embodiments when running the computer program.

[0140] The suction cup 3 is driven to move by a suction cup 3 control unit.

[0141] The multi-view binocular waste metal size measurement discrimination method disclosed by the embodiments of the present application can be applied to a processor or implemented by a processor. The processor can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the multi-view binocular waste metal size measurement discrimination method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The processor mentioned above can be a general processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute each method, step and logic block diagram disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly embodied to execute the steps, or the hardware and software modules in the decoding processor can be combined to execute the steps. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines the hardware to complete the steps of the multi-view binocular waste metal size measurement discrimination method provided in the embodiments of the present application.

[0142] In the exemplary embodiments, the multi-view binocular waste metal size measurement discrimination device can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general processors, controllers, microcontroller units (MCUs), microprocessors, or other electronic elements, for executing the foregoing method.

[0143] It is to be understood that the memory can be a volatile memory or a nonvolatile memory, and can also include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0144] A computer storage medium, in which a computer program is stored, wherein the computer program is executed by a processor to implement the steps of any one of the multi-view binocular waste metal size measurement discrimination methods.

[0145] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A multi-view binocular scrap metal size measurement discrimination method, characterized in that, Comprise the following steps: Step 1, set up a multi-view binocular camera array composed of multiple binocular cameras, move the waste metal to the center of the field of view of the multi-view binocular camera array by the suction cup; in step 1, the multi-view binocular camera array comprises at least four binocular cameras, when four binocular cameras are used, the four binocular cameras are arranged in a square shape, forming four views, and the optical axes of adjacent binocular cameras are perpendicular to each other; Step 2, when the suction cup enters the center of the field of view, multiple binocular cameras simultaneously take two-dimensional images, generating images under multiple views; Step 3, identifying and segmenting the waste metal under multiple views, and preliminarily identifying the over-thick waste metal; Step 4, three-dimensional reconstruction of the over-thick waste metal segmented under different views; the step 4 specifically comprises: Step 401, after arranging multiple binocular cameras, calibrate each binocular camera to obtain intrinsic and extrinsic parameters, and perform distortion correction and epipolar line correction on the captured pictures; Step 402, for binocular camera, spatial point After perspective projection, form and According to the method of triangulation, the following formula is obtained: After conversion, we get: After further conversion, we get: For and there are: wherein, is the distance of the optical center of the left and right cameras on the X axis; is the focal length, is the parallax of the left and right cameras, is the pixel coordinate of the projection point on the left camera, is the distance of the optical center to the edge of the plane, is the world coordinate corresponding to the pixel point of the left camera, is the coordinate on the photographic plane coordinate system of the left and right cameras, and the optical center is the origin of the photographic coordinate system; Step 403, combined with the disparity map and the calibration parameters, the three-dimensional coordinates of the waste metal under each view are obtained, and thus the three-dimensional point cloud model is obtained; Step 5, filtering the three-dimensional point cloud to remove outlier point clouds; Step 6, matching the point clouds of the same waste metal under different views, according to the camera pose relationship calibrated when the binocular cameras are installed, the point clouds under different views of the same waste metal are spliced; Step 7, obtaining the length and width of each waste metal point cloud, forming a length-width plane, drawing a perpendicular line perpendicular to the length-width plane, and taking the length between the intersection points of the perpendicular line and the front and back surfaces of the waste metal point cloud as the thickness of the waste metal; Step 8, obtaining the maximum thickness of each waste metal, if the maximum thickness is greater than the threshold, the suction cup is full of waste metal and is sent to the gas cutting machine for processing, after processing by the gas cutting machine, it is sent to the gantry shearing machine for processing, if it is less than the threshold, it is directly sent to the gantry shearing machine for processing.

2. The multi-view binocular waste metal size measurement and discrimination method according to claim 1, wherein: In step 1, the binocular camera pose relationship under different views is calibrated by two cameras of adjacent views, the conversion relationship between the coordinate systems of adjacent views is obtained, and then the coordinate systems of other binocular cameras are transformed into the reference coordinate system according to the transformation relationship between adjacent views, taking one of the binocular camera coordinate systems as the reference coordinate system, and the coordinate transformation of adjacent binocular cameras satisfies the following formula: After conversion, we get: wherein, is the point cloud coordinate in the reference coordinate system; is the point cloud coordinate in the coordinate system of the other binocular camera; are parameters within the rotation matrix and the translation matrix, , ; is the rotation matrix and the translation matrix; is the system coordinate system and the non-system coordinate system.

3. The multi-view binocular waste metal size measurement and discrimination method according to claim 1, wherein: In step 3, a large number of images are collected by the multi-view binocular camera array, an instance segmentation data set is made, a deep learning model is established, the instance segmentation data set is trained, the deep learning model is segmented, the weight of the deep learning model is obtained, and then the model freezing graph is output and deployed to the terminal, and the images collected under four views are recognized and segmented.

4. The multi-view binocular waste metal size measurement and discrimination method according to claim 1, wherein: In step 6, the point cloud center of gravity distance and the image features are combined, the multiple perspective filtered point clouds are coordinate transformed and converted into the same reference coordinate system, the center of gravity distance between the point clouds is calculated, the point cloud center of gravity distance and the image-based twin neural network are combined, the target tracking and matching under multiple perspectives are obtained, and the point clouds of the same scrap metal under different perspectives are spliced together.

5. The multi-perspective binocular scrap metal size measurement discrimination method according to claim 1, characterized in that: In step 7, the OBB (Oriented Bounding Box) method is adopted, the three main directions of the point cloud are obtained by using the PCA (Principal Component Analysis) method, the centroid is obtained, the covariance is calculated, the covariance matrix is obtained, the eigenvalues and eigenvectors of the covariance matrix are calculated, the eigenvectors are the main directions, the input point cloud is converted to the origin by using the main directions and the centroid, the main directions are returned to the coordinate system direction, the bounding box of the point cloud transformed to the origin is established, and the length and width of the scrap metal are directly obtained through the bounding box.

6. A multi-view binocular scrap metal size measurement discrimination device for implementing the multi-view binocular scrap metal size measurement discrimination method according to claim 1, characterized by, It comprises: A multi-perspective setting module is used to set a multi-perspective binocular camera array composed of multiple binocular cameras, and the scrap metal is moved to the center of the field of view of the multi-perspective binocular camera array by a suction cup. A shooting module is used to shoot a two-dimensional image when the suction cup enters the center of the field of view, and multiple binocular cameras simultaneously shoot multiple images under multiple perspectives. An identification module is used to identify and segment the scrap metal under multiple perspectives, and preliminarily identify the over-thick scrap metal. A reconstruction module is used to perform three-dimensional reconstruction on the over-thick scrap metal segmented under different perspectives. A filtering module is used to filter the three-dimensional point cloud and remove outlier point clouds. A splicing module is used to perform target matching on the point clouds under different perspectives of the same scrap metal, and the point clouds under different perspectives of the same scrap metal are spliced according to the camera pose relationship calibrated when the binocular camera is installed. A thickness identification module is used to obtain the length and width of each scrap metal point cloud, form a length-width plane, draw a perpendicular line perpendicular to the length-width plane, and intercept the length between the intersection points of the perpendicular line and the front and back surfaces of the scrap metal point cloud as the thickness of the scrap metal. A judgment module is used to obtain the maximum thickness of each scrap metal, and if the maximum thickness is greater than a threshold value, the suction cup is full of scrap metal and is sent to a gas cutting machine for processing, and after processing by the gas cutting machine, the scrap metal is sent to a gantry shearing machine for processing, and if the maximum thickness is less than the threshold value, the scrap metal is directly sent to the gantry shearing machine for processing.

7. A multi-view binocular scrap metal dimension measurement discrimination system, characterized in that, It comprises: A suction cup is used to adsorb the scrap metal. An image acquisition unit comprises multiple binocular cameras, and the multiple binocular cameras are used to form a multi-perspective binocular camera array. A data processing unit comprises a processor and a memory for storing a computer program capable of running on the processor, and the processor is used to run the computer program to execute the steps of the multi-perspective binocular scrap metal size measurement discrimination method according to any one of claims 1-5.

8. A computer storage medium having stored therein a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the multi-perspective binocular scrap metal size measurement discrimination method according to any one of claims 1-5.

Images