Conveyor belt material flow volume correction method based on light flow vanishing points

Abstract

The invention provides a conveyor belt material flow volume correction method based on light flow shadow elimination points, and the method comprises the steps: employing a material section plane correction algorithm of an imaging system, and obtaining the shadow elimination points of a material flow conveyed by a conveyor belt according to a material light flow method; secondly, correcting a laser plane according to a vanishing point algorithm to obtain a laser plane equation perpendicular to the advancing direction of the conveying belt; and finally, obtaining a material three-dimensional point cloud through a laser triangulation method, projecting the material three-dimensional point cloud to a reference laser plane, and obtaining a corrected material section plane accurate calculation result which is used for measuring the material flow volume of the conveyor belt and carrying out automatic detection and correction. According to the invention, automatic detection and correction can be carried out on the existing volume measurement, so that the adaptability and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of conveyor belts, and in particular to a method for correcting the volume of conveyor belt material flow based on optical flow vanishing points. Background Art

[0002] Conveyor belts are an effective means of transporting materials in airports, ports, and other locations, offering the advantage of long-distance transport. With the rapid development of industry, production processes are becoming increasingly demanding, and the volume of materials on conveyor belts has become a crucial metric in industrial production. Existing methods for measuring the volume of materials on conveyor belts include weighing, ultrasonic methods, deep learning, binocular vision, and laser triangulation. Laser triangulation is a crucial method in volume measurement applications.

[0003] The weighing method is primarily concerned with belt volume being affected by belt tension, deadweight, stiffness, and belt installation angle. The ultrasonic method has high requirements for ambient temperature, humidity, and atmospheric pressure, making it unsuitable for industrial production environments. The deep learning method suffers from inaccurate cross-sectional fitting or a lack of generalization of the fitted model. The binocular vision method is not optimal for conveyor belt volume measurement due to errors in binocular vision calibration and laser center calibration, which affect measurement accuracy. Laser triangulation offers clearer structural features, excellent stability, and a high degree of automation. This method holds great promise for conveyor belt volume measurement.

[0004] While laser triangulation offers numerous advantages, practical measurement challenges remain. Existing laser triangulation methods account for laser hardware errors, the effects of laser beam jitter, and the vertical jitter of the laser spot and the belt. However, they fail to consider the actual positional relationship between the laser plane and the conveyor belt. In fact, a mere 25° rotation of the laser plane can increase the cross-sectional area difference by 18.5156%, affecting the accuracy of the volume of material on the conveyor belt.

[0005] Therefore, to address the issue of laser plane rotation, this patent proposes a novel method for correcting the material cross-section using an optical flow vanishing point algorithm. This method overcomes the problem of the material cross-section plane becoming larger due to laser plane rotation, thereby achieving the goal of accurately capturing the material cross-section plane. This patent utilizes a color camera to capture the material's motion trajectory and corrects the cross-section plane obtained by an industrial camera and laser plane to obtain the accurate cross-section area. Summary of the Invention

[0006] The present invention proposes a conveyor belt material flow volume correction method based on optical flow vanishing points, which can improve the adaptability and accuracy of existing volume measurements by automatically detecting and correcting them.

[0007] The present invention adopts the following technical solutions.

[0008] A method for correcting the volume of material flow on a conveyor belt based on the optical flow vanishing point is disclosed. The method uses a material cross-section plane correction algorithm of an imaging system to first obtain the vanishing point of the material flow transported by the conveyor belt according to the material optical flow method; then, the laser plane is corrected according to the vanishing point algorithm to obtain the laser plane equation perpendicular to the forward direction of the conveyor belt; finally, a three-dimensional point cloud of the material is obtained through laser triangulation and projected onto a reference laser plane to obtain an accurate calculation result of the corrected material cross-section plane, which is used to measure the volume of the material flow on the conveyor belt and perform automatic detection and correction.

[0009] The material cross-section plane correction algorithm extracts vanishing points to obtain a corrected plane, automatically detecting and correcting deviations. Specifically, the algorithm selects a motion trajectory that meets the requirements by screening spatial position and velocity. The intersection points of each segment of the motion trajectory are then determined as the vanishing point set. Clustering and optical flow methods are used to determine the vanishing points that meet the requirements. The positions of the vanishing points in the camera coordinate system are then determined using the camera's internal and external parameters. The positions of the vanishing points in the camera coordinate system are connected to the origin of the camera coordinate system to obtain the normal vector of the reference laser plane. The material's cross-sectional profile is then projected onto this reference laser plane to determine the perpendicular material motion direction and cross-sectional area.

[0010] The method comprises the following steps:

[0011] Step 1: Acquisition of motion trajectory;

[0012] Step 2: Screening of motion trajectories;

[0013] Step 3: Obtaining the vanishing point;

[0014] Step 4: Obtain a reference laser plane;

[0015] Step 5: Obtain the material volume.

[0016] The specific method of step 1 is as follows: after using the imaging system to obtain a video with the material movement trajectory, the pyramid LK optical flow method is used to obtain the material movement trajectory;

[0017] Before using the pyramid LK optical flow method, Harris corner detection is used to extract the key feature points of the material, and then the pyramid LK optical flow method is used to track these feature points to estimate the target's motion trajectory; first, the Sobel operator is used to obtain the grayscale gradient to construct the autocorrelation matrix M, as shown in the following formula 1. In the formula, ω(p, q) is the weight function; I x (p, q)I y (p, q) are the gradients of the image in the x and y directions respectively;

[0018]

[0019] Then, according to the determinant det(M) and trace trace(M) of the autocorrelation matrix M, the response function R of the corner point is calculated, as shown in the following formula 2;

[0020] R = det(M) - α × (trace(M)) 2 Formula 2;

[0021] Where α represents an empirical parameter; finally, the corner points are obtained based on non-maximum suppression; the image brightness function within the time t+Δt can be expressed as I(x, y, t), and the error function can be constructed as shown in Formula 3;

[0022]

[0023] By taking the partial derivative of the error function and writing it in the form of a matrix, the optical flow point can be expressed as Formula 4:

[0024] Divide the image layer from low resolution to high resolution, and sample the optical flow on the low resolution image Correct the optical flow according to formula 4, and use the result as the initial value of the previous layer image, and so on, to finally obtain high-precision optical flow points Estimation; Optical flow results are as follows Figure 2 shown.

[0025] The specific method of step 2 is as follows: in order to avoid the problem of inaccurate motion information representation caused by the similar grayscale values ​​of the belt and the material, the obtained optical flow segments are screened for motion trajectories based on motion characteristics;

[0026] Motion features include: spatial position, time, speed, and acceleration. This step uses spatial position and speed for screening.

[0027] Speed ​​screening means that the speed of any two frames of optical flow points changes within a certain range; assuming that between the two frames of images taken by the imaging system, the speeds of optical flow points i and j are and The velocity between these two points can be obtained by calculating the difference in coordinates:

[0028]

[0029] Define v0>4 pixels per frame to ensure that points representing motion are filtered out;

[0030] Spatial position screening: refers to the absolute value of the slope of any two frames of optical flow segments greater than a certain value, ensuring the point moving on the belt; assuming that between the two frames of image, the coordinates of optical flow points i and j are (x i ,y i ) and (x j ,yj ). Then the slope between the two points is found by taking the difference in their coordinates:

[0031]

[0032] θ ij >θ0; define θ0>80° to retain the required optical flow segments. Figure 3 shown.

[0033] The specific method of step three is: find the focus of the optical flow segments that meet the requirements, use these focuses as the vanishing point set, use the DBSCAN clustering algorithm as the clustering method for processing the vanishing point data set, and eliminate noise through the DBSCAN clustering algorithm; the specific operation method is to calculate the number of points within the neighborhood radius of each point in the vanishing point data set. If it is greater than the minimum number of points, then this is determined as the core point; finally, the core point is output as the largest vanishing point cluster; the vanishing point cluster that has been eliminated of the error points is defined as A, and by finding a vanishing point a, it is connected to all vanishing points a. j The weighted distance square sum is the smallest, and its weight is w j . Weight w j is defined as the sum of the lengths of the optical flow segments passing through the vanishing point. The specific formula is shown in Formula 8;

[0034]

[0035] Table 1 Relationship between the actual calculated cross-sectional areas of objects after the cluster vanishing point coordinates are stabilized

[0036] Original cross-sectional area Theoretical cross-sectional area Clustering to obtain the coordinates of the vanishing points Cross-sectional area of ​​the object Video A 207.5542 175.9267 [1078.9683，-1172.8960] 175.4591 Video B 211.6068 183.6072 [1188.2671，-2406.6360] 184.3502

[0037] The specific method of step 4 is: in the camera coordinate system S c (O c X c Y c Z c ) Any two parallel lines are as shown in formula 9;

[0038] X i1 (λ)=a i1 +λ i1 d i X i2 (λ)=a i2 +λ i2 d i (i=1,2,...) Formula 9;

[0039] Assume that in the camera coordinate system of the imaging system, the position of the vanishing point is P wvi , then through the expression, through the straight line equation of formula 9, the position of the vanishing point is obtained, and formula 10 is:

[0040] P wvi =X i1 (λ ∞ )=a i1 +λ i∞ d i ≈λ i∞ d i Formula 10;

[0041] The origin O c With the camera coordinate system S c (O c X c Y c Z c )'s vanishing point P wvi The connected line is denoted as l i ; Let the straight line l i The direction is d vi , then d vi The expression of is shown in formula 11;

[0042]

[0043] According to formula 11, l i Direction d vi Direction d of a parallel line in space i Consistent; when a point a in the camera coordinate system is known i1 When (x0, y0, z0), the expression of the reference plane π is shown in formula 12;

[0044] a(x-x0)+b(y-y0)+c(z-z0)=0 Formula 12;

[0045] In step 5, the material volume is obtained using the panel method;

[0046] The surface element method performs approximate calculations by dividing the object into multiple surface elements (small planes). Each surface element represents a part of the surface of the object. By accumulating the areas of these surface elements, the total volume of the object can be obtained; for example, Figure 5 shown.

[0047] Assume that the speed of the material transported by the conveyor belt is v(t), and the volume is expressed as the accumulation of q bins s(t). By analyzing the variation of the belt speed v(t) and the bin method s(t), the volume A is calculated. By calculating the start time t1 and the end time t2, the volume of the material flow is obtained, as shown in Formula 13.

[0048]

[0049] The surface element method mainly refers to the object contour. The object contour is mainly divided into the occluded contour line and the unoccluded contour line.

[0050] In the facet method of step five, the object contours of the conveyor belt material are divided into obscured contours and unobstructed contours;

[0051] Using the geometric triangulation relationship between the material image on the imaging plane and the actual material, the actual cross-sectional area of ​​the material is obtained by laser triangulation.

[0052] In this step, the measuring plane is used to represent the unobstructed contour line, and the reference plane is used to represent the obstructed contour line. The specific diagram of the laser triangulation method is as follows: Figure 6 shown.

[0053] In this step, the reference plane represents the obstructed contour line, and the measuring plane represents the unobstructed contour line. Images are obtained when the belt is empty and when the belt is loaded. The images are merged into a closed image through image processing. The cross-sectional area of ​​the material is regarded as an arbitrary polygon and the cross-sectional area is calculated according to the formula for the cross-sectional area of ​​an arbitrary polygon. The cross-sectional area of ​​an arbitrary polygon is shown in Formula 14.

[0054]

[0055] Among them, P i ,P i+1 is the three-dimensional coordinate of the image; A i Represents the area of ​​each small triangle; A represents the overall area; that is, the area of ​​the cross-section plane is obtained; when the material flow moves on the belt, the cross-section plane is projected onto the plane obtained in step 4, that is, the size of the cross-section plane after correction is obtained, and automatic detection and correction are achieved.

[0056] In step five, the contour line is obtained by laser triangulation; the laser triangulation method uses a laser transmitter and an imaging system (including an imaging lens and an imaging plane) containing an industrial camera or a color camera to measure the material to be measured; the laser emission direction of the laser transmitter is perpendicular to the conveying surface of the conveyor belt, and the shooting direction of the imaging system is obliquely pointed at the conveying surface of the conveyor belt.

[0057] In step 5, the measuring surface is the conveying surface of the conveyor belt carrying materials, and the reference surface is the empty conveying surface of the conveyor belt. The more stable the frame rate of the imaging system, the closer the corrected cross-sectional area is to the actual area.

[0058] The relationship between the number of frames shot by the imaging system and the size of the correction plane is as follows: Figure 7 shown.

[0059] The colored dotted line represents the actual cross-sectional area of ​​the object, the other colored dotted line represents the cross-sectional area obtained in step five, and the blue broken line represents the cross-sectional area of ​​the object obtained by the correction algorithm.

[0060] After obtaining the cross-sectional area occupied by the material in the scanning area of ​​the imaging system on the conveyor belt, combined with the conveyor belt running speed, the corresponding material conveying volume per unit time can be calculated.

[0061] This paper proposes a material cross-section plane correction method using an industrial camera, a color camera, and a laser transmitter. The method uses the material optical flow method to obtain the vanishing point. Then, the laser plane is corrected using the vanishing point algorithm to obtain the laser plane equation perpendicular to the belt's travel direction. Finally, laser triangulation is used to project the three-dimensional material point cloud onto a reference laser plane to accurately calculate the corrected material cross-section plane. This algorithm can automatically detect and correct existing volume measurements, improving their adaptability and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0063] Attachment Figure 1 4 is a schematic diagram of the principle of an imaging system for a conveyor belt material flow volume correction method based on an optical flow vanishing point in an embodiment;

[0064] Attachment Figure 2 2 is a schematic diagram of an optical flow result of obtaining a motion trajectory in an embodiment;

[0065] Attachment Figure 3 1 is a schematic diagram of line segments that conform to optical flow segments when screening motion trajectories in an embodiment;

[0066] Attachment Figure 4 1 is a schematic diagram of the principle of obtaining a reference laser platform in an embodiment;

[0067] Attachment Figure 5 1 is a schematic diagram of the principle of using the object panel method for materials moving on a conveyor belt in an embodiment;

[0068] Attachment Figure 6 1 is a schematic diagram of the principle of using laser triangulation in an embodiment;

[0069] Attachment Figure 7 4 is a schematic diagram showing a comparison between the cross-sectional area after correction and the number of frames in an embodiment;

[0070] Attachment Figure 8 It is a schematic flow diagram of the present invention. DETAILED DESCRIPTION

[0071] As shown in the figure, a method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point is described. The method uses the material cross-section plane correction algorithm of the imaging system to first obtain the vanishing point of the material flow transported by the conveyor belt according to the material optical flow method; then, the laser plane is corrected according to the vanishing point algorithm to obtain the laser plane equation perpendicular to the forward direction of the conveyor belt; finally, the three-dimensional point cloud of the material is obtained by laser triangulation and projected onto the reference laser plane to obtain the accurate calculation result of the corrected material cross-section plane, which is used to measure the volume of the conveyor belt material flow and perform automatic detection and correction.

[0072] The material cross-section plane correction algorithm extracts vanishing points to obtain a corrected plane, automatically detecting and correcting deviations. Specifically, the algorithm selects a motion trajectory that meets the requirements by screening spatial position and velocity. The intersection points of each segment of the motion trajectory are then determined as the vanishing point set. Clustering and optical flow methods are used to determine the vanishing points that meet the requirements. The positions of the vanishing points in the camera coordinate system are then determined using the camera's internal and external parameters. The positions of the vanishing points in the camera coordinate system are connected to the origin of the camera coordinate system to obtain the normal vector of the reference laser plane. The material's cross-sectional profile is then projected onto this reference laser plane to determine the perpendicular material motion direction and cross-sectional area.

[0073] The method comprises the following steps:

[0074] Step 1: Acquisition of motion trajectory;

[0075] Step 2: Screening of motion trajectories;

[0076] Step 3: Obtaining the vanishing point;

[0077] Step 4: Obtain a reference laser plane;

[0078] Step 5: Obtain the material volume.

[0079] The specific method of step 1 is as follows: after using the imaging system to obtain a video with the material movement trajectory, the pyramid LK optical flow method is used to obtain the material movement trajectory;

[0080] Before using the pyramid LK optical flow method, Harris corner detection is used to extract the key feature points of the material, and then the pyramid LK optical flow method is used to track these feature points to estimate the target's motion trajectory; first, the Sobel operator is used to obtain the grayscale gradient to construct the autocorrelation matrix M, as shown in the following formula 1. In the formula, ω(p,q) is the weight function; I x (p,q)I y (p,q) are the gradients of the image in the x and y directions respectively;

[0081]

[0082] Then, according to the determinant det(M) and trace trace(M) of the autocorrelation matrix M, the response function R of the corner point is calculated, as shown in the following formula 2;

[0083] R = det(M) - α × (trace(M)) 2 Formula 2;

[0084] Where α represents an empirical parameter; finally, the corner points are obtained based on non-maximum suppression; the image brightness function within the time t+Δt can be expressed as I(x, y, t), and the error function can be constructed as shown in Formula 3;

[0085]

[0086] By taking the partial derivative of the error function and writing it in the form of a matrix, the optical flow point can be expressed as Formula 4:

[0087] Divide the image layer from low resolution to high resolution, and sample the optical flow on the low resolution image Correct the optical flow according to formula 4, and use the result as the initial value of the previous layer image, and so on, to finally obtain high-precision optical flow points Estimation; Optical flow results are as follows Figure 2 shown.

[0088] The specific method of step 2 is as follows: in order to avoid the problem of inaccurate motion information representation caused by the similar grayscale values ​​of the belt and the material, the obtained optical flow segments are screened for motion trajectories based on motion characteristics;

[0089] Motion features include: spatial position, time, speed, and acceleration. This step uses spatial position and speed for screening.

[0090] Speed ​​screening means that the speed of any two frames of optical flow points changes within a certain range; assuming that between the two frames of images taken by the imaging system, the speeds of optical flow points i and j are and The velocity between these two points can be obtained by calculating the difference in coordinates:

[0091]

[0092] Define v0>4 pixels per frame to ensure that points representing motion are filtered out;

[0093] Spatial position screening: refers to the absolute value of the slope of any two frames of optical flow segments greater than a certain value, ensuring the point moving on the belt; assuming that between the two frames of image, the coordinates of optical flow points i and j are (x i ,y i ) and (x j ,yj ). Then the slope between the two points is found by taking the difference in their coordinates:

[0094]

[0095] θ ij >θ0; define θ0>80° to retain the required optical flow segments. Figure 3 shown.

[0096] The specific method of step three is: find the focus of the optical flow segments that meet the requirements, use these focuses as the vanishing point set, use the DBSCAN clustering algorithm as the clustering method for processing the vanishing point data set, and eliminate noise through the DBSCAN clustering algorithm; the specific operation method is to calculate the number of points within the neighborhood radius of each point in the vanishing point data set. If it is greater than the minimum number of points, then this is determined as the core point; finally, the core point is output as the largest vanishing point cluster; the vanishing point cluster that has been eliminated of the error points is defined as A, and by finding a vanishing point a, it is connected to all vanishing points a. j The weighted distance square sum is the smallest, and its weight is w j . Weight w j is defined as the sum of the lengths of the optical flow segments passing through the vanishing point. The specific formula is shown in Formula 8;

[0097]

[0098] Table 1 Relationship between the actual calculated cross-sectional areas of objects after the cluster vanishing point coordinates are stabilized

[0099] Original cross-sectional area Theoretical cross-sectional area Clustering to obtain the coordinates of the vanishing points Cross-sectional area of ​​the object Video A 207.5542 175.9267 [1078.9683，-1172.8960] 175.4591 Video B 211.6068 183.6072 [1188.2671，-2406.6360] 184.3502

[0100] The specific method of step 4 is: in the camera coordinate system S c (O c X c Y c Z c ) Any two parallel lines are as shown in formula 9;

[0101] X i1 (λ)=a i1 +λ i1 d i X i2 (λ)=a i2 +λ i2 d i (i=1, 2, ...) Formula 9;

[0102] Assume that in the camera coordinate system of the imaging system, the position of the vanishing point is P wvi , then through the expression, through the straight line equation of formula 9, the position of the vanishing point is obtained, and formula 10 is:

[0103] P wvi =X i1 (λ ∞ )=a i1 +λ i∞ d i ≈λ i∞ d i Formula 10;

[0104] The origin O c With the camera coordinate system S c (O c X c Y c Z c )'s vanishing point P wvi The connected line is denoted as l i ; Let the straight line l i The direction is d vi , then d vi The expression of is shown in formula 11;

[0105]

[0106] According to formula 11, l i Direction d vi Direction d of a parallel line in space i Consistent; when a point a in the camera coordinate system is known i1 When (x0, y0, z0), the expression of the reference plane π is shown in formula 12;

[0107] a(x-x0)+b(y-y0)+c(z-z0)=0 Formula 12;

[0108] In step 5, the material volume is obtained using the panel method;

[0109] The surface element method performs approximate calculations by dividing the object into multiple surface elements (small planes). Each surface element represents a part of the surface of the object. By accumulating the areas of these surface elements, the total volume of the object can be obtained; for example, Figure 5 shown.

[0110] Assume that the speed of the material transported by the conveyor belt is v(t), and the volume is expressed as the accumulation of q bins s(t). By analyzing the variation of the belt speed v(t) and the bin method s(t), the volume A is calculated. By calculating the start time t1 and the end time t2, the volume of the material flow is obtained, as shown in Formula 13.

[0111]

[0112] The surface element method mainly refers to the object contour. The object contour is mainly divided into the occluded contour line and the unoccluded contour line.

[0113] In the facet method of step five, the object contours of the conveyor belt material are divided into obscured contours and unobstructed contours;

[0114] Using the geometric triangulation relationship between the material image on the imaging plane and the actual material, the actual cross-sectional area of ​​the material is obtained by laser triangulation.

[0115] In this step, the measuring plane is used to represent the unobstructed contour line, and the reference plane is used to represent the obstructed contour line. The specific diagram of the laser triangulation method is as follows: Figure 6 shown.

[0116] In this step, the reference plane represents the obstructed contour line, and the measuring plane represents the unobstructed contour line. Images are obtained when the belt is empty and when the belt is loaded. The images are merged into a closed image through image processing. The cross-sectional area of ​​the material is regarded as an arbitrary polygon and the cross-sectional area is calculated according to the formula for the cross-sectional area of ​​an arbitrary polygon. The cross-sectional area of ​​an arbitrary polygon is shown in Formula 14.

[0117]

[0118] Among them, P i , P i+1 is the three-dimensional coordinate of the image; A i Represents the area of ​​each small triangle; A represents the overall area; that is, the area of ​​the cross-section plane is obtained; when the material flow moves on the belt, the cross-section plane is projected onto the plane obtained in step 4, that is, the size of the cross-section plane after correction is obtained, and automatic detection and correction are achieved.

[0119] In step five, the contour line is obtained by laser triangulation; the laser triangulation method uses a laser transmitter and an imaging system (including an imaging lens and an imaging plane) containing an industrial camera or a color camera to measure the material to be measured; the laser emission direction of the laser transmitter is perpendicular to the conveying surface of the conveyor belt, and the shooting direction of the imaging system is obliquely pointed at the conveying surface of the conveyor belt.

[0120] In step 5, the measuring surface is the conveying surface of the conveyor belt carrying materials, and the reference surface is the empty conveying surface of the conveyor belt. The more stable the frame rate of the imaging system, the closer the corrected cross-sectional area is to the actual area.

[0121] The relationship between the number of frames shot by the imaging system and the size of the correction plane is as follows: Figure 7 shown.

[0122] The colored dotted line represents the actual cross-sectional area of ​​the object, the other colored dotted line represents the cross-sectional area obtained in step five, and the blue broken line represents the cross-sectional area of ​​the object obtained by the correction algorithm.

[0123] After obtaining the cross-sectional area occupied by the material in the scanning area of ​​the imaging system on the conveyor belt, combined with the conveyor belt running speed, the corresponding material conveying volume per unit time can be calculated.

Claims

1. A method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point, characterized by: The method uses a material cross-section plane correction algorithm of an imaging system. First, the vanishing point of the material flow transported by a conveyor belt is obtained according to a material optical flow method. Then, the laser plane is corrected according to the vanishing point algorithm to obtain a laser plane equation perpendicular to the forward direction of the conveyor belt. Finally, a three-dimensional point cloud of the material is obtained by laser triangulation and projected onto a reference laser plane to obtain an accurate calculation result of the corrected material cross-section plane, which is used to measure the volume of the material flow on the conveyor belt and perform automatic detection and correction.

2. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 1, characterized in that: The material section plane correction algorithm extracts the vanishing points to obtain the corrected plane, which can automatically detect and correct deviations. Specifically, the motion trajectory that meets the requirements is selected by screening the spatial position and speed; the intersection points of the line segments of the motion trajectory are then obtained as the vanishing point set; the vanishing points that meet the requirements are obtained through clustering and optical flow methods, and the position of the vanishing points in the camera coordinate system is obtained through the internal and external parameters of the camera. The position of the vanishing point in the camera coordinate system is connected to the origin in the camera coordinate system to obtain the normal vector of the reference laser plane; the cross-sectional profile of the material is projected onto this reference laser plane; Obtain the vertical material movement direction and the cross-sectional area of ​​the material.

3. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 2, characterized in that: The method comprises the following steps: Step 1: Acquisition of motion trajectory; Step 2: Screening of motion trajectories; Step 3: Obtaining the vanishing point; Step 4: Obtain a reference laser plane; Step 5: Obtain the material volume.

4. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 3, characterized in that: The specific method of step 1 is as follows: after using the imaging system to obtain a video with the material movement trajectory, the pyramid LK optical flow method is used to obtain the material movement trajectory; Before using the pyramid LK optical flow method, Harris corner detection is used to extract the key feature points of the material, and then the pyramid LK optical flow method is used to track these feature points to estimate the target's motion trajectory. First, the Sobel operator is used to obtain the grayscale gradient to construct the autocorrelation matrix M, as shown in the following formula 1. In the formula, ω(p, q) is the weight function; I x (p, q)I y (p, q) are the gradients of the image in the x and y directions respectively; Then, according to the determinant det(M) and trace trace(M) of the autocorrelation matrix M, the response function R of the corner point is calculated, as shown in the following formula 2; R = det(M) - α × (trace(M)) 2 Formula 2; Where α represents an empirical parameter; finally, the corner points are obtained based on non-maximum suppression; the image brightness function within the time t+Δt can be expressed as I(x, y, t), and the error function can be constructed as shown in Formula 3; By taking the partial derivative of the error function and writing it in the form of a matrix, the optical flow point can be expressed as Formula 4: Divide the image layer from low resolution to high resolution, and sample the optical flow on the low resolution image Correct the optical flow according to formula 4, and use the result as the initial value of the previous layer image, and so on, to finally obtain high-precision optical flow points estimate.

5. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 3, characterized in that: The specific method of step 2 is as follows: in order to avoid the problem of inaccurate motion information representation caused by the similar grayscale values ​​of the belt and the material, the obtained optical flow segments are screened for motion trajectories based on motion characteristics; Motion features include: spatial position, time, speed, and acceleration. This step uses spatial position and speed for screening. Speed ​​screening means that the speed of any two frames of optical flow points changes within a certain range; assuming that between the two frames of images taken by the imaging system, the speeds of optical flow points i and j are and The velocity between these two points can be found by calculating the difference in coordinates: Define v0>4 pixels per frame to ensure that points representing motion are filtered out; Spatial position screening: refers to the absolute value of the slope of any two frames of optical flow segments greater than a certain value, ensuring the point moving on the belt; assuming that between the two frames of image, the coordinates of optical flow points i and j are (x i ,y i ) and (x j ,y j ). Then the slope between the two points is found by taking the difference in their coordinates: θ ij >θ0; define θ0>80° to retain the required optical flow segments.

6. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 3, characterized in that: The specific method of step three is: find the focus of the optical flow segments that meet the requirements, use these focuses as the vanishing point set, use the DBSCAN clustering algorithm as the clustering method for processing the vanishing point data set, and eliminate noise through the DBSCAN clustering algorithm; the specific operation method is to calculate the number of points within the neighborhood radius of each point in the vanishing point data set. If it is greater than the minimum number of points, then this is determined as the core point; finally, the core point is output as the largest vanishing point cluster; the vanishing point cluster that has been eliminated of the error points is defined as A, and by finding a vanishing point a, it is connected to all vanishing points a. j The weighted distance square sum is the smallest, and its weight is w j . Weight w j is defined as the sum of the lengths of the optical flow segments passing through the vanishing point; the specific formula is shown in Formula 8; 7. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 3, characterized in that: The specific method of step 4 is: in the camera coordinate system S c (O c X c Y c Z c ) Any two parallel lines are as shown in formula 9; X i1 (λ) = a i1 + λ i1 d i X i2 (λ) = a i2 + λ i2 d i (i = 1, 2,...) Formula 9; Assume that in the camera coordinate system of the imaging system, the position of the vanishing point is P wvi , then through the expression, through the straight line equation of formula 9, the position of the vanishing point is obtained, and formula 10 is: P wvi = X i1 (λ ∞ ) = a i1 + λ i∞ d i ≈ λ i∞ d i Equation 10; The origin O c With the camera coordinate system S c (O c X c Y c Z c )'s vanishing point P wvi The connected line is denoted as l i ; Let the straight line l i The direction is d vi , then d vi The expression of is shown in formula 11; According to formula 11, l i Direction d vi Direction d of a parallel line in space i Consistent; when a point a in the camera coordinate system is known i1 When (x0, y0, z0), the expression of the reference plane π is shown in formula 12; a(x-x0)+b(y-y0)+c(z-z0)=0 Formula 12.

8. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 3, characterized in that: In step 5, the volume of the material is obtained using the panel method; Assume that the speed of the material transported by the conveyor belt is v(t), and the volume is expressed as the accumulation of q bins s(t). By analyzing the variation of the belt speed v(t) and the bin method s(t), the volume A is calculated. By calculating the start time t1 and the end time t2, the volume of the material flow is obtained, as shown in Formula 13. In the facet method of step five, the object contours of the conveyor belt material are divided into obscured contours and unobstructed contours; Using the geometric triangulation relationship between the material image on the imaging plane and the actual material, the actual cross-sectional area of ​​the material is obtained by laser triangulation. In this step, the reference plane represents the obstructed contour line, and the measuring plane represents the unobstructed contour line. Images are obtained when the belt is empty and when the belt is loaded. The images are merged into a closed image through image processing. The cross-sectional area of ​​the material is regarded as an arbitrary polygon and the cross-sectional area is calculated according to the formula for the cross-sectional area of ​​an arbitrary polygon. The cross-sectional area of ​​an arbitrary polygon is shown in Formula 14. Among them, P i , P i+1 is the three-dimensional coordinate of the image; A i Represents the area of ​​each small triangle; A represents the overall area; that is, the area of ​​the cross-section plane is obtained; when the material flow moves on the belt, the cross-section plane is projected onto the plane obtained in step 4, that is, the size of the cross-section plane after correction is obtained, and automatic detection and correction are achieved.

9. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 8, characterized in that: In step 5, the contour line is obtained by laser triangulation; laser triangulation uses a laser transmitter and an imaging system including an industrial camera or a color camera to measure the material to be measured; The laser emission direction of the laser emitter is perpendicular to the conveying surface of the conveyor belt, and the shooting direction of the imaging system is obliquely pointed to the conveying surface of the conveyor belt.

10. The method for correcting the volume of conveyor belt material flow based on the optical flow vanishing point according to claim 9, characterized in that: In step 5, the measuring surface is the conveying surface of the conveyor belt carrying materials, and the reference surface is the empty conveying surface of the conveyor belt. The more stable the frame rate of the imaging system, the closer the corrected cross-sectional area is to the actual area.

Images