Laser coal flow recognition method for coal conveyor belt based on video monitoring

Through video monitoring and laser technology, the laser line characteristics of coal transport belts are extracted, combined with the camera ranging principle, and the coal flow is calculated, which solves the shortcomings of real-time monitoring of coal flow in underground belt conveyors of coal mines, real-time and accurate coal flow monitoring is achieved and automatic adjustment improve coal mine production efficiency.

CN114549508BActive Publication Date: 2024-11-26JIANGSU SHINE TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210194225.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-01
Publication Date
2024-11-26
Estimated Expiration
2042-03-01

AI Technical Summary

Technical Problem

In the real-time monitoring of coal flow of underground belt conveyors in coal mines, traditional weighing methods cannot obtain coal flow information in real time. Although radar technology can monitor in real time, it cannot effectively present the coal flow change diagram, and lacks real-time monitoring and automatic Adjustment ability.

Method used

Using the laser coal flow recognition method of coal transport belt based on video surveillance, the coal transport belt is scanned in real time through an optical camera, and the characteristics of empty belts and real-time coal flow laser lines are extracted. Combined with the camera ranging principle and trapezoidal calculation method, the coal flow rate is calculated to realize Real-time coal quantity monitoring.

Benefits of technology

Real-time monitoring and automatic adjustment of coal flow is realized, the production efficiency of coal mines is improved, and the coal flow change diagram can be accurately presented, solving the shortcomings of real-time and accuracy of traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549508B_ABST
    Figure CN114549508B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for laser coal flow identification of a coal conveyor belt based on video monitoring, which comprises: step S1, scanning the underground coal conveyor belt in real time through a 2-megapixel optical camera, and then uploading the scanned belt picture to a host computer for subsequent analysis; step S2, the user draws a designated analysis area on the coal conveyor belt picture according to actual needs, and configures the basic parameters of the camera, the basic parameters of the camera including the picture analysis size, focal length, film size, and camera film inch ratio; step S3, extracting the empty belt laser line in the designated analysis area according to the designated analysis area drawn by the user. The present invention provides a method for laser coal flow identification of a coal conveyor belt based on video monitoring, which realizes coal quantity statistics and performs real-time coal quantity monitoring through video acquisition, empty belt feature extraction based on laser lines, initial spatial coordinate establishment, and real-time coal flow feature acquisition based on laser lines.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a laser coal flow identification method for a coal conveying belt based on video monitoring. Background Art

[0002] At present, with the continuous development of intelligent technology, coal mine automation production and control occupy an increasingly important position in the field of coal mines, and an important prerequisite for promoting its rapid development is to realize the monitoring of coal flow in underground belt conveyors of coal mines. Therefore, the use of laser and image processing technology to monitor the coal flow in real time, and adjust the gate opening under the warehouse according to the real-time monitoring results, so that the belt conveyor can flow continuously, without leakage, and with uniform flow, is of great significance for realizing automatic monitoring of belt conveyor flow and automatic control of feeding system, as well as improving the work and production efficiency of coal mines.

[0003] At present, the real-time monitoring of coal flow in underground belt conveyors in coal mines is mostly carried out by traditional weighing methods or based on radar technology. Although the traditional weighing method is accurate in calculation, it cannot obtain coal flow information in real time; although real-time monitoring based on radar technology can effectively solve the problem of obtaining coal flow information in real time, it cannot present the real-time change graph of coal flow to users well. Real-time monitoring of coal flow using laser and image processing technology can help effectively solve this problem. Therefore, laser coal flow identification of coal conveyor belts based on video monitoring is very practical in the process of real-time monitoring of coal flow. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a coal transport belt laser coal flow identification method based on video monitoring, which realizes coal quantity statistics and performs real-time coal quantity monitoring through video acquisition, empty belt feature extraction based on laser lines, establishment of initial spatial coordinates, and real-time coal flow feature acquisition based on laser lines.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] The present invention provides a method for laser coal flow recognition of a coal conveyor belt based on video monitoring, which comprises:

[0007] Step S1, using a 2-megapixel optical camera to scan the underground coal conveyor belt in real time, and then uploading the scanned belt image to the host computer for subsequent analysis;

[0008] Step S2: The user draws a designated analysis area on the coal conveyor belt image according to actual needs, and configures basic camera parameters, which include image analysis size, focal length, film size, and camera film inch ratio;

[0009] Step S3, according to the designated analysis area drawn by the user, extracting the empty belt laser line in the designated analysis area, wherein the laser line is the interface feature of the empty belt;

[0010] Step S4: Based on the camera ranging principle, the initial space coordinates are established according to the basic parameters of the camera and the calibration information of the reference object;

[0011] Step S5, according to the designated analysis area drawn by the user, extracting the real-time coal flow laser line in the designated analysis area, wherein the real-time coal flow laser line is the interface feature when the belt is transporting coal;

[0012] Step S6, according to the real-time coal flow laser line and the empty belt laser line, combined with the trapezoidal calculation method and the integral solution method, calculate the cross-sectional area Mianji of the current interface, then calculate the coal flow volume v in combination with the actual belt running speed s, and finally calculate the current coal flow m in combination with the coal flow density ρ and the coal flow volume v.

[0013] Further, in the step S3, according to the designated analysis area drawn by the user, the empty belt laser line in the designated analysis area is extracted, and the laser line is the interface feature of the empty belt;

[0014] Step S31, according to the designated analysis area drawn by the user, a laser light source is used to emit a laser line in the designated analysis area, and then a BGR picture of the designated analysis area of ​​the empty belt is taken by an optical camera, and the obtained BGR picture of the empty belt is converted into an HSV three-channel picture;

[0015] Step S32, extracting all red pixels in the designated analysis area according to the HSV red feature value range and the vertices of the designated analysis area, generating a corresponding empty belt binary image, wherein the red part is represented by white and the rest is represented by black;

[0016] Step S33, denoising the binary image obtained in step S32 by using morphological denoising, firstly using corrosion operation to remove interfering noise points, and then using dilation operation to fill holes;

[0017] Step S34, performing curve compression on the binary image of the empty belt obtained after the denoising process, and converting the feature points of the empty belt into a single-point curve;

[0018] Step S35: Then, the fault part of the compressed curve is interpolated using the equal division interpolation method.

[0019] Further, in step S34, curve compression is performed on the binary image of the empty belt obtained after the denoising process, including:

[0020] Step S341, taking the middle white point in each column of the empty belt binary image;

[0021] Step S342: determine whether the white point is located in the specified analysis area. If the white point is located in the specified analysis area, keep the white point. If the white point is not in the specified analysis area, change the white point to black.

[0022] Further, in step S35, the fault part of the compressed curve is interpolated using an equal division interpolation method, including:

[0023] Step S351: Assume that the height difference between the white dots retained in the previous column and the white dots retained in the next column is ΔH. If ΔH≤200 or the input value is -1, perform interpolation processing, otherwise skip;

[0024] Step S352: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is greater than 100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the previous column;

[0025] Step S353: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is less than -100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the next column;

[0026] Step S354: When the situation is not included in step S351 to step S353, interpolate according to the equal value. The interpolation formula is as follows:

[0027]

[0028] Furthermore, in step S4, based on the camera ranging principle, according to the basic parameters of the camera and the calibration information of the reference object, the initial space coordinates are established, including:

[0029] Step S41, selecting a reference object and obtaining the actual size of the reference object, and placing the reference object on an empty belt;

[0030] Step S42, obtaining the size and position information of the image of the reference object in the camera;

[0031] Step S43: constructing spatial coordinates using the camera ranging principle.

[0032] Furthermore, in step S43, the spatial coordinates are constructed using the camera ranging principle, including:

[0033] Step S431, measuring the distance from any point in the camera imaging picture to the center of the camera lens, comprising:

[0034] Assume an object point X on the belt, the image point Y of the object point X in the image, and measure the distance OY from the image point Y to the center O of the camera lens:

[0035]

[0036] Among them, OT is the focal length of the camera; TY can be obtained by multiplying the pixel distance between points T and Y in the image by the actual film distance represented by each two adjacent pixels in the film;

[0037] Step S432, measuring the distance between two object points on the belt, comprising:

[0038] Assume that object points H and I are on the belt, the center of the camera lens is O, the image point J of object point H in the imaged picture, the image point K of object point I in the imaged picture, the distances OH and OI are known, and the distance HI between object point H and object point I is calculated. The step S431 can calculate the distance from any image point in the film to the center O of the camera lens, so OK, OJ and KJ are all known quantities. According to the law of cosines, it can be obtained:

[0039]

[0040] And ∠HOI=∠KOJ, so

[0041]

[0042] Step S433, when the distances from two of the three collinear points on the belt to the camera are known, the position of the third point is calculated, including:

[0043] Assume that there are collinear points A, B and C on the belt, the center of the camera lens is O, the image point A in the image picture is A', the image point B in the image picture is B', the image point C in the image picture is C', and OB and OA are known. Find OC and CA:

[0044]

[0045] ∠AOC=∠A'OC';

[0046] Make AD perpendicular to the line OC and point D;

[0047] AD=OA*sin∠AOC;

[0048] Given OB and OA, the distance between the two points on the belt can be measured in step S432, and the AB distance can be obtained:

[0049]

[0050] When point C is between point A and point B or point C is outside point B, we have:

[0051] ∠OAC=∠OAB;

[0052] Otherwise, when A is between points B and C, we have:

[0053] ∠OAC=π-∠OAB;

[0054] Next step:

[0055] sin∠ACD=sin(∠OAC+∠AOC)

[0056] AC=AD / sin∠ACD

[0057]

[0058] At this time, OC and AC are obtained;

[0059] Step S434: In the actual scene, a plane is calibrated with several rectangles, and the distances from the four vertices of the rectangle to the center of the camera lens are measured. Then, the distance from any point on the plane where the rectangle is located to the center of the camera lens, as well as the actual distance between any two points, can be calculated:

[0060] It is known that the distances from the four vertices EFGH to the camera lens center O are OE, OF, OG, and OH. For any point TP1 in the image, draw a straight line through point TP1 that is parallel to EH and intersects the rectangle at points NP1 and NP2. Using the distances from two of the three collinear points on the belt to the camera in step S433, the position of the third point is calculated. The distance from the camera lens center O to point NP2 can be obtained from points H, G, and NP2, and the distance from the camera lens center O to point NP1 can be obtained from points E, F, and NP1. Then, the distance from point TP1 to the camera lens center O and the distance from point TP2 to the camera lens center O can be obtained through points NP1, NP2, and TP1. Then, the distance between two object points on the belt measured in step S432 can be used to obtain the distance from point TP1 to point TP2.

[0061] Step S435, when the distance from the vertex of the rectangle to the center of the camera lens is known, finding the point position on the plane where the rectangle is located, including:

[0062] First, put the reference rectangle RSTU into the viewing range to obtain an image containing the rectangle, and measure the actual distances of the sides of the rectangle, that is, the lengths of RS, ST, TU, and UR are known:

[0063] Mark the four points R', S', T', and U' corresponding to the rectangle in the image. It can be seen from step S434 that the angle between any two points and the center O of the camera lens can be obtained according to the coordinates of the four points, and then the sizes of ∠ROS, ∠SOT, ∠TOU, and ∠UOR can be obtained;

[0064] Assume that the length of RO is x, then:

[0065]

[0066] ∠SRO=π-∠RSO-∠ROS

[0067]

[0068] ∠OST=π-∠SOT-∠STO

[0069]

[0070] That is, for any given x, the length of RT can be found, and the true value of the length of RT is:

[0071]

[0072] Select the possible value range of x-OR, then traverse all possible x values ​​within the value range according to the specific step size d, substitute the above process to obtain the theoretical value of RT and make a difference with the actual value, and the point with the smallest difference is recorded as x1,

[0073] Update the value range:

[0074] [x1-dtx1+d];

[0075] The update step size is:

[0076] d1=d / n;

[0077] Where n is the number of partition intervals in each iteration;

[0078] Traverse and solve again;

[0079] Repeat the above process until the error between the theoretical value and the true value of RT is small enough or the step length d is small enough, and the value of x at this time is set as the length of OR;

[0080] After obtaining the length of OR, the distances of the other three sides OS, OT, and OU can be obtained using the related theorems of trigonometric functions;

[0081] When the actual side lengths of the four sides RS, ST, TU, UR of the reference rectangle RSTU and the side lengths of the four vertex coordinate systems of the reference object imaging figure R'S'T'U' are known, and the distances OR, OS, OT, OU from the four vertices of the reference rectangle RSTU to the center O of the camera lens are obtained, the distance between any two points in the plane of the rectangle in the camera imaging figure can be calculated through step S434.

[0082] Furthermore, in step S6, the cross-sectional area Mianji of the current interface is calculated according to the real-time coal flow laser line and the empty belt laser line, in combination with the trapezoidal calculation method and the integral solution method, and then the coal flow volume v is calculated in combination with the actual belt running speed s, and finally the current coal flow m is calculated in combination with the coal flow density ρ and the coal flow volume v, including:

[0083] The area formed between the real-time coal flow laser line and the empty belt laser line is the cross-sectional area Mianji of the coal flow;

[0084] Take points (i, iiCoal[i]) and (i+1, iiCoal[i+1]) on the coal flow laser line;

[0085] Take points (i, iiEmptyBelt[i]) and (i+1, iiEmptyBelt[i+1]) on the empty belt laser line;

[0086] The points (i, iiCoal[i]), (i+1, iiCoal[i+1]), (i, iiEmptyBelt[i]) and (i+1, iiEmptyBelt[i+1]) form a trapezoid, and the area lenHor of the trapezoid is:

[0087] lenHor=0.5*(a+b)*h;

[0088] in,

[0089] a is the distance between point (i,iiCoal[i]) and point (i,iiEmptyBelt[i]);

[0090] b is the distance between point (i+1,iiCoal[i+1]) and point (i+1,iiEmptyBelt[i+1]);

[0091] h is the height of the trapezoid, and the value of h is 1 pixel;

[0092] The cross-sectional area Mianji of the coal flow is the sum of the areas of all trapezoids between the real-time coal flow laser line and the empty belt laser line. The expression of the cross-sectional area Mianji of the coal flow is:

[0093] Mianji = lenVer*len((i+1)-i);

[0094] Given the actual running speed s of the belt, the coal flow volume v is:

[0095] v = s*Mianji;

[0096] Given the coal density ρ, the coal flow rate m is:

[0097] m=ρ*v.

[0098] The above technical scheme is adopted. The present invention uses an optical camera to scan the underground coal conveyor belt in real time, obtains the contour features of a cross section of the empty belt by laser line to extract the empty belt features, constructs the initial space coordinates for calculation according to the camera ranging principle, obtains the contour features of an interface of the real-time coal flow by laser line to obtain the real-time coal flow features, calculates the volume according to the cross-sectional area of ​​the coal flow, and then uses the belt speed to count the current coal quantity. The present invention realizes the statistics of the coal conveyor belt's coal transportation quantity through video and laser, realizes the monitoring of coal quantity, and improves the work and production efficiency of coal mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 It is a flow chart of the method for laser coal flow identification of a coal conveyor belt based on video monitoring of the present invention;

[0100] Figure 2 It is a configuration interface of basic parameters of the camera of the present invention;

[0101] Figure 3 is the color range of HSV of the present invention;

[0102] Figure 4 An exemplary diagram of the etching operation of the present invention;

[0103] Figure 5 is an exemplary diagram of the expansion operation of the present invention;

[0104] Figure 6 It is a schematic diagram of curve compression of the present invention;

[0105] Figure 7 A schematic diagram of region determination of the present invention;

[0106] Figure 8 It is a parsing diagram of step S431 of the present invention;

[0107] Fig. 9 is a parsing diagram of step S432 of the present invention;

[0108] Fig.10 is a parsing diagram of step S433 of the present invention;

[0109] Fig.11 is a parsing diagram of step S434 of the present invention;

[0110] Fig.12 is a parsing diagram of step S435 of the present invention;

[0111] Fig.13 It is a cross-sectional view of the coal flow of the present invention. DETAILED DESCRIPTION

[0112] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0113] like Figure 1 As shown, this embodiment provides a method for laser coal flow recognition of a coal conveyor belt based on video monitoring, which includes:

[0114] Step S1, using a 2-megapixel optical camera to scan the underground coal conveyor belt in real time, and then uploading the scanned belt image to the host computer for subsequent analysis;

[0115] Step S2: Figure 2 As shown, the user draws the designated analysis area on the coal conveyor belt image according to actual needs and configures the basic parameters of the camera, which include image analysis size, focal length, film size, and camera film inch ratio;

[0116] Step S3, according to the designated analysis area drawn by the user, extract the empty belt laser line in the designated analysis area, the laser line is the interface feature of the empty belt;

[0117] Step S4: Based on the camera ranging principle, the initial space coordinates are established according to the basic parameters of the camera and the calibration information of the reference object;

[0118] Step S5, according to the designated analysis area drawn by the user, extract the real-time coal flow laser line in the designated analysis area, the real-time coal flow laser line is the interface feature when the belt is transporting coal;

[0119] Step S6, according to the real-time coal flow laser line and the empty belt laser line, combined with the trapezoidal calculation method and the integral solution method, calculate the cross-sectional area Mianji of the current interface, then calculate the coal flow volume v in combination with the actual belt running speed s, and finally calculate the current coal flow m in combination with the coal flow density ρ and the coal flow volume v.

[0120] Specifically, in step S3, according to the designated analysis area drawn by the user, the empty belt laser line in the designated analysis area is extracted, and the laser line is the interface feature of the empty belt;

[0121] Step S31, according to the designated analysis area drawn by the user, a laser line is emitted in the designated analysis area by a laser light source, and then a BGR picture of the designated analysis area of ​​the empty belt is taken by an optical camera, and the obtained BGR picture of the empty belt is converted into an HSV three-channel picture.

[0122] BGR is the color space we are most exposed to. An image is represented by three channels, namely blue (B), green (G) and red (R). Different combinations of these three colors can form almost all other colors. BGR color space is the most basic, most commonly used, and hardware-oriented color space in image processing, and it is relatively easy to understand. BGR color space uses a linear combination of three color components to represent colors. Any color is related to these three components, and these three components are highly correlated, so it is not intuitive to continuously change colors. If you want to adjust the color of the image, you need to change these three components. Images obtained in natural environments are easily affected by natural lighting, occlusion, and shadows, that is, they are sensitive to brightness. The three components of the BGR color space are closely related to brightness, that is, as long as the brightness changes, the three components will change accordingly, and there is no more intuitive way to express it. However, the sensitivity of the human eye to these three color components is different. Among monochrome colors, the human eye is least sensitive to red and most sensitive to blue, so the BGR color space is a color space with poor uniformity. If the similarity of colors is directly measured by Euclidean distance, the result will have a large deviation from human vision. For a certain color, it is difficult for us to infer the more accurate three-component values ​​to represent it. Therefore, the BGR color space is suitable for display systems but not for image processing.

[0123] For the above reasons, the HSV color space is widely used in image processing. It is closer to people's perception of color than BGR. It expresses the hue, brightness and brightness of the color very intuitively, which is convenient for color comparison. In the HSV color space, it is easier to track objects of a certain color than BGR, and it is often used to segment objects of a specified color. Among them, the HSV color space is composed of three parts: Hue (hue), Saturation (saturation, color purity), and Value (brightness).

[0124] Specifically, step S32, according to the HSV red feature value range and the vertices of the specified analysis area, all red pixels in the specified analysis area are extracted to generate a corresponding empty belt binary image, where the red part is represented by white and the rest is represented by black.

[0125] like Figure 3 As shown, the color range of HSV is introduced:

[0126] Hue H: measured by angle, ranging from 0° to 360°, starting from red and counting counterclockwise, red is 0°, green is 120°, and blue is 240°. Their complementary colors are: yellow is 60°, cyan is 180°, and magenta is 300°; Figure 1 As shown:

[0127] Saturation S: value range is 0.0~1.0;

[0128] Brightness V: The value range is 0.0 (black) ~ 1.0 (white);

[0129] The range of HSV components corresponding to basic colors is shown in the following table, where the following table is based on the approximate color component division based on: H (0-180), S (0-255), V (0-255). In this invention, we need to extract the red component, Figure 3 The range within the red box is the range division of each red component.

[0130] Step S33: De-noising the binary image obtained in step S32 by using morphological denoising. First, an erosion operation is used to remove interfering noise points, and then a dilation operation is used to fill the holes.

[0131] Morphological denoising: The main function of mathematical morphology in image processing is to transform the geometric parameters or grayscale values ​​of the target in the image. Commonly used methods include erosion, dilation, opening operation, closing operation and morphological reconstruction. Dilation has the function of expanding the image, that is, expanding the edge of the image to fill the gap of the target edge or internal edge. Erosion has the function of shrinking the image. This operation can remove noise in the image and remove the "burrs" of the target edge.

[0132] Corrosion operations include:

[0133] Dilation is the operation of finding the local minimum. Convolution with B is to find the minimum value of the pixels in the area covered by B and assign the minimum value to the pixel specified by the reference point, thereby increasing the dark area. Its operator is "-":

[0134] dst(x,y)=min(src(x+x',y+y')), where (x',y'): element(x',y')≠0.

[0135] From the perspective of binary images, when the element values ​​corresponding to the convolution kernel are all 1, its value is 1, otherwise its value is changed to 0. Figure 4 As shown, this is an example diagram of the corrosion operation (the light blue part is the corrosion part):

[0136] Expansion operation:

[0137] Dilation is the operation of finding the local maximum value. Convolution with B is to find the maximum value of the pixels in the area covered by B and assign the maximum value to the pixel specified by the reference point, thereby increasing the highlight area. Its operator is "⊕":

[0138] dst(x,y)=max(src(x+x',y+y'))where (x',y'):element(x',y')≠0From the perspective of binary images, when one of the element values ​​corresponding to the convolution kernel is 1, its value is 1, otherwise its value is modified to 0. Figure 5 As shown, this is an example diagram of the expansion operation (the light gray part is the expansion part):

[0139] Step S34, curve compression is performed on the binary image of the empty belt obtained after the denoising process, and the abstract feature points of the empty belt are converted into more intuitive single-point curves, so as to reduce the amount of calculation for the subsequent area calculation.

[0140] Step S35: Then, the fault part of the compressed curve is interpolated using the equal division interpolation method.

[0141] Specifically, in step S34, curve compression is performed on the binary image of the empty belt obtained after the denoising process, including:

[0142] Step S341: Figure 6 As shown, take the middle white point in each column of the empty belt binary image.

[0143] Step S342: Figure 7 As shown, it is determined whether the white point is located in the specified analysis area. If the white point is located in the specified analysis area, the white point is retained. If the white point is not in the specified analysis area, the white point is changed to black and hidden. In this embodiment, for the convenience of display, the white point to be retained is drawn into a four-pointed star shape.

[0144] Specifically, in step S35, the fault part of the compressed curve is interpolated using an equal division interpolation method, including:

[0145] Step S351: Assume that the height difference between the white dots retained in the previous column and the white dots retained in the next column is ΔH. If ΔH≤200 or the input value is -1, perform interpolation processing, otherwise skip;

[0146] Step S352: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is greater than 100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the previous column;

[0147] Step S353: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is less than -100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the next column;

[0148] Step S354: When the situation is not included in step S351 to step S353, interpolate according to the equal value. The interpolation formula is as follows:

[0149]

[0150] Specifically, in step S4, based on the camera ranging principle, according to the basic parameters of the camera and the calibration information of the reference object, the initial space coordinates are established, including:

[0151] Step S41, selecting a reference object and obtaining the actual size of the reference object, and placing the reference object on an empty belt;

[0152] Step S42, obtaining the size and position information of the image of the reference object in the camera;

[0153] Step S43: constructing spatial coordinates using the camera ranging principle.

[0154] Specifically, in step S43, the spatial coordinates are constructed using the camera ranging principle, including:

[0155] Step S431, measuring the distance from any point in the camera imaging picture to the center of the camera lens, comprising:

[0156] like Figure 8 As shown, suppose any object point X on the belt. In this algorithm, for the convenience of calculation, the imaging principle of the camera is simplified to pinhole imaging. In simple terms, the straight line between the object point in the actual scene and the center point of the lens will pass through the image point in the film. The image is the film. The image point Y of the object point X in the image is measured. The distance OY from the image point Y to the center O of the camera lens is:

[0157] from Figure 8 It can be seen that

[0158] Where OT is the focal length of the camera; TY can be obtained by multiplying the pixel distance between points T and Y in the image (in pixels) by the actual film distance represented by each two adjacent pixels in the film;

[0159] Step S432, measuring the distance between two object points on the belt, comprising:

[0160] like Fig. 9 As shown, suppose that object points H and I on the belt, the center of the camera lens is O, the image point J of object point H in the imaged picture, the image point K of object point I in the imaged picture, the distances OH and OI are known, and the distance HI between object point H and object point I is calculated. In step S431, the distance from any image point in the film to the center O of the camera lens can be calculated. Therefore, OK, OJ and KJ are all known quantities. According to the law of cosines, we can get:

[0161]

[0162] And ∠HOI=∠KOJ, so

[0163]

[0164] Step S433, when the distances from two of the three collinear points on the belt to the camera are known, the position of the third point is calculated, including:

[0165] like Fig.10 As shown, suppose that the collinear object points A, B and C are on the belt, the center of the camera lens is O, the image point A in the image picture is A', the image point B in the image picture is B', the image point C in the image picture is C', OB and OA are known, find OC and CA:

[0166]

[0167] ∠AOC=∠A'OC';

[0168] Make AD perpendicular to the line OC and point D;

[0169] AD=OA*sin∠AOC;

[0170] Given OB and OA, the distance between the two points on the belt can be measured in step S432, and the AB distance can be obtained:

[0171]

[0172] When point C is between point A and point B or point C is outside point B, that is, point B and point C are on the same side of point A, we have:

[0173] ∠OAC=∠OAB;

[0174] Otherwise, when A is between points B and C, we have:

[0175] ∠OAC=π-∠OAB

[0176] Next step:

[0177] sin∠ACD=sin(∠OAC+∠AOC)

[0178] AC=AD / sin∠ACD

[0179]

[0180] At this time, OC and AC are obtained;

[0181] Step S434: In the actual scene, a plane is marked with several rectangles, and the distances from the four vertices of the rectangles to the center of the camera lens are measured. Then, the distance from any point on the plane where the rectangles are located to the center of the camera lens, as well as the actual distance between any two points, can be calculated. Fig.11 As shown:

[0182] It is known that the distances from the four vertices EFGH to the camera lens center O are OE, OF, OG, and OH. For any point TP1 in the image, draw a straight line through point TP1 that is parallel to EH and intersects the rectangle at points NP1 and NP2. Using the distances from two of the three collinear points on the belt to the camera in step S433, the position of the third point is calculated. The distance from the camera lens center O to point NP2 can be obtained from points H, G, and NP2, and the distance from the camera lens center O to point NP1 can be obtained from points E, F, and NP1. Then, the distance from point TP1 to the camera lens center O and the distance from point TP2 to the camera lens center O can be obtained through points NP1, NP2, and TP1. Then, the distance between two object points on the belt measured in step S432 can be used to obtain the distance from point TP1 to point TP2.

[0183] Step S435: When the distance from the vertex of the rectangle to the center of the camera lens is known, the point position of the plane where the rectangle is located is calculated. However, it is difficult to measure the distance from the vertex of the rectangle to the camera in practical applications. Therefore, a calibration method is now used to directly obtain the distance, including:

[0184] First, put the reference rectangle RSTU into the viewing range to obtain an image containing the rectangle, and measure the actual distances of the sides of the rectangle, that is, the lengths of RS, ST, TU, and UR are known, such as Fig.12 As shown:

[0185] Mark the four points R', S', T', and U' corresponding to the rectangle in the image. As can be seen from step S434, the angle between any two points and the center O of the camera lens can be obtained according to the coordinates of the four points, and then the angles of ∠ROS, ∠SOT, ∠TOU, and ∠UOR can be obtained;

[0186] Assume that the length of RO is x, then:

[0187]

[0188] ∠SRO=π-∠RSO-∠ROS

[0189]

[0190] ∠OST=π-∠SOT-∠STO

[0191]

[0192] That is, for any given x, the length of RT can be found, and the true value of the length of RT is:

[0193]

[0194] Select the possible value range of x-OR, then traverse all possible x values ​​within the value range according to the specific step size d, substitute the above process to obtain the theoretical value of RT and make a difference with the actual value, and the point with the smallest difference is recorded as x1,

[0195] Update the value range:

[0196] [x1-dtx1+d];

[0197] The update step size is:

[0198] d1=d / n;

[0199] Where n is the number of partition intervals in each iteration;

[0200] Traverse and solve again;

[0201] Repeat the above process until the error between the theoretical value and the true value of RT is small enough or the step length d is small enough, and the value of x at this time is set as the length of OR; the idea of ​​iterative solution is mainly used here. Since x has only one solution in the real number range, the above process can obtain the approximate solution x of OR.

[0202] After obtaining the length of OR, the distances of the other three sides OS, OT, and OU can be obtained using the related theorems of trigonometric functions;

[0203] When the actual side lengths of the four sides RS, ST, TU, UR of the reference rectangle RSTU and the side lengths of the four vertex coordinate systems of the reference object imaging figure R'S'T'U' are known, and the distances OR, OS, OT, OU from the four vertices of the reference rectangle RSTU to the center O of the camera lens are obtained, the distance between any two points in the plane of the rectangle in the camera imaging figure can be calculated through step S434.

[0204] Specifically, in step S6, Fig.13 As shown, according to the real-time coal flow laser line and the empty belt laser line, combined with the trapezoidal calculation method and the integral solution method, the cross-sectional area Mianji of the current interface is calculated, and then the coal flow volume v is calculated in combination with the actual belt running speed s. Finally, the current coal flow m is calculated by combining the coal flow density ρ and the coal flow volume v, including:

[0205] The area formed between the real-time coal flow laser line and the empty belt laser line is the cross-sectional area Mianji of the coal flow;

[0206] Take points (i, iiCoal[i]) and (i+1, iiCoal[i+1]) on the coal flow laser line;

[0207] Take points (i, iiEmptyBelt[i]) and (i+1, iiEmptyBelt[i+1]) on the empty belt laser line;

[0208] Points (i, iiCoal[i]), (i+1, iiCoal[i+1]), (i, iiEmptyBelt[i]), and (i+1, iiEmptyBelt[i+1]) form a trapezoid. The area of ​​the trapezoid, lenHor, is:

[0209] lenHor=0.5*(a+b)*h;

[0210] in,

[0211] a is the distance between point (i,iiCoal[i]) and point (i,iiEmptyBelt[i]);

[0212] b is the distance between point (i+1,iiCoal[i+1]) and point (i+1,iiEmptyBelt[i+1]);

[0213] h is the height of the trapezoid, and the value of h is 1 pixel;

[0214] The cross-sectional area Mianji of the coal flow is the sum of the areas of all trapezoids between the real-time coal flow laser line and the empty belt laser line. The expression of the cross-sectional area Mianji of the coal flow is:

[0215] Mianji = lenVer*len((i+1)-i); (loop accumulation)

[0216] Given the actual running speed s of the belt, the volume v of the coal flow is:

[0217] v = s*Mianji;

[0218] Given the coal density ρ, the coal flow rate m is:

[0219] m=ρ*v.

[0220] The specific embodiments described above further illustrate the technical problems, technical solutions and beneficial effects solved by the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for laser coal flow identification on coal conveyor belt based on video monitoring, It is characterized in that It includes: Step S1, using a 2-megapixel optical camera to scan the underground coal conveyor belt in real time, and then uploading the scanned belt image to the host computer for subsequent analysis; Step S2: The user draws a designated analysis area on the coal conveyor belt image according to actual needs, and configures basic camera parameters, which include image analysis size, focal length, film size, and camera film inch ratio; Step S3, according to the designated analysis area drawn by the user, extracting the empty belt laser line in the designated analysis area, wherein the laser line is the interface feature of the empty belt; Step S4: Based on the camera ranging principle, the initial space coordinates are established according to the basic parameters of the camera and the calibration information of the reference object; Step S5, according to the designated analysis area drawn by the user, extracting the real-time coal flow laser line in the designated analysis area, wherein the real-time coal flow laser line is the interface feature when the belt is transporting coal; Step S6, according to the real-time coal flow laser line and the empty belt laser line, combined with the trapezoidal calculation method and the integral solution method, calculate the cross-sectional area Mianji of the current interface, then calculate the coal flow volume v in combination with the actual belt running speed s, and finally calculate the current coal flow m in combination with the coal flow density ρ and the coal flow volume v; In step S4, based on the camera ranging principle, according to the basic parameters of the camera and the calibration information of the reference object, the initial space coordinates are established, including: Step S41, selecting a reference object and obtaining the actual size of the reference object, and placing the reference object on an empty belt; Step S42, obtaining the size and position information of the image of the reference object in the camera; Step S43, constructing space coordinates using the camera ranging principle; In step S43, the spatial coordinates are constructed using the camera ranging principle, including: Step S431, measuring the distance from any point in the camera imaging picture to the center of the camera lens, comprising: Assume an object point X on the belt, the image point Y of the object point X in the image, and measure the distance OY from the image point Y to the center O of the camera lens: Among them, OT is the focal length of the camera; TY can be obtained by multiplying the pixel distance between points T and Y in the image by the actual film distance represented by each two adjacent pixels in the film; Step S432, measuring the distance between two object points on the belt, comprising: Assume that object points H and I are on the belt, the center of the camera lens is O, the image point J of object point H in the imaged picture, the image point K of object point I in the imaged picture, the distances OH and OI are known, and the distance HI between object point H and object point I is calculated. The step S431 can calculate the distance from any image point in the film to the center O of the camera lens, so OK, OJ and KJ are all known quantities. According to the law of cosines, it can be obtained: And ∠HOI=∠KOJ, so Step S433, when the distances from two of the three collinear points on the belt to the camera are known, the position of the third point is calculated, including: Assume that there are collinear points A, B and C on the belt, the center of the camera lens is O, the image point A in the image picture is A', the image point B in the image picture is B', the image point C in the image picture is C', and OB and OA are known. Find OC and CA: ∠AOC=∠A'OC'; Make AD perpendicular to the line OC and point D; AD=OA*sin∠AOC; Given OB and OA, the distance between the two points on the belt can be measured in step S432, and the AB distance can be obtained: When point C is between point A and point B or point C is outside point B, we have: ∠OAC=∠OAB; Otherwise, when A is between points B and C, we have: ∠OAC=π-∠OAB; Next step: sin∠ACD=sin(∠OAC+∠AOC) AC=AD / sin∠ACD At this time, OC and AC are obtained; Step S434: In the actual scene, a plane is calibrated with several rectangles, and the distances from the four vertices of the rectangle to the center of the camera lens are measured. Then, the distance from any point on the plane where the rectangle is located to the center of the camera lens, as well as the actual distance between any two points, can be calculated: It is known that the distances from the four vertices EFGH to the camera lens center O are OE, OF, OG, and OH. For any point TP1 in the image, draw a straight line through point TP1 that is parallel to EH and intersects the rectangle at points NP1 and NP2. Using the distances from two of the three collinear points on the belt to the camera in step S433, the position of the third point is calculated. The distance from the camera lens center O to point NP2 can be obtained from points H, G, and NP2, and the distance from the camera lens center O to point NP1 can be obtained from points E, F, and NP1. Then, the distance from point TP1 to the camera lens center O and the distance from point TP2 to the camera lens center O can be obtained through points NP1, NP2, and TP1. Then, the distance between two object points on the belt measured in step S432 can be used to obtain the distance from point TP1 to point TP2. Step S435, when the distance from the vertex of the rectangle to the center of the camera lens is known, finding the point position on the plane where the rectangle is located, including: First, put the reference rectangle RSTU into the viewing range to obtain an image containing the rectangle, and measure the actual distances of the sides of the rectangle, that is, the lengths of RS, ST, TU, and UR are known: Mark the four points R', S', T', and U' corresponding to the rectangle in the image. It can be seen from step S434 that the angle between any two points and the center O of the camera lens can be obtained according to the coordinates of the four points, and then the sizes of ∠ROS, ∠SOT, ∠TOU, and ∠UOR can be obtained; Assume that the length of RO is x, then: ∠SRO=π-∠RSO-∠ROS ∠OST=π-∠SOT-∠STO That is, for any given x, the length of RT can be found, and the true value of the length of RT is: Select the possible value range of x-OR, then traverse all possible x values ​​within the value range according to the specific step size d, substitute the above process to obtain the theoretical value of RT and then make a difference with the actual value. The point with the smallest difference is recorded as x 1 , Update the value range: [x 1 -d,x 1 +d]; The update step size is: d 1 =d / n; Where n is the number of partition intervals in each iteration; Traverse and solve again; Repeat the above process until the error between the theoretical value and the true value of RT is small enough or the step length d is small enough, and the value of x at this time is set as the length of OR; After obtaining the length of OR, the distances of the other three sides OS, OT, and OU can be obtained using the related theorems of trigonometric functions; When the actual side lengths of the four sides RS, ST, TU, UR of the reference rectangle RSTU and the side lengths of the four vertices of the reference object imaging figure R'S'T'U' are known, and the distances OR, OS, OT, OU from the four vertices of the reference rectangle RSTU to the camera lens center O are obtained, the distance between any two points in the plane where the rectangle in the camera imaging figure is located can be calculated through step S434; In step S6, the cross-sectional area Mianji of the current interface is calculated according to the real-time coal flow laser line and the empty belt laser line, combined with the trapezoidal calculation method and the integral solution method, and then the coal flow volume v is calculated in combination with the actual belt running speed s, and finally the current coal flow m is calculated in combination with the coal flow density ρ and the coal flow volume v, including: The area formed between the real-time coal flow laser line and the empty belt laser line is the cross-sectional area Mianji of the coal flow; Take points (i, iiCoal[i]) and (i+1, iiCoal[i+1]) on the coal flow laser line; Take points (i, iiEmptyBelt[i]) and (i+1, iiEmptyBelt[i+1]) on the empty belt laser line; The points (i, iiCoal[i]), (i+1, iiCoal[i+1]), (i, iiEmptyBelt[i]) and (i+1, iiEmptyBelt[i+1]) form a trapezoid, and the area lenHor of the trapezoid is: lenHor=0.5*(a+b)*h; in, a is the distance between point (i,iiCoal[i]) and point (i,iiEmptyBelt[i]); b is the distance between point (i+1,iiCoal[i+1]) and point (i+1,iiEmptyBelt[i+1]); h is the height of the trapezoid, and the value of h is 1 pixel; The cross-sectional area Mianji of the coal flow is the sum of the areas of all trapezoids between the real-time coal flow laser line and the empty belt laser line. The expression of the cross-sectional area Mianji of the coal flow is: Mianji = lenVer*len((i+1)-i); Given the actual running speed s of the belt, the coal flow volume v is: v = s*Mianji; Given the coal density ρ, the coal flow rate m is: m=ρ*v.

2. The method for laser coal flow identification of coal conveyor belt based on video monitoring according to claim 1, It is characterized in that In the step S3, according to the designated analysis area drawn by the user, the empty belt laser line in the designated analysis area is extracted, and the laser line is the interface feature of the empty belt; Step S31, according to the designated analysis area drawn by the user, a laser light source is used to emit a laser line in the designated analysis area, and then a BGR picture of the designated analysis area of ​​the empty belt is taken by an optical camera, and the obtained BGR picture of the empty belt is converted into an HSV three-channel picture; Step S32, extracting all red pixels in the designated analysis area according to the HSV red feature value range and the vertices of the designated analysis area, generating a corresponding empty belt binary image, wherein the red part is represented by white and the rest is represented by black; Step S33, denoising the binary image obtained in step S32 by using morphological denoising, firstly using corrosion operation to remove interfering noise points, and then using dilation operation to fill holes; Step S34, performing curve compression on the binary image of the empty belt obtained after the denoising process, and converting the feature points of the empty belt into a single-point curve; Step S35: Then, the fault part of the compressed curve is interpolated using the equal division interpolation method.

3. The method for laser coal flow identification of coal conveyor belt based on video monitoring according to claim 2, It is characterized in that In the step S34, curve compression is performed on the binary image of the empty belt obtained after the denoising process, including: Step S341, taking the middle white point in each column of the empty belt binary image; Step S342: determine whether the white point is located in the specified analysis area. If the white point is located in the specified analysis area, keep the white point. If the white point is not in the specified analysis area, change the white point to black.

4. The method for laser coal flow identification of coal conveyor belt based on video monitoring according to claim 3, It is characterized in that In step S35, the fault part of the compressed curve is interpolated using an equal division interpolation method, including: Step S351: Assume that the height difference between the white dots retained in the previous column and the white dots retained in the next column is ΔH. If ΔH≤200 or the input value is -1, perform interpolation processing, otherwise skip; Step S352: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is greater than 100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the previous column; Step S353: when the vertical division value between the white dots retained in the previous column and the white dots retained in the next column is less than -100 and the horizontal division value is greater than 100, the missing value in the middle is equal to the value of the white dots retained in the next column; Step S354: When the situation is not included in step S351 to step S353, interpolate according to the equal value. The interpolation formula is as follows:

Citation Information

Patent Citations

  • The Fiber Bragg Grating Intelligent Device And Method For Monitoring Coal Level In Bunker

    AU2020100848A4

  • Coal flow monitoring system based on laser radar and speed information

    CN107101683A

  • Coal flow detection method based on surface laser radar

    CN111747062A

  • Method and system for identifying coal flow in conveying belt based on video monitoring

    CN111814678A