Experiment table and method for simulating and monitoring coal flow volume of scraper conveyor of fully mechanized coal mining face

Abstract

The invention provides an experiment table and method for simulating and monitoring the coal flow volume of a scraper conveyor of a fully mechanized coal mining face, and belongs to the technical field of coal flow volume monitoring. And as a reference basis for the scraper conveyor to adjust the coal conveying speed according to the change of the coal flow in an actual fully-mechanized coal mining face, the working efficiency of the fully-mechanized coal mining face can be improved. The experiment table comprises a conveyor simulation device, a coal mining simulation device, at least one binocular camera, an upper computer and a conveying control board.

Description

Technical Field

[0001] The present invention relates to the technical field of coal flow volume monitoring, and particularly to an experimental bench and method for simulating and monitoring the coal flow volume of a scraper conveyor in a fully mechanized coal mining face. Background Art

[0002] As an important energy source, coal is often mined and transported using a fully mechanized coal mining face. Specifically, a fully mechanized coal mining face includes equipment such as a coal shearer, a scraper conveyor for transporting coal, and a hydraulic support. These devices work together within the fully mechanized coal mining face to achieve coal mining and transportation.

[0003] During the process of coal mining and transportation, a scraper conveyor is often used to transport coal. However, since the coal flow rate obtained from the fully mechanized coal mining face is not stable, and the existing scraper conveyor cannot adjust the coal conveying speed according to the change in coal flow rate, it is impossible to achieve coordinated coal mining and transportation, resulting in low work efficiency of the fully mechanized coal mining face. Summary of the Invention

[0004] The present invention provides an experimental bench and method for simulating and monitoring the coal flow volume of a scraper conveyor in a fully mechanized coal mining face. The experimental bench simulates and monitors the real-time coal flow volume of the scraper conveyor in the fully mechanized coal mining face, and adjusts the coal conveying speed of the scraper conveyor according to the coal flow volume. Under the condition of achieving coordinated coal mining and transportation, the process of adjusting the coal conveying speed of the scraper conveyor according to the coal flow volume is used as a reference for the scraper conveyor in the actual fully mechanized coal mining face to adjust the coal conveying speed according to the change in coal flow rate, thereby helping to improve the work efficiency of the fully mechanized coal mining face.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] An experimental bench for simulating and monitoring the coal flow volume of a scraper conveyor in a fully mechanized coal mining face. The fully mechanized coal mining face includes a coal shearer, a scraper conveyor for transporting coal, and a hydraulic support. The experimental bench includes:

[0007] A conveyor simulation device for simulating the operation of the scraper conveyor, and a middle trough for carrying coal is arranged on the conveyor simulation device;

[0008] A coal mining simulation device for simulating the movement state of the hydraulic support relative to the scraper conveyor during coal mining. The coal mining simulation device includes at least one combined guide rail; each of the at least one combined guide rails includes a transverse guide rail slide and a longitudinal guide rail slide slidably connected to the transverse guide rail slide. The transverse guide rail slide is used to simulate the coal pushing action and the frame pulling action of the hydraulic support, and the longitudinal guide rail slide is used to simulate the frame lowering action and the frame raising action of the hydraulic support;

[0009] At least one binocular camera, which is used to collect real-time images of the coal flow on the conveyor simulation device, obtain multiple coal flow images, and extract the point cloud data of each coal flow image among the multiple coal flow images; each binocular camera in the at least one binocular camera is correspondingly arranged on the longitudinal guide rail slide of the combined guide rail; each binocular camera includes a left camera and a right camera;

[0010] A host computer, which is used to receive in real time the point cloud data of each coal flow image among the multiple coal flow images extracted by at least one binocular camera, generate a coal flow contour surface mesh based on the point cloud data, and calculate the coal flow volume on the conveyor simulation device through the coal flow contour surface mesh; the host computer is electrically connected to at least one binocular camera;

[0011] A conveying console, which is used to control the speed of the conveyor simulation device according to the coal flow volume on the conveyor simulation device; the conveying console is electrically connected to the host computer and the conveyor simulation device;

[0012] A guide rail control device, which is used to control at least one combined guide rail in the simulation device to simulate the action changes of the fully-mechanized mining equipment and the scraper conveyor during the coal mining process; the guide rail control device is electrically connected to the coal mining simulation device.

[0013] In one implementation manner of the present invention, extracting the point cloud data of each coal flow image among the multiple coal flow images includes:

[0014] For each binocular camera in the at least one binocular camera, obtain the disparity information between the left camera and the right camera in the binocular camera, and calculate the depth value corresponding to each pixel point in each coal flow image among the multiple coal flow images through the disparity information;

[0015] Calculate the point cloud data of each coal flow image among the multiple coal flow images by using the depth value, the internal parameters of the left camera, and the internal parameters of the right camera.

[0016] In one implementation manner of the present invention, generating a coal flow contour surface mesh based on the point cloud data includes:

[0017] Adopt image stitching technology to fuse the point cloud data of each coal flow image among the multiple coal flow images after denoising and downsampling processing;

[0018] Identify the boundary and shape features of the fused point cloud data, and extract the coal flow contour line of the fused point cloud data;

[0019] Calculate the normal vector of the coal flow contour line, and construct a coal flow contour surface mesh according to the normal vector of the contour line; where the normal vector of the coal flow contour line satisfies the following formula:

[0020] E 3×3 ·vj = λ j ·v j ,where j ∈ {1, 2, 3}

[0021] where E 3×3 represents the point neighborhood covariance matrix of any point p on the coal flow contour line; λ j represents the j-th eigenvalue in E 3×3 ; v j represents the eigenvector corresponding to the j-th eigenvalue in E 3×3 ; the eigenvector corresponding to the smallest eigenvalue in E 3×3 is the normal vector of point p.

[0022] In one implementation of the present invention, the coal flow volume on the conveyor simulation device is calculated through the surface grid of the coal flow contour, including:

[0023] Calculating the coal flow volume on the conveyor simulation device in the world coordinate system according to the surface grid of the coal flow contour; the coal flow volume on the conveyor simulation device in the world coordinate system satisfies the following formula;

[0024]

[0025] where V represents the coal flow volume on the conveyor simulation device, Δ j represents the j-th triangle obtained after Delaunay triangulation of the surface grid of the coal flow contour in the world coordinate system, S(Δ j ) represents the area of the j-th triangle, Z j1 represents the coordinate value of the first vertex of the j-th triangle on the z-axis in the world coordinate system; Z j2 represents the coordinate value of the second vertex of the j-th triangle on the z-axis in the world coordinate system; Z j3 represents the coordinate value of the third vertex of the j-th triangle on the z-axis in the world coordinate system; V0 represents the volume occupied by the middle trough on the conveyor simulation device under no-load conditions.

[0026] In one implementation of the present invention, the transverse guide rail slider is C-shaped. One end of the transverse guide rail slider is fixed to one side of the scraper conveyor. A transverse first ball screw is arranged inside both ends of the transverse guide rail slider. One end of the first ball screw is rotatably connected to one end of the transverse guide rail slider, and the other end of the first ball screw is connected to the output end of the first motor. The first motor is fixed to the other end of the transverse guide rail slider;

[0027] A connecting piece is fixed on the first ball screw, and a longitudinal guide rail sliding table is fixed above the connecting piece. The longitudinal guide rail sliding table is C-shaped, and second ball screws in the longitudinal direction are arranged at both ends of the longitudinal guide rail sliding table. One end of the second ball screw is rotationally connected to one end of the longitudinal guide rail sliding table, and the other end of the second ball screw is connected to the output end of the second motor. The second motor is fixed at the other end of the transverse guide rail sliding table;

[0028] A rotary servo is fixed on the second ball screw, and a binocular camera is arranged on the rotary servo. The binocular camera rotates with the rotation of the rotary servo.

[0029] The present invention also provides a method for simulating and monitoring the coal flow volume of a scraper conveyor in a fully mechanized coal mining face. Based on the above experimental bench, it includes:

[0030] Taking the middle trough of the scraper conveyor covered by the field of view of the binocular camera as the target, calibrating at least one binocular camera;

[0031] The guide rail control device controls the transverse sliding of the transverse guide rail sliding table in at least one combined guide rail and controls the longitudinal sliding of the longitudinal guide rail sliding table in at least one combined guide rail to simulate the action changes of the fully mechanized coal mining equipment and the scraper conveyor during the coal mining process;

[0032] Real-time collection of coal on the scraper conveyor is carried out through at least one binocular camera to obtain multiple coal flow images, and the point cloud data of each coal flow image in the multiple coal flow images is extracted;

[0033] The upper computer receives in real time the point cloud data of each coal flow image in the multiple coal flow images extracted by at least one binocular camera, generates a coal flow contour surface mesh according to the point cloud data, and calculates the coal flow volume on the conveyor simulation device through the coal flow contour surface mesh;

[0034] The conveying console controls the speed of the conveyor simulation device according to the coal flow volume on the conveyor simulation device.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] In the on-shore environment, the present invention simulates the relative movements of the scraper conveyor, the shearer, and the hydraulic support in the actual coal mining process of the fully mechanized coal mining face through an experimental bench, calculates the coal flow volume on the conveyor simulation device by collecting images through a binocular camera, and then uses the conveying console to adjust the conveying speed of the conveyor simulation device for coal according to the coal flow volume, realizing the coordinated coal mining and conveying during this process. In the case of realizing the coordinated coal mining and conveying, the process of adjusting the conveying speed of the scraper conveyor according to the coal flow volume can be used as a reference for the actual scraper conveyor in the fully mechanized coal mining face to adjust the conveying speed according to the change of the coal flow rate, thus helping to improve the working efficiency of the fully mechanized coal mining face. Description of the Drawings

[0037] Figure 1 It is one of the schematic diagrams of the experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face provided by the embodiments of the present application;

[0038] Figure 2 It is the second schematic diagram of the experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face provided by the embodiments of the present application;

[0039] Figure 3 It is the third schematic diagram of the experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face provided by the embodiments of the present application;

[0040] Figure 4 It is the schematic diagram of the method for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face provided by the embodiments of the present application;

[0041] Among them, 1-conveyor simulation device, 2-combined guide rails, 21-lateral guide rail slider, 211-first ball screw, 212-first motor, 22-longitudinal guide rail slider, 221-second ball screw, 222-second motor, 223-rotary steering gear, 3-binocular camera, 4-host computer, 5-conveyor console, 6-guide rail control device. Detailed implementation manners

[0042] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner.

[0043] In the description of the present invention, unless otherwise specified, the meaning of "a plurality of" refers to two or more. For example, at least one binocular camera means two or more binocular cameras.

[0044] The method and device provided by the embodiments of the present application relate to coal flow volume monitoring, and can simulate and monitor the coal flow volume and adjust the coal transportation speed to achieve the coordination of coal mining and transportation, and use the above strategy of adjusting the coal transportation speed according to the coal flow volume to achieve the coordination of coal mining and transportation as a reference basis for the actual operation of the fully-mechanized mining face. Specifically, a binocular camera is used to monitor the coal flow rate on the conveyor simulation device to obtain the point cloud data of multiple coal flow rate images, and then the point cloud data of multiple coal flow rate images is processed to obtain the coal flow volume on the conveyor simulation device. Finally, the transportation speed of the conveyor simulation device is adjusted according to the coal flow volume on the conveyor simulation device to achieve the coordination of coal mining and transportation.

[0045] It is understandable that the fully mechanized coal mining face (fully known as the fully mechanized coal winning face) is the main link in modern coal mine production, including mechanical equipment such as a shearer for cutting coal, a scraper conveyor for transporting coal, and a hydraulic support for supporting the shearer and adjusting the cutting position of the shearer for coal. The fully mechanized coal mining face realizes the mining and transportation of coal by comprehensively applying mechanical equipment such as a shearer, a hydraulic support, and a scraper conveyor.

[0046] Specifically, in order to realize the mining and transportation of coal, the working process of the fully mechanized coal mining face includes processes such as coal cutting, support moving, conveyor pushing, transfer, and transportation. The following briefly introduces the working process of the fully mechanized coal mining face.

[0047] Coal cutting: The shearer, in cooperation with the scraper conveyor, cuts the coal seam according to a certain coal cutting method and feed method, and mines the coal from the coal seam. During the coal cutting process, the cutting drum of the shearer rotates continuously, and at the same time, the shearer moves along the direction of the scraper conveyor to realize continuous cutting of the coal seam.

[0048] Support moving: After the shearer cuts the coal, the hydraulic support moves to a new working position in a timely manner to support the newly exposed roof. During the support moving process, the push jack of the hydraulic support pushes the support forward, and at the same time adjusts the support height and support strength of the support to ensure the stability of the roof.

[0049] Conveyor pushing: After the support is moved, the scraper conveyor needs to be pushed forward to ensure the normal operation of the shearer and the smooth transportation of coal. When pushing the scraper conveyor, the push jack of the scraper conveyor pushes the scraper conveyor forward, and at the same time adjusts the position and attitude of the scraper conveyor to make it adapt to the working positions of the shearer and the hydraulic support.

[0050] Transfer and transportation: The scraper conveyor transports the mined coal to the transfer machine, the transfer machine then transfers the coal to the belt conveyor, and the belt conveyor transports the coal to the ground or other designated locations. During the transfer and transportation process, attention needs to be paid to the safety of coal transfer and transportation to prevent coal spillage and equipment failures.

[0051] In order to solve the problem in the background technology that the existing scraper conveyor cannot adjust the coal conveying speed according to the change of coal flow rate, and cannot achieve the coordination of coal mining and transportation, resulting in low working efficiency of the fully mechanized coal mining face, the embodiment of the present application provides an experimental bench and method for simulating and monitoring the coal flow volume of the scraper conveyor in the fully mechanized coal mining face. The experimental bench simulates and monitors the real-time coal flow volume of the scraper conveyor in the fully mechanized coal mining face, and adjusts the coal conveying speed of the scraper conveyor according to the coal flow volume. In the case of realizing the coordination of coal mining and transportation, the process of adjusting the coal conveying speed of the scraper conveyor according to the coal flow volume is used as a reference basis for the scraper conveyor in the actual fully mechanized coal mining face to adjust the coal conveying speed according to the change of coal flow rate, thereby helping to improve the working efficiency of the fully mechanized coal mining face.

[0052] Before introducing the experimental platform and method for simulating and monitoring the coal flow volume of the scraper conveyor in the fully mechanized coal mining face provided by the embodiments of the present application, some professional terms in this technical field will be explained to facilitate the understanding of the technical solutions provided by the embodiments of the present application.

[0053] 1. Lowering the support - pulling the support - raising the support - pushing the scraper conveyor

[0054] During the coal mining process in the fully mechanized coal mining face, the hydraulic support realizes the support for the shearer and adjusts the cutting position of the shearer for coal by repeatedly executing the actions of "lowering the support - pulling the support - raising the support - pushing the scraper conveyor".

[0055] Specifically, lowering the support means operating the columns of the hydraulic support to lower the roof beam and disengage from the roof, preparing for moving the support. Pulling the support means that after lowering the support, operating the push jack to move the support one step along the advancing direction of the working face. Raising the support means that after the support reaches the position, operating the columns to raise the roof beam and re - support the roof to ensure the support strength. Pushing the scraper conveyor means that after the hydraulic support is firmly supported, operating the push jack to push the scraper conveyor towards the coal wall to complete a cycle.

[0056] 2. Camera coordinate system, pixel coordinate system, image coordinate system, and world coordinate system

[0057] In the field of image processing technology, camera calibration and three - dimensional reconstruction of objects (coal in the embodiments of the present application) are often carried out in combination with the camera coordinate system, pixel coordinate system, image coordinate system, and world coordinate system.

[0058] Among them, the world coordinate system (World Coordinate System) is used to describe the position of an object in the real three - dimensional space, with the unit of meters or millimeters. The world coordinate system usually takes a certain fixed reference point as the origin, and the x - axis, y - axis, and z - axis form a three - dimensional rectangular coordinate system.

[0059] The camera coordinate system (Camera Coordinate System) is used to describe the position of an object relative to the camera. The camera coordinate system takes the camera optical center as the origin, and the Z - axis of the camera coordinate system is along the optical axis direction, and the X - axis and Y - axis are parallel to the image plane. The camera coordinate system is a three - dimensional rectangular coordinate system, usually taking the optical center of the camera as the origin, the x - axis and y - axis are respectively parallel to the horizontal and vertical directions of the image plane, and the z - axis points in front of the camera, consistent with the optical axis direction.

[0060] The image coordinate system (Image Coordinate System) is used to describe the projection position of an object on the image plane, with the unit of millimeters. The image coordinate system is a two - dimensional rectangular coordinate system, usually taking the center of the image as the origin, and the x - axis and y - axis are respectively parallel to the horizontal and vertical directions of the image.

[0061] The Pixel Coordinate System is used to describe the specific position of an object in a digital image, with the unit being pixels. The Pixel Coordinate System is a two-dimensional rectangular coordinate system, usually with the upper left corner of the image as the origin, and the u-axis and v-axis representing the horizontal and vertical pixel positions in the image respectively.

[0062] The above four coordinate systems can be mutually converted through conversion relationships.

[0063] The above world coordinate system and the above camera coordinate system can be converted through the external parameters of the camera (rotation matrix R and translation vector t). This conversion relationship satisfies the following formula (1).

[0064]

[0065] Among them, (X C , Y C , Z C ) represents the coordinates in the camera coordinate system, and (X W , Y W , Z W ) represents the coordinates in the world coordinate system.

[0066] The above camera coordinate system and the above image coordinate system can be converted through perspective projection. This conversion relationship satisfies the following formula (2).

[0067]

[0068] Among them, (x, y) represents the coordinates in the image coordinate system, and f represents the focal length of the camera.

[0069] The above image coordinate system and the above pixel coordinate system can be converted through the internal parameter matrix K of the camera. This conversion relationship satisfies the following formula (3).

[0070]

[0071] Among them, (u, v) represents the coordinates in the pixel coordinate system. The internal parameter matrix K satisfies the following formula.

[0072]

[0073] Among them, f x represents the equivalent focal length of the focal length on the x-axis of the image coordinate system, f y represents the equivalent focal length of the focal length on the y-axis of the image coordinate system, and (c x , c y ) is the principal point coordinate, referring to the position of the origin of the image coordinate system in the pixel coordinate system.

[0074] In summary, the embodiments of the present application can convert the coordinates of coal in the image into the world coordinate system through the above formulas (1) to (3).

[0075] Next, an experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face provided by the embodiments of the present application will be introduced.

[0076] Exemplarily, as Figure 1 shown, the embodiments of the present application provide an experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor in the fully-mechanized mining face. The experimental platform includes a conveyor simulation device 1, a coal mining simulation device, a binocular camera 3, a host computer 4, and a conveyor console 5.

[0077] The conveyor simulation device 1 is used to simulate the operation of the scraper conveyor. A middle trough for carrying coal is provided on the conveyor simulation device 1.

[0078] Optionally, the above conveyor simulation device 1 is a scaled-down scraper conveyor.

[0079] Specifically, the above conveyor simulation device 1 includes a head, a middle trough, and a tail fixedly connected in sequence. The head is provided with a driving sprocket, and the tail is provided with a driven sprocket. Both the driving sprocket and the driven sprocket are engaged with the scraper chain. A driving motor is provided outside the head, and the output end of the driving motor is rotationally connected to the driving sprocket, so that the scraper chain circulates in the middle trough driven by the driving sprocket and the driven sprocket. In addition, a guide rail is provided in the middle trough body, and the guide rail is used to guide the smooth movement of the scraper chain and carry materials. A tensioning device is also provided at the tail part, and the tensioning device is used to support the chain and adjust its tightness. A plurality of scrapers are provided on the scraper chain, and the scrapers are used to push the coal from the feed port to the discharge port.

[0080] The coal mining simulation device is used to simulate the movement state of the hydraulic support relative to the scraper conveyor during the coal mining process. The coal mining simulation device includes a combined guide rail 2; each combined guide rail 2 in a combined guide rail 2 includes a transverse guide rail slide 21 and a longitudinal guide rail slide 22 slidably connected to the transverse guide rail slide 21. The transverse guide rail slide 21 is used to simulate the pushing and pulling actions of the hydraulic support, and the longitudinal guide rail slide 22 is used to simulate the lowering and raising actions of the hydraulic support. Optionally, the coal mining simulation device may also include two combined guide rails 2 (as Figure 2 shown) or more than two combined guide rails 2 (as Figure 3 shown). The embodiments of the present application do not limit the number of combined guide rails 2.

[0081] In one implementation manner, the above-mentioned horizontal guide rail slider 21 is C-shaped. One end of the horizontal guide rail slider 21 is fixed to one side of the scraper conveyor. There is a horizontal first ball screw 211 arranged inside both ends of the horizontal guide rail slider 21. One end of the first ball screw 211 is rotationally connected to one end of the horizontal guide rail slider 21, and the other end of the first ball screw 211 is connected to the output end of the first motor 212. The first motor 212 is fixed to the other end of the horizontal guide rail slider 21. A connecting piece is fixed on the above-mentioned first ball screw 211, and a longitudinal guide rail slider 22 is fixed above the connecting piece. The longitudinal guide rail slider 22 is C-shaped. There is a longitudinal second ball screw 221 arranged inside both ends of the longitudinal guide rail slider 22. One end of the second ball screw 221 is rotationally connected to one end of the longitudinal guide rail slider 22, and the other end of the second ball screw 221 is connected to the output end of the second motor 222. The second motor 222 is fixed to the other end of the horizontal guide rail slider 21. A rotary servo 223 is fixed on the second ball screw 221, and the binocular camera 3 is arranged on the rotary servo 223. The binocular camera 3 rotates with the rotation of the rotary servo 223.

[0082] The binocular camera 3 is used to perform real-time image acquisition on the coal flow rate on the conveyor simulation device 1 to obtain multiple coal flow rate images, and extract the point cloud data of each coal flow rate image in the multiple coal flow rate images. The binocular camera 3 is correspondingly arranged on the longitudinal guide rail slider 22 of the combined guide rail 2. Each binocular camera 3 includes a left camera and a right camera. Optionally, two binocular cameras 3 (as shown in Figure 2 ) or more than two binocular cameras 3 (as shown in Figure 3 ) can also be set on the above-mentioned test bench. The number of binocular cameras 3 is not limited in the embodiments of the present application.

[0083] The host computer 4 is used to receive in real time the point cloud data of each coal flow rate image in the multiple coal flow rate images extracted by the binocular camera 3, generate a coal flow contour surface mesh according to the point cloud data, and calculate the coal flow volume on the conveyor simulation device 1 through the coal flow contour surface mesh; the host computer 4 is electrically connected to the binocular camera 3.

[0084] The conveying console 5 is used to control the speed of the conveyor simulation device 1 according to the coal flow volume on the conveyor simulation device 1. The conveying console 5 is electrically connected to the host computer 4 and the conveyor simulation device 1.

[0085] The guide rail control device 6 is used to control the action changes of the combined guide rail 2 in the simulation device to simulate the fully-mechanized mining equipment and the scraper conveyor during the coal mining process; the guide rail control device 6 is electrically connected to the coal mining simulation device.

[0086] It can be understood that the above-mentioned conveying console 5 can control the conveyor simulation device 1 through a Programmable Logic Controller (PLC for short). Similarly, the above-mentioned guide rail control device 6 can also control the coal mining simulation device through the PLC. Since the above content belongs to common technical means in this field, the embodiments of the present application will not elaborate further herein.

[0087] For the binocular camera 3, the point cloud data of each coal flow image in multiple coal flow images is extracted, including the following steps.

[0088] Step 1: For the binocular camera 3, obtain the disparity information between the left camera and the right camera in the binocular camera 3, and calculate the depth value corresponding to each pixel point in each coal flow image in multiple coal flow images through the disparity information.

[0089] It can be understood that the disparity information between the left camera and the right camera in the above-mentioned binocular camera 3 refers to the difference between the pixel positions of the same object in the image captured by the left camera and the image captured by the right camera, that is, the disparity between the matching feature points between the image captured by the left camera and the image captured by the right camera. The disparity information between the left camera and the right camera in the above-mentioned binocular camera 3 can be obtained through the following steps.

[0090] Step 1: Image correction

[0091] Align the epipolar lines of the image captured by the left camera with the epipolar lines of the image captured by the right camera through stereo correction.

[0092] Step 2: Feature extraction

[0093] Use a feature extraction algorithm to extract feature points that can be used for matching from the image captured by the left camera and the image captured by the right camera. Optionally, the above-mentioned feature extraction algorithm can be algorithms such as SIFT, SURF, ORB, etc. The embodiments of the present application do not limit this here, and since the above algorithms belong to common algorithms in this technical field, the embodiments of the present application do not elaborate on the specific implementation process of the above algorithms.

[0094] Step 3: Feature matching

[0095] Use a feature matching algorithm to calculate the disparity between the matching feature points between the image captured by the left camera and the image captured by the right camera. Exemplarily, the above-mentioned feature matching algorithm can be algorithms such as BFMatcher, FLANN, etc. The embodiments of the present application do not limit this here, and since the above algorithms belong to common algorithms in this technical field, the embodiments of the present application do not elaborate on the specific implementation process of the above algorithms.

[0096] Step 4: Disparity map calculation

[0097] Generate a disparity map using the SGBM (Semi - Global Block Matching) or BM (Block Matching) algorithm. The disparity map is used to represent the disparity value of each matched pixel.

[0098] Based on the disparity information between the left camera and the right camera in the binocular camera 3 obtained by the above method, calculate the depth value corresponding to each pixel point in each coal flow image among multiple coal flow images. The above depth value satisfies the following formula.

[0099]

[0100] Among them, Z represents the depth value, f represents the focal length, B represents the baseline, that is, the optical center distance between the left camera and the right camera, and d represents the disparity value.

[0101] Step 5: Use the depth value, the internal parameters of the left camera, and the internal parameters of the right camera to calculate the point cloud data of each coal flow image among multiple coal flow images.

[0102] The internal parameters of the above - mentioned left camera satisfy the following formula.

[0103]

[0104] Among them, f xl represents the equivalent focal length of the left camera on the x - axis of the image coordinate system, f yl represents the equivalent focal length of the left camera on the y - axis of the image coordinate system, (c xl , c yl ) represents the principal point coordinates of the left camera, referring to the position of the origin of the image coordinate system in the pixel coordinate system.

[0105] The internal parameters of the above - mentioned right camera satisfy the following formula.

[0106]

[0107] Among them, f xr represents the equivalent focal length of the right camera on the x - axis of the image coordinate system, f yr represents the equivalent focal length of the right camera on the x - axis of the image coordinate system, (c xr , c yr ) represents the principal point coordinates of the right camera.

[0108] For the host computer 4, the calculation process of the point cloud data of each coal flow image among the above - mentioned multiple coal flow images is as follows.

[0109] Step 1: Obtain the pixel coordinates

[0110] Obtain the pixel coordinates (u l of each pixel point from each coal flow image among multiple coal flow images., u l )。

[0111] Step 2, Normalize coordinates

[0112] Convert pixel coordinates to normalized coordinates, and this conversion process satisfies the following formula.

[0113]

[0114] Step 3, Calculate three-dimensional coordinates

[0115] Calculate the three-dimensional coordinates of each pixel point using the depth value Z, and this three-dimensional coordinate satisfies the following formula.

[0116] X = x l ·Z

[0117] Y = y l ·Z

[0118] Z = Z

[0119] Step 4, Generate point cloud data

[0120] Point Cloud Data is a set composed of a large number of points in three-dimensional space; combine the three-dimensional coordinates (X, Y, Z) of each pixel point obtained in the above Step 3 to obtain point cloud data.

[0121] Optionally, for the host computer 4, generating a coal flow contour surface mesh based on the point cloud data includes the following steps.

[0122] Step 1, Adopt image stitching technology to fuse the point cloud data of each coal flow image in multiple denoised and downsampled coal flow images.

[0123] After combining the point cloud data from multiple three-dimensional coordinates (X, Y, Z), it is necessary to implement the above Step 1 through image stitching, point cloud alignment, and point cloud fusion, and finally obtain a three-dimensional model of the coal flow in the middle trough of the conveyor simulation device 1. Among them, the purpose of image stitching is to stitch multiple coal flow images into a panoramic image, and the process of image stitching is achieved through sub-steps such as feature extraction, feature matching, image registration, and image fusion. The purpose of point cloud alignment is to align the point cloud data of multiple coal flow images to the same coordinate system, and the process of point cloud alignment is achieved through sub-steps such as feature extraction, feature matching, transformation estimation, and point cloud transformation. The purpose of point cloud fusion is to fuse the aligned point cloud data into a point cloud data set, and the process of point cloud fusion includes sub-steps such as point cloud filtering, point cloud sampling, point cloud merging, and point cloud optimization to generate a smooth three-dimensional model.

[0124] It can be understood that since the operations of the above-mentioned image stitching, point cloud alignment, and point cloud fusion belong to common techniques in the technical field, the embodiments of the present application will not further explain the above operations.

[0125] Step 2: Identify the boundary and shape features of the fused point cloud data, and extract the coal flow contour line of the fused point cloud data.

[0126] As can be seen from the above, the three-dimensional model of the coal flow in the middle trough of the conveyor simulation device 1 is obtained in the above Step 1, and the coal flow contour line is extracted from the three-dimensional model of the coal flow in the middle trough of the conveyor simulation device 1 in this step.

[0127] Specifically, the extraction of the coal flow contour line from the fused point cloud data is realized by three sub-steps: point cloud data preprocessing, point cloud data projection, and contour extraction.

[0128] Among them, the purpose of the above-mentioned point cloud data preprocessing is to remove noise and outliers and simplify the point cloud data. The process of point cloud data preprocessing includes filtering and downsampling. Exemplarily, statistical filtering, radius filtering, etc. can be used for filtering to remove noise; voxel grid filtering, etc. can be used for downsampling to reduce the amount of point cloud data.

[0129] The purpose of the above-mentioned point cloud data projection is to project the three-dimensional point cloud data onto a two-dimensional plane for easy contour extraction. The process of the above-mentioned point cloud data projection includes two sub-steps: plane projection (projecting the point cloud data onto a certain plane, such as the XY plane) and perspective projection (projecting the point cloud data onto a two-dimensional image plane according to the perspective).

[0130] The purpose of the above-mentioned contour extraction is to extract the contour line from the projected two-dimensional data. The method for extracting the contour line can be the convex hull algorithm, the Alpha Shapes algorithm, or the Canny edge detection algorithm, which is not limited in the embodiments of the present application.

[0131] Step 3: Calculate the normal vector of the coal flow contour line, and construct the coal flow contour surface grid according to the normal vector of the contour line.

[0132] For each point p on the coal flow contour line i , calculate the covariance matrix E of the neighborhood points of point p i , and solve its eigenvalues and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector n of this point 3×3 . i .

[0133] The above covariance matrix E 3×3 has the following calculation formula.

[0134]

[0135] Among them, pi is the point p i is a neighborhood point, μ is the mean of the neighborhood points, and N is the number of neighborhood points.

[0136] The normal vector of the coal flow contour line satisfies the following formula, which is also the formula for solving eigenvalues and eigenvectors.

[0137] E 3×3 ·v j = λ j ·v j , j ∈ {1, 2, 3}

[0138] Among them, E 3×3 represents the point neighborhood covariance matrix of any point p on the coal flow contour line; λ j represents the j-th eigenvalue in E 3×3 , and v j represents the eigenvector corresponding to the j-th eigenvalue in E 3×3 ; the eigenvector v 3×3 corresponding to the smallest eigenvalue in E mjn is the normal vector of point p. It can be understood that by obtaining the normal vector of each point on the coal flow contour line, the normal vector of the coal flow contour line is obtained.

[0139] Furthermore, after obtaining the normal vector of the coal flow surface contour, the process of Delaunay triangulation of the coal flow contour is constrained by the normal vector of the coal flow surface contour.

[0140] Specifically, the content of this constraint is to map the two-dimensional Delaunay triangle back to three-dimensional space and constrain the grid direction of the Delaunay triangle by the normal vector. The process of this constraint includes normal vector consistency check and triangle normal vector direction correction, so as to ensure the closure of the grid surface of the coal flow contour surface.

[0141] It can be understood that the core of Delaunay triangulation is the empty circle property: for any triangle, its circumcircle does not contain other points. Mathematically, the triangles obtained by the Delaunay triangulation of the point set P = {p i} satisfy and there are no other p m ∈ P inside its circumcircle. Since Delaunay triangulation is a commonly used technical means in this technical field, the specific composition of Delaunay triangulation is not further elaborated in the embodiments of this application.

[0142] The following details the process of constraining the triangles obtained by Delaunay triangulation of the coal flow contour through the above normal vector consistency check and the above triangle normal vector direction correction.

[0143] (1) Normal vector consistency check

[0144] According to the cosine value of the angle θ between two normal vectors, determine whether the directions of the two normal vectors are similar. For a candidate triangle Δp i p j p k of an edge p i p j , determine whether the cosine value of the angle θ between the normal vectors at both ends of the edge p i p j is less than the threshold ∈. If cosθ = n i ·n j > ∈, it is considered that the directions of the two normal vectors are similar, and these two points can be connected; otherwise, it is considered that the directions are not similar, and the connection is rejected to avoid incorrect cross-surface meshing.

[0145] (2) Triangular normal vector direction correction

[0146] For the triangle Δp i p j p k obtained by Delaunay triangulation, calculate the normal vector n i p j p k of the triangle Δp T . The normal vector n i p j p k of the triangle Δp T satisfies the following formula.

[0147] n T =(p j -p i )×(p k -p i )

[0148] Adjust the normal vector n i p j p k of the triangle Δp T so that the normal vector n i p j p k of the triangle Δp T is in the same direction as the contour line normal vector n i . When n T ·n i > 0, the vertex order is correctly selected. If n T ·n i < 0, then swap the vertex order. It should be noted that the above contour line normal vector n i refers to the triangle Δp i pj p k The normal vector of the contour line passed through.

[0149] As can be seen from the above, the normal vector of the coal flow contour line provides the orientation information and geometric constraints of the local surface direction of the coal flow contour, can construct the neighborhood topological relationship of the points on the contour line, assist the algorithm to distinguish the inner and outer surfaces, and is used to ensure that the coal flow grid generated by Delaunay triangulation conforms to the real surface morphology, is a smooth and closed grid, and prevents the generation of "folded" or "penetrated" triangles (such as the incorrect connection between the middle trough of the scraper conveyor and the coal flow contour line) when using Delaunay triangulation, laying a foundation for the calculation of the coal flow volume.

[0150] The process of calculating the coal flow volume on the conveyor simulation device 1 through the coal flow contour surface grid is introduced below.

[0151] According to the coal flow contour surface grid, calculate the coal flow volume on the conveyor simulation device 1 in the world coordinate system; the coal flow volume on the conveyor simulation device 1 in the world coordinate system satisfies the following formula;

[0152]

[0153] Among them, V represents the coal flow volume on the conveyor simulation device 1, Δ j represents the j-th triangle obtained after Delaunay triangulation of the coal flow contour surface grid in the world coordinate system, S(Δ j ) represents the area of the j-th triangle, Z j1 represents the coordinate value of the first vertex of the j-th triangle on the z-axis in the world coordinate system; Z j2 represents the coordinate value of the second vertex of the j-th triangle on the z-axis in the world coordinate system; Z j3 represents the coordinate value of the third vertex of the j-th triangle on the z-axis in the world coordinate system; V0 represents the volume occupied by the middle trough on the conveyor simulation device 1 under no-load conditions.

[0154] In summary, in the experimental bench for simulating and monitoring the coal flow volume of the scraper conveyor in the fully mechanized coal mining face provided by the embodiments of the present application, the combined guide rail 2 in the coal mining simulation device is used to simulate the process of "lowering the support - pulling the support - raising the support - pushing the scraper conveyor". During the process where the position of the binocular camera 3 on the combined guide rail 2 changes as the position of the combined guide rail 2 changes, the images captured by the binocular camera 3 simplify and simulate the position change of the shearer and the scraper conveyor relative to the binocular camera 3 during the actual coal mining process. At the same time, the coal flow volume on the conveyor simulation device 1 is calculated by collecting images through the binocular camera 3, and then the conveying console 5 adjusts the conveying speed of the conveyor simulation device 1 for coal according to the coal flow volume, realizing the coordinated coal mining and conveying during this process. In the case of realizing the coordinated coal mining and conveying, the process of adjusting the conveying speed of the scraper conveyor according to the coal flow volume can be used as a reference for the scraper conveyor in the actual fully mechanized coal mining face to adjust the conveying speed according to the change of the coal flow rate, thus helping to improve the working efficiency of the fully mechanized coal mining face.

[0155] As Figure 4 shown, the method for simulating and monitoring the coal flow volume of the scraper conveyor in the fully mechanized coal mining face provided by the embodiments of the present application is based on the above experimental bench, and this method includes S101 - S104.

[0156] S101. Aim at covering the middle trough of the scraper conveyor with the field of view of the binocular camera, and calibrate the binocular camera.

[0157] S102. Real - time collect the coal on the scraper conveyor through the binocular camera to obtain multiple coal flow rate images, and extract the point cloud data of each coal flow rate image among the multiple coal flow rate images.

[0158] S103. The guide rail control device controls the lateral sliding of the lateral guide rail slide in the combined guide rail, and controls at least the longitudinal sliding of the longitudinal guide rail slide in the combined guide rail to simulate the action changes of the fully mechanized coal mining equipment and the scraper conveyor during the coal mining process.

[0159] S104. The host computer receives in real - time the point cloud data of each coal flow rate image among the multiple coal flow rate images extracted by the binocular camera, generates a coal flow contour surface mesh according to the point cloud data, and calculates the coal flow volume on the conveyor simulation device through the coal flow contour surface mesh.

[0160] S105. The conveying console controls the speed of the conveyor simulation device according to the coal flow volume on the conveyor simulation device.

[0161] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key points of each embodiment are the differences from other embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. The experimental platform for simulating and monitoring the coal flow volume of the scraper conveyor of the fully mechanized mining working face is characterized by: The fully mechanized mining face includes a coal mining machine, a scraper conveyor for conveying coal, and a hydraulic support, and the experimental platform includes: A conveyor simulation device (1) is used to simulate the operation of a scraper conveyor, wherein the conveyor simulation device (1) is provided with a middle trough for carrying coal; A coal mining simulation device, used for simulating the motion state of the hydraulic support relative to the scraper conveyor during the coal mining process; the coal mining simulation device comprises at least one combined guide rail (2); each combined guide rail (2) in the at least one combined guide rail (2) comprises a transverse guide rail slide (21), and a longitudinal guide rail slide (22) slidably connected to the transverse guide rail slide (21), the transverse guide rail slide (21) is used for simulating the pushing and sliding action and the pulling action of the hydraulic support, and the longitudinal guide rail slide (22) is used for simulating the lowering action and the raising action of the hydraulic support; At least one binocular camera (3) is used to perform real-time image acquisition of the coal flow on the conveyor simulation device (1), obtain multiple coal flow images, and extract point cloud data of each of the multiple coal flow images; each binocular camera (3) in the at least one binocular camera (3) is correspondingly arranged on the longitudinal guide rail slide (22) of the combined guide rail (2); each binocular camera (3) includes a left camera and a right camera; A host computer (4) is used to receive in real time the point cloud data of each coal flow image among the multiple coal flow images extracted by the at least one binocular camera (3), generate a coal flow contour surface mesh according to the point cloud data, and calculate the coal flow volume on the conveyor simulation device (1) through the coal flow contour surface mesh; the host computer (4) is electrically connected to the at least one binocular camera (3); A conveying control panel (5) is used to control the speed of the conveyor simulation device (1) according to the volume of the coal flow on the conveyor simulation device (1); the conveying control panel (5) is electrically connected to the host computer (4) and the conveyor simulation device (1); A guide rail control device (6) is used to control at least one combined guide rail (2) in the simulation device to simulate the movement changes of the fully mechanized mining equipment and the scraper conveyor during the coal mining process; the guide rail control device (6) is electrically connected to the coal mining simulation device.

2. The test bench for simulating and monitoring the coal flow volume of the scraper conveyor of the fully mechanized mining working face according to claim 1 is characterized in that: The step of extracting point cloud data of each coal flow image from the plurality of coal flow images comprises: For each binocular camera (3) in at least one binocular camera (3), parallax information between a left camera and a right camera in the binocular camera (3) is obtained, and a depth value corresponding to each pixel point in each of the multiple coal flow images is calculated based on the parallax information; The point cloud data of each coal flow image in the plurality of coal flow images is calculated using the depth value, the internal parameters of the left camera, and the internal parameters of the right camera.

3. The test bench for simulating and monitoring the coal flow volume of the scraper conveyor of the fully mechanized mining working face as claimed in claim 2 is characterized in that: The step of generating a coal flow contour surface mesh according to the point cloud data comprises: Using image stitching technology, the point cloud data of each coal flow image in the plurality of coal flow images after denoising and downsampling are fused; Identifying the boundary and shape features of the fused point cloud data, and extracting the coal flow contour line of the fused point cloud data; Calculate the normal vector of the coal flow contour line, and construct the coal flow contour surface mesh according to the normal vector of the contour line; wherein the normal vector of the coal flow contour line satisfies the following formula: The 3×3 ·v j =λ j ·vj,j∈{1,2,3} Among them, E 3×3 represents the point neighborhood covariance matrix of any point p in the coal flow contour line; j Indicates E 3×3 The jth eigenvalue in j Indicates E 3×3 The eigenvector corresponding to the jth eigenvalue in E; 3×3 The eigenvector corresponding to the smallest eigenvalue in is the normal vector of point p.

4. The test bench for simulating and monitoring the coal flow volume of the scraper conveyor of the fully mechanized mining working face as claimed in claim 3 is characterized in that: The coal flow volume on the conveyor simulation device (1) is obtained by calculating the coal flow contour surface grid, comprising: The volume of the coal flow on the conveyor simulation device (1) in the world coordinate system is calculated according to the coal flow contour surface grid; the volume of the coal flow on the conveyor simulation device (1) in the world coordinate system satisfies the following formula; Wherein, V represents the volume of coal flow on the conveyor simulation device (1), Δ j represents the jth triangle obtained by Delaunay triangulation of the coal flow contour surface mesh in the world coordinate system, S(Δ j ) represents the area of ​​the j-th triangle, Z j1 represents the coordinate value of the first vertex of the j-th triangle on the z-axis of the world coordinate system; Z j2 represents the coordinate value of the second vertex of the j-th triangle on the z-axis of the world coordinate system; Z j3 represents the coordinate value of the third vertex of the jth triangle on the z-axis of the world coordinate system; V0 represents the volume occupied by the middle slot on the conveyor simulation device (1) in the empty state.

5. The test bench for simulating and monitoring the coal flow volume of the scraper conveyor of the fully mechanized mining working face according to claim 1, characterized in that: The transverse guide rail slide (21) is C-shaped, one end of the transverse guide rail slide (21) is fixed to one side of the scraper conveyor, and a transverse first ball screw (211) is arranged at both ends of the transverse guide rail slide (21), one end of the first ball screw (211) is rotatably connected to one end of the transverse guide rail slide (21), and the other end of the first ball screw (211) is connected to the output end of the first motor (212), and the first motor (212) is fixed to the other end of the transverse guide rail slide (21); A connecting piece is fixed on the first ball screw (211), and the upper part of the connecting piece is fixedly connected to the longitudinal guide rail slide (22), and the longitudinal guide rail slide (22) is C-shaped. A longitudinal second ball screw (221) is arranged at both ends of the longitudinal guide rail slide (22), and one end of the second ball screw (221) is rotatably connected to one end of the longitudinal guide rail slide (22), and the other end of the second ball screw (221) is connected to the output end of the second motor (222), and the second motor (222) is fixed to the other end of the transverse guide rail slide (21); A rotating steering gear (223) is fixed on the second ball screw (221), the binocular camera (3) is arranged on the rotating steering gear (223), and the binocular camera (3) rotates along with the rotating steering gear (223).

6. A method for simulating and monitoring the volume of coal flow in a fully mechanized mining face scraper conveyor, based on the test bench according to any one of claims 1 to 5, characterized in that: include: Calibrate the at least one binocular camera (3) with the goal of the binocular camera (3) field of view covering the middle groove of the scraper conveyor; The guide rail control device (6) controls the transverse guide rail slide (21) in the at least one combined guide rail (2) to slide transversely, and controls the longitudinal guide rail slide (22) in the at least one combined guide rail (2) to slide longitudinally, so as to simulate the movement changes of the fully mechanized mining equipment and the scraper conveyor during the coal mining process; The at least one binocular camera (3) is used to collect coal on the scraper conveyor in real time to obtain a plurality of coal flow images, and point cloud data of each of the plurality of coal flow images is extracted; The host computer (4) receives point cloud data of each coal flow image in the plurality of coal flow images extracted by the at least one binocular camera (3) in real time, generates a coal flow contour surface mesh according to the point cloud data, and calculates the coal flow volume on the conveyor simulation device (1) through the coal flow contour surface mesh; The conveying control console (5) controls the speed of the conveyor simulator (1) according to the volume of coal flow on the conveyor simulator (1).

Images