Method for detecting siltation degree of underground coal mine sump based on image gradient information

Abstract

The invention provides an underground coal mine sump siltation degree detection method based on image gradient information, and the method comprises the steps: arranging a plurality of position labels at the top of a sump, and correspondingly arranging a plurality of calibration discs at the bottom of the sump; acquiring image information in the water sump; an image coordinate system is established, and the water sump bottom area in the image is divided into a plurality of recognition areas according to the position information of the position label and the calibration disc; constructing a deposition degree classification and identification neural network; and sending each identification area in the real-time water sump image into the deposition degree classification and identification neural network for identification and outputting the deposition degree of each identification area. A camera, a position label and a calibration disc are arranged in a water sump, an image coordinate system is established after an image in the water sump is obtained, a deposition area at the bottom of the water sump is divided into a plurality of identification areas through coordinate information of a reference object, and after the deposition degree of each identification area is identified through a deposition degree classification identification neural network, the deposition degree of the water sump is identified. And judging the overall siltation degree by looking up a table.

Description

Technical Field

[0001] The invention relates to the field of underground water tank excavation in mines, and in particular to a method for detecting the siltation degree of underground water tanks in coal mines based on image gradient information. Background Art

[0002] The mine water produced during the coal mine production process contains a large amount of impurities, mainly coal slime and sand, which settle at the bottom of the water tank, reducing the effective volume of the water tank. If the siltation is not cleared in time, the water tank will not only fail to play a buffering and storage role, but may also cause a flooding accident; the impurities in the gushing water will also enter the suction well of the main pump room, causing the wear of the flow parts of the water pump to increase and accelerate the damage of the water pump; when there is too much silt in the suction well, it will also block the suction tap, causing drainage difficulties or even impossible drainage, which seriously threatens the safety of the mine. Therefore, timely siltation of mine water tanks is one of the effective ways to ensure mine drainage safety, extend the life of water pumps, and improve the efficiency of drainage systems.

[0003] At present, there are few methods for automatically detecting the siltation degree of water tanks. In engineering projects, it is determined by regular manual inspections on site, which is time-consuming and labor-intensive.

[0004] The patent of this invention proposes a water tank siltation degree detection method based on image gradient information, with a machine vision system as the core, to establish a water tank siltation degree automatic identification neural network to realize automatic identification of water tank siltation degree. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention proposes a method for detecting the siltation degree of underground water tanks in coal mines based on image gradient information.

[0006] A method for detecting siltation of underground water tanks in coal mines based on image gradient information comprises the following steps:

[0007] S100, setting a plurality of position signs on the top of the water tank, and correspondingly setting a plurality of calibration plates on the bottom of the water tank;

[0008] S200, obtaining image information in the water tank;

[0009] S300, establishing an image coordinate system, and dividing the bottom area of ​​the water tank in the image into multiple identification areas according to the position information of the position sign and the calibration disk;

[0010] S400, constructing a neural network for classification and recognition of siltation degree;

[0011] S500, sending the information of each identified area in the acquired real-time water tank image to the siltation degree classification recognition neural network to identify and output the siltation degree of each identified area.

[0012] Based on the above, including step S600, an overall siltation degree discrimination strategy table is constructed, and corresponding overall siltation degree information is configured corresponding to each group of siltation degree information combinations of the identified areas.

[0013] Based on the above, in step S100, a corner of the location sign is selected as a reference point, and the reference points of all the location signs are set on the same straight line; identification characters are also set on each location sign.

[0014] Based on the above, in step S100, the calibration disks are arranged in the same horizontal plane, and a reference point of a position sign corresponding to a corner of each calibration disk is arranged vertically below the position sign.

[0015] Based on the above, in step S200, a camera with a selected resolution is set in the water tank, and image information in the water tank is obtained through the camera.

[0016] Based on the above, in step S300, an image coordinate system is established, and after confirming the coordinate information of each location sign reference point, the coordinate information of the corresponding calibration plate corner point and the coordinate information of the two side lines of the bottom of the water tank, the coordinate area range of each identification area is obtained and divided.

[0017] Based on the above, step S400 includes:

[0018] S410, extracting gradient information of the acquired water tank image;

[0019] S420, splicing the original water tank image and its gradient information into new feature information, the dimension of which is Ih×Id×4, where Ih represents the height of the image, Id represents the width of the image, and 4 represents the number of feature information channels;

[0020] S430, resetting the width and height of the new feature information image to an image of 224×224 to form feature information of 224×224×4;

[0021] S440, classify and output through ResNet50 backbone network;

[0022] S450, through the Linear fully connected layer, outputs the classification of the siltation degree of the water tank.

[0023] Based on the above, there are three classifications for the degree of siltation, where Class 1 indicates no siltation, Class 2 indicates a small amount of siltation, and Class 3 indicates a large amount of siltation.

[0024] Based on the above, in step S600, for each group of siltation degree information combination of the identified areas in the real-time water tank image, the overall siltation degree information of the water tank is obtained by looking up a table.

[0025] The present invention has outstanding substantial features and remarkable progress compared with the prior art. Specifically, the present invention sets a camera in the water tank and respectively sets a position sign and a calibration disk as fixed reference objects, establishes an image coordinate system after acquiring the image in the water tank, and divides the siltation area at the bottom of the water tank into multiple identification areas according to the coordinate information of the reference object. After the siltation degree of each identification area is identified by the siltation degree classification and identification neural network, the overall siltation degree is determined by table lookup. The present invention has the advantages of low cost, high automation, high efficiency and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of an underground water tank environment and equipment installation of the present invention.

[0027] Figure 2 It is a schematic diagram of the corresponding relationship between the position label and the calibration disk of the present invention.

[0028] Figure 3 It is a schematic diagram of the image coordinate system and the recognition area of ​​the present invention.

[0029] Figure 4 It is a schematic diagram of the neural network structure for classification and identification of siltation degree of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] A method for detecting the siltation degree of underground water tanks in coal mines based on image gradient information comprises the following steps: S100, setting a plurality of position signs on the top of the water tank, and correspondingly setting a plurality of calibration disks on the bottom of the water tank; S200, acquiring image information in the water tank; S300, establishing an image coordinate system, and dividing the bottom area of ​​the water tank in the image into a plurality of identification areas according to the position information of the position signs and the calibration disks; S400, constructing a siltation degree classification and recognition neural network; S500, sending each identification area information in the acquired real-time water tank image to the siltation degree classification and recognition neural network for identification and outputting the siltation degree of each identification area; S600, constructing an overall siltation degree discrimination strategy table, respectively corresponding to each group of siltation degree information combinations of identification areas, configuring corresponding overall siltation degree information, and acquiring the overall siltation degree information of the water tank by looking up the table for each group of identification area siltation degree information combinations in the real-time water tank image.

[0032] Specifically, take a water tank in a mine as an example. Figure 1As shown in the figure, it is a schematic diagram of the environment and equipment installation of a water tank in a mine. A camera and a light source are set on the top of the water tank opening and face the direction of the water tank respectively. The camera is the core equipment for environmental perception. A camera that meets the safety standards of underground coal mines is selected. In this embodiment, the resolution is 1080P, that is, the pixels of a single frame image are 1920×1080. The light source is a device that provides lighting for the water tank. In this embodiment, a light with a focusing unidirectional irradiation function that meets the safety standards of underground coal mines is selected, and the power is about 150W. In other embodiments, the power or quantity of the light source can be increased or decreased according to actual conditions.

[0033] The position sign is used to reference the position information. The position sign is installed on the top of the water tank. There are at least two position signs, and the one farthest from the camera must be installed on the rear retaining wall of the water tank. Increasing the number of position signs can increase the accuracy of position recognition. In this embodiment, four are taken as an example; in other embodiments, a reasonable number can be set according to actual conditions. In addition, during the installation of the position signs, the following two requirements must be met: 1) A certain fixed position of all signboards, such as the corner point of an angle, must be on the same line; 2) The rear signboard cannot be blocked by the front signboard in the imaging. The calibration disk is composed of a two-color checkerboard and is used to calibrate the vertical position of the position sign. The calibration disk is located in the same horizontal plane and each calibration disk corresponds to a position sign and is vertically arranged below the position sign. A reference angle of the calibration disk is on a vertical line with the reference angle of the corresponding position sign, such as Figure 2 shown.

[0034] The camera obtains the image of the siltation in the water tank, such as Figure 3 As shown in the figure, assume that the image width is D pixels and the height is H pixels, and establish the coordinate systems x and y along the width and height respectively. A1, A2, A3, and A4 represent the image points of the reference corner points of the position signs No. 1 to No. 4 in the image; B1, B2, and B3 represent the image points of the reference corner points corresponding to the calibration disk in the image. Since the water tank, location sign, calibration plate and camera are all fixed, after acquiring an image and establishing the image coordinate system, the image coordinate system will be applicable to each image. By manually analyzing and measuring pixels, the coordinate information of A1-A4 and B1-B3 and the linear function relationship of the two sides of the bottom of the water tank in the image coordinate system can be obtained. Then, the coordinate point information of C1-C3 is obtained according to the coordinate information of B1-B3. Then, the coordinate range corresponding to the recognition area in the figure is divided according to the coordinate points B1-B3 and C1-C3, that is, the area enclosed by B1B2C2C1 (S1 area), the area enclosed by C2B2B3C3 (S2 area) and the area enclosed by C3B3 and the endpoints of the two sides ( Figure 3 The area (S3 area) enclosed by the image coordinate system (not shown) is divided into three recognition areas. Since the recognition area is also fixed, it can be applied to other subsequent monitoring images after a division and calibration. After the image coordinate system and the recognition area are calibrated, the calibration disk can also be removed.

[0035] Construct a neural network for classification and recognition of siltation degree, and select ResNet50 as the main body of the classification network. The structure is as follows Figure 4 As shown, 1) first use the gradient operator module to extract the gradient information of the image; 2) in the splicing module, the original image and the gradient information are spliced ​​into new feature information, and the dimension of the information is Ih×Id×4, where Ih represents the height of the image, Id represents the width of the image, and 4 represents the number of feature information channels, which are GRB three channels and gradient channels respectively; 3) the Resize module resets the width and height of the image to a 224X224 image to form 224×224×4 feature information; 4) ResNet50 is the backbone network, where the number of channels of the input layer is set to 4, and the output of the final output layer is 1000 categories; the rest of the settings remain the original settings; 5) Linear is a fully connected layer, whose input is 1000 and output is 3, forming a discriminant classification of the siltation degree of the underground water tank; in this embodiment, the three categories of category 1 represent no siltation, category 2 represent a small amount of siltation, and category 3 represent a large amount of siltation. Collect a large number of data samples and classify them into three categories according to workers' experience: no siltation, small amount of siltation, and large amount of siltation. Store them in three folders for use in training neural networks. Keep the number of samples in each category balanced, and the number of samples in a single category is at least 100. When training neural networks, the cross entropy loss function is used as the loss function, the Adam optimizer is used, the learning rate is 0.01, the location of the data sample folder is input, and training and classification are performed according to the number of folders.

[0036] After the acquired real-time water tank image is divided into regions, the image of each identified region is sent to the trained siltation degree classification and recognition neural network for recognition, and the siltation degree information of each identified region is obtained respectively.

[0037] Preferably, since there is a certain error in the determination of the siltation degree of each sub-graph, an overall siltation degree determination strategy table as shown in Table 1 is established:

[0038]

[0039]

[0040] Table 1

[0041] After obtaining the sedimentation degree information of each identified area in the image, the overall sedimentation degree information can be obtained by looking up the table.

[0042] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for detecting siltation of underground water tanks in coal mines based on image gradient information, characterized in that: Includes steps: S100, setting a plurality of position signs on the top of the water tank, and correspondingly setting a plurality of calibration plates on the bottom of the water tank; S200, obtaining image information in the water tank; S300, establishing an image coordinate system, and dividing the bottom area of ​​the water tank in the image into multiple identification areas according to the position information of the position sign and the calibration disk; S400, constructing a neural network for classification and recognition of siltation degree; S500, sending the information of each identified area in the acquired real-time water tank image to the siltation degree classification recognition neural network to identify and output the siltation degree of each identified area.

2. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 1 is characterized in that: The method comprises step S600 of constructing an overall siltation degree discrimination strategy table, respectively corresponding to each group of siltation degree information combinations of the identified areas, and configuring corresponding overall siltation degree information.

3. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 1 is characterized in that: In step S100, a corner of a location sign is selected as a reference point, and the reference points of all location signs are set on the same straight line; identification characters are also set on each location sign.

4. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 3 is characterized in that: In step S100, the calibration disks are arranged in the same horizontal plane, and a reference point of a position sign corresponding to a corner of each calibration disk is arranged vertically below the position sign.

5. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 4 is characterized in that: In step S200, a camera with a selected resolution is set in the water tank, and image information in the water tank is obtained through the camera.

6. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 1, characterized in that: In step S300, an image coordinate system is established, and after confirming the coordinate information of each position sign reference point, the coordinate information of the corresponding calibration plate corner point and the coordinate information of the two sides of the water tank bottom, the coordinate area range of each identification area is obtained and divided.

7. The method for detecting siltation degree of underground coal mine water tanks based on image gradient information according to claim 1, characterized in that: Step S400 includes: S410, extracting gradient information of the acquired water tank image; S420, splicing the original water tank image and its gradient information into new feature information, the dimension of which is Ih×Id×4, where Ih represents the height of the image, Id represents the width of the image, and 4 represents the number of feature information channels; S430, resetting the width and height of the new feature information image to an image of 224×224 to form feature information of 224×224×4; S440, classify and output through ResNet50 backbone network; S450, through the Linear fully connected layer, outputs the discrimination classification of the water tank siltation degree.

8. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 7 is characterized in that: There are three classifications for the degree of siltation, where Class 1 indicates no siltation, Class 2 indicates a small amount of siltation, and Class 3 indicates a large amount of siltation.

9. The method for detecting siltation degree of underground water tank in coal mine based on image gradient information according to claim 2, characterized in that: In step S600, for each group of siltation degree information combination of the identified areas in the real-time water tank image, the overall siltation degree information of the water tank is obtained by looking up a table.

Images