A method for autonomous inspection of drones in underground coal mines

Abstract

The present invention discloses a method for autonomous inspection of drones in coal mines, which relates to the technical field of image analysis. It includes: S1, importing the internal layout map of the coal mine to be inspected into the intelligent inspection terminal of the drone; S2, the intelligent inspection terminal performs real-time image acquisition of the interior of the coal mine according to the internal layout map of the coal mine to be inspected. First, the present invention analyzes the features of the internal color image and the internal depth image of the coal mine to determine the position information of the obstacles, the actual shape information of the obstacles, and the actual size information of the obstacles. Then, it performs a risk analysis on the position information of the obstacles, the actual shape information of the obstacles, and the actual size information of the obstacles to determine the obstacle risk level. Finally, it performs a spacing analysis on the obstacles and the surrounding well walls of the coal mine to determine whether the drone can cross the obstacles for subsequent inspections, shortening the inspection cycle of the coal mine. At the same time, it reduces the safety hazards of the inspection personnel to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and specifically relates to a method for autonomous inspection of drones in coal mines Background Art

[0002] A coal mine shaft refers to an underground or open-pit mine used for coal resource extraction. It is the main place for coal production and usually includes structures such as shafts, roadways, and working faces

[0003] When the length of the coal mine shaft is long or the depth of the coal mine shaft is deep, the manual inspection period is too long, increasing the working intensity of workers. In addition, with the coal mining, certain dangers may occur inside the coal mine shaft, and workers may be injured during the inspection Summary of the Invention

[0004] To solve the above technical problems, a method for autonomous inspection of drones in coal mines is provided. This technical solution solves the problems raised in the above background art that when the length of the coal mine shaft is long or the depth of the coal mine shaft is deep, the manual inspection period is too long, increasing the working intensity of workers. In addition, with the coal mining, certain dangers may occur inside the coal mine shaft, and workers may be injured during the inspection

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows

[0006] A method for autonomous inspection of drones in coal mines, comprising

[0007] S1. Import the internal layout map of the coal mine shaft to be inspected into the intelligent inspection terminal of the drone, where the internal layout map of the coal mine shaft to be inspected is obtained from the database system

[0008] It can be understood that the drone conducts inspections based on the internal layout images of the coal mine shaft. Because the layout in the coal mine shaft is formed according to the coal distribution, there may be multiple mine tunnels

[0009] S2. The intelligent inspection terminal performs real-time image acquisition of the inside of the coal mine shaft according to the internal layout map of the coal mine shaft to be inspected, and obtains the internal images of the coal mine shaft, where the internal images of the coal mine shaft include the internal color images and the internal depth images of the coal mine shaft

[0010] It can be understood that the depth image represents the actual distance between the shooting device and the shooting object. Therefore, the position of the obstacle can be determined by shooting the depth image

[0011] The internal depth image represents the distance between the depth camera and the obstacles inside the coal mine shaft, that is, each pixel value in the depth image represents the distance between a certain position inside the coal mine shaft and the camera

[0012] S3. Based on the intelligent inspection terminal, perform feature analysis and processing on the internal images of the coal mine shaft to determine the relevant data of the internal obstacles in the coal mine shaft. Among them, the relevant data of the internal obstacles in the coal mine shaft include obstacle position information, obstacle shape information, and obstacle size information;

[0013] S4. Based on the laser scanner, verify and process the obstacle shape information and obstacle size information with the obstacle position information as the feature to obtain the actual obstacle shape information and the actual obstacle size information;

[0014] S5. Based on the intelligent inspection terminal, perform risk assessment and analysis on the actual obstacle shape information and the actual obstacle size information to determine the obstacle risk level;

[0015] S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle risk level and the obstacle position information, and sends the warning information to the electronic devices of each worker;

[0016] It can be understood that when it is determined that the obstacle has a certain risk, in order to avoid scratching the workers, the position of the obstacle is sent to the electronic devices of the workers to improve the vigilance of the workers;

[0017] S7. Based on the intelligent inspection terminal, perform feature analysis on the obstacle position information and the actual obstacle shape information to determine whether the drone can pass.

[0018] Preferably, the real-time image acquisition of the interior of the coal mine shaft in step S2 includes the following steps:

[0019] S21. The intelligent inspection terminal constructs a three-dimensional space rectangular coordinate system according to the internal trend map of the coal mine shaft to be inspected, sets the X-axis in the width direction of the coal mine shaft, sets the Y-axis in the internal trend of the coal mine shaft, and sets the Z-axis in the height direction of the coal mine shaft;

[0020] S22. The intelligent inspection terminal controls the depth camera to take a color image of the interior of the coal mine shaft to obtain the internal color image of the coal mine shaft, and records the shooting coordinates of the depth camera;

[0021] It can be understood that in order to determine the obstacle position in the depth image through the obstacle position coordinates in the color image, it is necessary to take pictures at the same shooting coordinates;

[0022] S23. The intelligent inspection terminal controls the depth camera to take a depth image of the interior of the coal mine shaft according to the shooting coordinates of the depth camera to obtain the internal depth image of the coal mine shaft.

[0023] Preferably, the feature analysis and processing of the internal image of the coal mine shaft in step S3 specifically includes the following steps:

[0024] S31. The intelligent patrol terminal respectively constructs a two-dimensional rectangular coordinate system for the internal color image and the internal depth image of the coal mine shaft. Taking the horizontal direction of the internal color image and the internal depth image of the coal mine shaft as the X-axis and the vertical direction of the internal color image and the internal depth image of the coal mine shaft as the Y-axis, the scale lengths represented by the coordinate axes of the two-dimensional rectangular coordinate system of the internal color image of the coal mine shaft and the two-dimensional rectangular coordinate system of the internal depth image of the coal mine shaft are the same;

[0025] It can be understood that constructing the same rectangular coordinate system for the color image and the depth image is to be able to determine the position of the obstacle in the depth image through the position of the obstacle in the color image in the subsequent process;

[0026] S32. Based on the blob detection algorithm, perform obstacle feature extraction processing on the internal color image of the coal mine shaft to determine the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information;

[0027] S33. The intelligent patrol terminal performs distance analysis on the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information to determine the obstacle position information;

[0028] S34. The intelligent patrol terminal performs measurement processing on the obstacle shape information in the internal color image of the coal mine shaft to obtain the reduced size information of the obstacle;

[0029] S35. The intelligent patrol terminal performs reduction calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information.

[0030] Preferably, the distance analysis of the internal depth image of the coal mine shaft in step S33 specifically includes the following steps:

[0031] S331. The intelligent patrol terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional rectangular coordinate system of the depth image;

[0032] S332. The intelligent patrol terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each position of the obstacle in the two-dimensional rectangular coordinate system of the depth image to determine the pixel value of each position of the obstacle in the depth image;

[0033] S333. The intelligent patrol terminal performs conversion processing on the pixel value of each position of the obstacle in the depth image according to the shooting parameters and coding method of the depth camera to determine the obstacle position information.

[0034] Preferably, the verification process of the obstacle shape information and the obstacle size information in step S4 specifically includes the following steps:

[0035] S41. The intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the standard obstacle shape information and the standard obstacle size information;

[0036] S42. The intelligent inspection terminal respectively performs comparison processing on the obstacle shape information and the obstacle size information with the standard obstacle shape information and the standard obstacle size information;

[0037] S43. If the standard obstacle shape information is consistent with the obstacle shape information and the standard obstacle size information is consistent with the obstacle size information, set the obstacle shape information as the actual obstacle shape information and set the obstacle size information as the actual obstacle size information;

[0038] S44. If the standard obstacle shape information is inconsistent with the obstacle shape information and the standard obstacle size information is inconsistent with the obstacle size information, set the standard obstacle shape information as the actual obstacle shape information, set the standard obstacle size information as the actual obstacle size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera.

[0039] Preferably, the risk assessment and analysis of the actual obstacle shape information and the actual obstacle size information in step S5 specifically includes the following steps:

[0040] S51. Based on the intelligent inspection terminal, send a data reading request to the external storage device to obtain the obstacle risk reference table. Among them, the obstacle risk reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence;

[0041] S52. The intelligent inspection terminal performs shape matching processing on the obstacle shape reference table with the actual obstacle shape information as a feature to determine the obstacle type and the obstacle shape risk level;

[0042] S53. The intelligent inspection terminal performs information matching processing on the obstacle size reference table with the obstacle type as a feature to determine the reference obstacle size corresponding to the obstacle type;

[0043] S54. The intelligent inspection terminal performs size matching processing on the reference obstacle size corresponding to the obstacle type with the actual obstacle size information as a feature to determine the obstacle size risk level;

[0044] S55. The intelligent inspection terminal performs risk weight analysis on the obstacle shape risk level and the obstacle size risk level to determine the obstacle risk level.

[0045] Preferably, the specific calculation formula for the hazard weight analysis is as follows:

[0046]

[0047] In the formula, the is the hazard level of the obstacle; is the hazard level of the obstacle shape; is the weight ratio of the hazard level of the obstacle shape; is the hazard level of the obstacle size; is the weight ratio of the hazard level of the obstacle size.

[0048] Preferably, the specific steps for feature analysis of the obstacle position information and the actual shape information of the obstacle in step S7 are as follows:

[0049] S71. Based on the intelligent inspection terminal, perform feature analysis on the actual shape information of the obstacle to determine the edge information of the obstacle. The edge information of the obstacle is specifically any three or any two or any one combination of the left edge position information of the obstacle, the right edge position information of the obstacle, the top edge position information of the obstacle, and the bottom edge position information of the obstacle;

[0050] It can be understood that the obstacle may appear in different positions, may appear at the top of the coal mine shaft, may also appear at the bottom of the coal mine shaft, or may completely block the top and bottom of the coal mine shaft, leaving only the left and right sides;

[0051] S72. The intelligent inspection terminal controls the laser scanner to collect and process information around the obstacle position information, and obtains the edge coordinates of the coal mine shaft at the obstacle position information. The edge coordinates of the coal mine shaft at the obstacle position information are three-dimensional coordinates;

[0052] S73. The intelligent inspection terminal performs comparison processing on the edge coordinates of the coal mine shaft at the obstacle position information to determine the minimum edge coordinates of the coal mine shaft at the obstacle position information;

[0053] S74. Based on the intelligent inspection terminal, perform calculation and analysis processing on the minimum edge coordinates of the coal mine shaft at the obstacle position information and the edge information of the obstacle to determine whether the UAV can pass.

[0054] Preferably, the specific steps for calculation and analysis processing of the minimum edge coordinates of the coal mine shaft at the obstacle position information and the edge information of the obstacle in step S74 are as follows:

[0055] S741. Based on the intelligent inspection terminal, perform a difference calculation on the minimum edge coordinates of the coal mine shaft at the obstacle position information and the edge information of the obstacle to obtain the passing spacing data;

[0056] S742. Based on the intelligent inspection terminal, perform dimensional analysis on the drone to obtain the height data and length data of the drone;

[0057] S743. Based on the intelligent inspection terminal, perform judgment processing on the passing spacing data, the height data of the drone, and the length data of the drone to determine whether the drone can pass.

[0058] Preferably, the judgment processing of the passing spacing data, the height data of the drone, and the length data of the drone in step S743 specifically includes the following steps:

[0059] S7431. If the passing spacing data is the spacing data between the top or bottom of the obstacle and the edge of the coal mine shaft;

[0060] S7432. Based on the intelligent inspection terminal, perform judgment processing on the height data of the drone and the passing spacing data;

[0061] S7433. If the height data of the drone is greater than or equal to the passing spacing data, the drone cannot pass;

[0062] S7434. If the height data of the drone is less than the passing spacing data, the drone can pass and can perform subsequent inspection tasks;

[0063] S7435. If the passing spacing data is the spacing data between the left or right side of the obstacle and the edge of the coal mine shaft;

[0064] S7436. Based on the intelligent inspection terminal, perform judgment processing on the length data of the drone and the passing spacing data;

[0065] S7437. If the length data of the drone is greater than or equal to the passing spacing data, the drone cannot pass;

[0066] S7438. If the length data of the drone is less than the passing spacing data, the drone can pass and can perform subsequent inspection tasks.

[0067] Furthermore, a drone autonomous inspection system for coal mine shafts is proposed, which is used to implement the method of drone autonomous inspection in coal mine shafts as described above, including:

[0068] An intelligent inspection terminal, which performs image acquisition, image analysis, and obstacle feature calculation according to the internal layout map of the coal mine shaft to be inspected, determines the obstacle danger level, and determines whether the drone can pass;

[0069] A database system, which is used to store the internal layout map of the coal mine shaft to be inspected;

[0070] Among them, the intelligent inspection terminal is internally integrated with:

[0071] A central control module, which is used to control data transmission and information interaction between each module;

[0072] A depth camera, which is used to collect color images and depth images inside the coal mine shaft;

[0073] An image analysis module, which is used to analyze the position of obstacles, calculate the size of obstacles, and analyze the shape of obstacles in the internal color image and the internal depth image of the coal mine shaft;

[0074] A feature verification module, which verifies the shape information and size information of obstacles by controlling a laser scanner to laser scan the obstacle position information and through the scanned data;

[0075] A danger level assessment module, which is used to analyze the danger weights of the obstacle shape danger level and the obstacle size danger level to determine the obstacle danger level;

[0076] A warning information generation module, which generates corresponding warning information according to the obstacle danger level and the obstacle position information to ensure the construction safety of workers;

[0077] A spacing judgment module, which is used to judge and process the passing spacing data, the height data of the unmanned aerial vehicle, and the length data of the unmanned aerial vehicle to determine whether the unmanned aerial vehicle can pass through to complete the subsequent inspection task.

[0078] Furthermore, a storage medium is proposed, on which a computer program is stored, and when the computer program is called and run, it executes a method for autonomous inspection of an unmanned aerial vehicle in a coal mine shaft as described above.

[0079] Compared with the prior art, the present invention provides a method for autonomous inspection of an unmanned aerial vehicle in a coal mine shaft, which has the following beneficial effects:

[0080] The present invention first analyzes the features of the internal color image and the internal depth image of the coal mine shaft to determine the obstacle position information, the actual shape information of the obstacle, and the actual size information of the obstacle. Then, it analyzes the danger degree of the obstacle position information, the actual shape information of the obstacle, and the actual size information of the obstacle to determine the obstacle danger level. Finally, it analyzes the spacing between the obstacle and the surrounding shaft walls of the coal mine shaft to determine whether the unmanned aerial vehicle can cross the obstacle to perform subsequent inspections, shortening the inspection cycle of the coal mine shaft and, at the same time, reducing the safety hazards of the inspection personnel to a certain extent. Description of the Drawings

[0081] Figure 1 It is a schematic flow diagram of steps S1 - S7 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0082] Figure 2 It is a schematic flow diagram of steps S21 - S23 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0083] Figure 3 It is a schematic flow diagram of steps S31 - S35 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0084] Figure 4 It is a schematic flow diagram of steps S331 - S333 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0085] Figure 5 It is a schematic flow diagram of steps S41 - S44 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0086] Figure 6 It is a schematic flow diagram of steps S51 - S55 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0087] Figure 7 It is a schematic flow diagram of steps S71 - S74 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0088] Figure 8 It is a schematic flow diagram of steps S741 - S743 in a method for autonomous inspection of underground coal mine drones proposed by the present invention;

[0089] Figure 9 It is a schematic flow diagram of steps S7431 - S7438 in a method for autonomous inspection of underground coal mine drones proposed by the present invention. Detailed implementation manner

[0090] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations.

[0091] Refer to Figure 1 As shown, a method for autonomous inspection of underground coal mine drones includes:

[0092] S1. Import the internal layout map of the coal mine to be inspected into the intelligent inspection terminal of the drone, where the internal layout map of the coal mine to be inspected is obtained from the database system;

[0093] S2. The intelligent inspection terminal performs real-time image acquisition on the interior of the coal mine shaft according to the internal layout map of the coal mine shaft to be inspected, and obtains the internal image of the coal mine shaft. Among them, the internal image of the coal mine shaft includes the internal color image and the internal depth image of the coal mine shaft;

[0094] S3. Based on the intelligent inspection terminal, perform feature analysis and processing on the internal image of the coal mine shaft to determine the relevant data of the internal obstacles in the coal mine shaft. Among them, the relevant data of the internal obstacles in the coal mine shaft include obstacle position information, obstacle shape information, and obstacle size information;

[0095] S4. Based on the laser scanner, verify and process the obstacle shape information and obstacle size information with the obstacle position information as the feature to obtain the actual obstacle shape information and the actual obstacle size information;

[0096] S5. Based on the intelligent inspection terminal, perform a risk assessment analysis on the actual obstacle shape information and the actual obstacle size information to determine the obstacle risk level;

[0097] S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle risk level and the obstacle position information, and sends the warning information to the electronic devices of each worker;

[0098] S7. Based on the intelligent inspection terminal, perform feature analysis on the obstacle position information and the actual obstacle shape information to determine whether the drone can pass;

[0099] Those skilled in the art can understand that with the excessive mining of coal, the depth of the coal mine shaft will become too deep and the lateral extension of the coal mine shaft will also become very long. If manual inspection is carried out on the too deep and too long coal mine shaft, a large amount of time will be wasted, and at the same time, the coal mining progress will also be delayed. Therefore, the interior of the coal mine shaft is imaged by a drone, and then the collected images are analyzed for features to determine the obstacle position information, the actual obstacle shape information, and the actual obstacle size information in the coal mine shaft, and the above information is evaluated for risk to determine the obstacle risk level. Finally, warning information is generated according to the obstacle risk level and the obstacle position information. Workers avoid obstacles according to the warning information, or maintenance personnel remove obstacles according to the warning information, thereby avoiding harm to workers caused by obstacles. In addition, the distance between the obstacles and the surrounding shaft walls of the coal mine shaft is also analyzed to determine whether the drone can cross the obstacles to complete subsequent inspections.

[0100] Refer to Figure 2 As shown, the real-time image acquisition of the interior of the coal mine shaft in step S2 includes the following steps:

[0101] S21. The intelligent inspection terminal constructs a three-dimensional space rectangular coordinate system according to the internal layout map of the coal mine to be inspected. The X-axis is set in the width direction of the coal mine, the Y-axis is set in the internal layout direction of the coal mine, and the Z-axis is set in the height direction of the coal mine;

[0102] S22. The intelligent inspection terminal controls the depth camera to take a color image of the inside of the coal mine, obtains the internal color image of the coal mine, and records the shooting coordinates of the depth camera;

[0103] S23. The intelligent inspection terminal controls the depth camera to take a depth image of the inside of the coal mine according to the shooting coordinates of the depth camera, and obtains the internal depth image of the coal mine;

[0104] In this embodiment, in order to more accurately determine the position of the obstacle and the shooting position of the UAV, a three-dimensional space rectangular coordinate system is constructed for the inside of the coal mine. The depth camera of the UAV can not only take color images but also take depth images. The purpose of taking color images is to better determine whether there are obstacles in the coal mine, and the depth image can represent the distance between the depth camera and the obstacle, that is, the obstacle position information.

[0105] Refer to Figure 3 As shown, the specific steps for feature analysis and processing of the internal image of the coal mine in step S3 are as follows:

[0106] S31. The intelligent inspection terminal respectively constructs a two-dimensional space rectangular coordinate system for the internal color image of the coal mine and the internal depth image of the coal mine. The X-axis is in the horizontal direction of the internal color image of the coal mine and the internal depth image of the coal mine, and the Y-axis is in the vertical direction of the internal color image of the coal mine and the internal depth image of the coal mine. The scale lengths represented by the coordinate axes of the two-dimensional space rectangular coordinate system of the internal color image of the coal mine and the two-dimensional space rectangular coordinate system of the internal depth image of the coal mine are the same;

[0107] S32. Based on the blob detection algorithm, perform obstacle feature extraction and processing on the internal color image of the coal mine to determine the position of the obstacle in the two-dimensional space rectangular coordinate system in the color image and the obstacle shape information;

[0108] S33. The intelligent inspection terminal performs distance analysis on the internal depth image of the coal mine according to the position of the obstacle in the two-dimensional space rectangular coordinate system in the color image and the obstacle shape information to determine the obstacle position information;

[0109] S34. The intelligent inspection terminal measures the obstacle shape information in the internal color image of the coal mine to obtain the reduced size information of the obstacle;

[0110] S35. The intelligent inspection terminal performs reduction calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information;

[0111] In this embodiment, by measuring the shape information of the obstacle in the color image, the size of the obstacle in the color image is determined. Then, combined with the zoom ratio of the depth camera, the obstacle size information, that is, the actual size of the obstacle in the coal mine shaft, can be calculated.

[0112] Refer to Figure 4 As shown in the figure, the distance analysis of the internal depth image of the coal mine shaft in step S33 specifically includes the following steps:

[0113] S331. The intelligent inspection terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional space rectangular coordinate system in the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional space rectangular coordinate system in the depth image;

[0114] S332. The intelligent inspection terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each position of the obstacle in the two-dimensional space rectangular coordinate system in the depth image, and determines the pixel value of each position of the obstacle in the depth image;

[0115] S333. The intelligent inspection terminal performs conversion processing on the pixel value of each position of the obstacle in the depth image according to the shooting parameters and coding method of the depth camera to determine the obstacle position information;

[0116] In this embodiment, when an obstacle is determined to exist in the color image, the position of the obstacle in the depth image can be determined according to the position of the obstacle in the color image. Because the color image and the depth image are taken at the same position, and the color image and the depth image taken with the same shooting parameters have the same size. Therefore, by constructing the same two-dimensional space rectangular coordinate system for the color image and the depth image, the position of the obstacle in the depth image can be determined through the position of the obstacle in the color image. Because the depth image represents the distance between the shooting device and the object, the position of the obstacle in the depth image is determined through the coordinates, and then the obstacle position information is obtained.

[0117] Refer to Figure 5 As shown in the figure, the verification processing of the obstacle shape information and the obstacle size information in step S4 specifically includes the following steps:

[0118] S41. The intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the obstacle standard shape information and the obstacle standard size information;

[0119] S42. The intelligent inspection terminal compares and processes the obstacle shape information and obstacle size information respectively with the obstacle standard shape information and obstacle standard size information;

[0120] S43. If the obstacle standard shape information is consistent with the obstacle shape information and the obstacle standard size information is consistent with the obstacle size information, set the obstacle shape information as the obstacle actual shape information and set the obstacle size information as the obstacle actual size information;

[0121] S44. If the obstacle standard shape information is inconsistent with the obstacle shape information and the obstacle standard size information is inconsistent with the obstacle size information, set the obstacle standard shape information as the obstacle actual shape information and set the obstacle standard size information as the obstacle actual size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera;

[0122] In this embodiment, during the image shooting process, the image may be interfered by external factors, resulting in a blurred image, and further may cause an error between the captured obstacle shape and the actual shape of the obstacle. If the error is too large, the drone cannot cross the obstacle, but an instruction that the drone can cross the obstacle is issued, which may cause the drone to collide with the obstacle and then crash, increasing the inspection cost. Therefore, the position of the obstacle is scanned by a laser scanner to determine the obstacle standard shape information and obstacle standard size information. And the reason why the laser scanner is not always used for inspection is that the energy consumption of the laser scanner is too high. If it is always used, the energy consumption of the drone will be too fast, and more energy needs to be invested in the inspection of the coal mine shaft. Moreover, it also takes time for the drone to replace the energy, indirectly increasing the inspection time.

[0123] Refer to Figure 6 As shown, the specific steps for the risk assessment and analysis of the obstacle actual shape information and obstacle actual size information in step S5 are as follows:

[0124] S51. Based on the intelligent inspection terminal, send a data reading request to an external storage device to obtain an obstacle risk reference table. Among them, the obstacle risk reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence;

[0125] S52. The intelligent inspection terminal performs shape matching processing on the obstacle shape reference table with the obstacle actual shape information as a feature to determine the obstacle type and the obstacle shape risk level;

[0126] S53. The intelligent inspection terminal performs information matching processing on the obstacle size reference table with the obstacle type as a feature to determine the obstacle reference size corresponding to the obstacle type;

[0127] S54. The intelligent inspection terminal performs a size matching process on the obstacle reference size corresponding to the obstacle type based on the actual size information of the obstacle to determine the obstacle size danger level.

[0128] S55. The intelligent inspection terminal performs a danger weight analysis on the obstacle shape danger level and the obstacle size danger level to determine the obstacle danger level.

[0129] Among them, the specific calculation formula for the danger weight analysis is:

[0130]

[0131] In the formula, the is the obstacle danger level; is the obstacle shape danger level; is the weight proportion occupied by the obstacle shape danger level; is the obstacle size danger level; is the weight proportion occupied by the obstacle size danger level.

[0132] In this embodiment, different shapes and sizes of obstacles will cause a certain degree of harm to workers. For example, for relatively sharp obstacles, if the length of the sharp obstacle is also relatively long, the wound caused to the worker will be relatively large. Therefore, the actual shape information and the actual size information of the obstacle are evaluated to determine the obstacle danger level. Then, a warning message is generated based on the obstacle danger level and the obstacle position information. Workers can avoid the obstacle according to the warning message. When the obstacle danger level is too high, the maintenance personnel can remove the obstacle.

[0133] Referring to Figure 7 shown, the specific steps of the feature analysis of the obstacle position information and the actual shape information of the obstacle in step S7 are as follows:

[0134] S71. Based on the intelligent inspection terminal, perform a feature analysis on the actual shape information of the obstacle to determine the edge information of the obstacle. The edge information of the obstacle is specifically any three or any two or any one combination of the obstacle left edge position information, the obstacle right edge position information, the obstacle top edge position information, and the obstacle bottom edge position information.

[0135] S72. The intelligent inspection terminal controls the laser scanner to perform information acquisition and processing around the obstacle position information to obtain the edge coordinates of the coal mine shaft at the obstacle position information. The edge coordinates of the coal mine shaft at the obstacle position information are three-dimensional coordinates.

[0136] S73. The intelligent patrol terminal compares and processes the edge coordinates of the obstacle position information in the coal mine shaft to determine the minimum edge coordinates of the obstacle position information in the coal mine shaft;

[0137] S74. Based on the intelligent patrol terminal, calculate and analyze the minimum edge coordinates of the obstacle position information in the coal mine shaft and the edge information of the obstacle to determine whether the drone can pass;

[0138] In this embodiment, the obstacle appears in the coal mine shaft and there must be a certain distance from the surrounding shaft walls of the coal mine shaft. When the distance is too large, the drone can pass through to perform subsequent patrol tasks. When the distance is too small, the drone cannot pass through and cannot perform subsequent patrol tasks. Therefore, it is necessary to determine the minimum edge coordinates of the obstacle position information in the coal mine shaft.

[0139] Refer to Figure 8 As shown, the specific steps for calculating and analyzing the minimum edge coordinates of the obstacle position information in the coal mine shaft and the edge information of the obstacle in step S74 are as follows:

[0140] S741. Based on the intelligent patrol terminal, perform a difference calculation on the minimum edge coordinates of the obstacle position information in the coal mine shaft and the edge information of the obstacle to obtain the passing distance data;

[0141] S742. Based on the intelligent patrol terminal, analyze the size of the drone to obtain the height data and length data of the drone;

[0142] S743. Based on the intelligent patrol terminal, judge and process the passing distance data, the height data of the drone, and the length data of the drone to determine whether the drone can pass.

[0143] Refer to Figure 9 As shown, the specific steps for judging and processing the passing distance data, the height data of the drone, and the length data of the drone in step S743 are as follows:

[0144] S7431. If the passing distance data is the distance data between the top or bottom of the obstacle and the edge of the coal mine shaft;

[0145] S7432. Based on the intelligent patrol terminal, judge and process the height data of the drone and the passing distance data;

[0146] S7433. If the height data of the drone is greater than or equal to the passing distance data, the drone cannot pass;

[0147] S7434. If the height data of the drone is less than the passing distance data, the drone can pass and can perform subsequent patrol tasks;

[0148] S7435. If the passing interval data is the interval data between the left or right side of the obstacle and the edge of the coal mine shaft;

[0149] S7436. Based on the intelligent inspection terminal, judge and process the length data and passing interval data of the unmanned aerial vehicle;

[0150] S7437. If the length data of the unmanned aerial vehicle is greater than or equal to the passing interval data, the unmanned aerial vehicle cannot pass;

[0151] S7438. If the length data of the unmanned aerial vehicle is less than the passing interval data, the unmanned aerial vehicle can pass and can perform subsequent inspection tasks;

[0152] In this embodiment, when the position of the obstacle is different, the formed passing interval data will be different. For example, when the obstacle is too large, part of the area of the coal mine shaft is blocked. For example, the top and bottom of the coal mine shaft are completely blocked by the obstacle, leaving only the left and right partial intervals. At this time, if the length of the interval is greater than the length of the unmanned aerial vehicle, the unmanned aerial vehicle can pass through this obstacle and perform subsequent inspections. If the left and right sides of the coal mine shaft are completely blocked by the obstacle, leaving only the top partial interval, at this time, it is necessary to judge the width of the unmanned aerial vehicle and this interval data to determine whether the unmanned aerial vehicle can cross this obstacle and perform subsequent inspections.

[0153] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for autonomous inspection of drones in underground coal mines, characterized in that, Including: S1. Import the internal layout map of the coal mine to be inspected into the intelligent inspection terminal of the UAV. The internal layout map of the coal mine to be inspected is obtained from the database system. S2. The intelligent inspection terminal performs real-time image acquisition of the inside of the coal mine according to the internal layout map of the coal mine to be inspected, and obtains the internal image of the coal mine. The internal image of the coal mine includes the internal color image of the coal mine and the internal depth image of the coal mine. S3. The intelligent inspection terminal performs feature analysis and processing on the internal image of the coal mine to determine the relevant data of the internal obstacles in the coal mine. The relevant data of the internal obstacles in the coal mine includes obstacle position information, obstacle shape information, and obstacle size information. S4. The laser scanner verifies the obstacle shape information and obstacle size information based on the obstacle position information to obtain the actual obstacle shape information and the actual obstacle size information. S5. The intelligent inspection terminal performs a risk assessment analysis on the actual obstacle shape information and the actual obstacle size information to determine the obstacle risk level. S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle risk level and the obstacle position information, and sends the warning information to the electronic devices of each worker. S7. The intelligent inspection terminal performs feature analysis on the obstacle position information and the actual obstacle shape information to determine whether the UAV can pass. Step S7 includes: S71. The intelligent inspection terminal performs feature analysis on the actual obstacle shape information to determine the edge information of the obstacle. The edge information of the obstacle is specifically any three or any two combinations or any one of the left edge position information of the obstacle, the right edge position information of the obstacle, the top edge position information of the obstacle, and the bottom edge position information of the obstacle. S72. The intelligent inspection terminal controls the laser scanner to collect and process information around the obstacle position information to obtain the edge coordinates of the coal mine at the obstacle position information. The edge coordinates of the coal mine at the obstacle position information are three-dimensional coordinates. S73. The intelligent inspection terminal performs a comparison process on the edge coordinates of the coal mine at the obstacle position information to determine the minimum edge coordinates of the coal mine at the obstacle position information. S74. The intelligent inspection terminal performs calculation and analysis processing on the minimum edge coordinates of the coal mine at the obstacle position information and the edge information of the obstacle to determine whether the UAV can pass.

2. The method for autonomous inspection of underground coal mine drones according to claim 1, characterized in that, The real-time image acquisition of the inside of the coal mine in step S2 includes the following steps: S21. The intelligent inspection terminal constructs a three-dimensional space rectangular coordinate system according to the internal layout map of the coal mine to be inspected, sets the X-axis in the width direction of the coal mine, sets the Y-axis in the internal layout direction of the coal mine, and sets the Z-axis in the height direction of the coal mine. S22. The intelligent inspection terminal controls the depth camera to take a color image of the inside of the coal mine to obtain the internal color image of the coal mine, and records the shooting coordinates of the depth camera. S23. The intelligent inspection terminal controls the depth camera to take a depth image of the inside of the coal mine according to the shooting coordinates of the depth camera to obtain the internal depth image of the coal mine.

3. A method for autonomous inspection of underground coal mine drones according to claim 1, characterized in that, In step S3, the feature analysis and processing of the internal image of the coal mine shaft specifically includes the following steps: S31. The intelligent inspection terminal respectively constructs a two-dimensional rectangular coordinate system for the internal color image and the internal depth image of the coal mine shaft. Taking the horizontal direction of the internal color image and the internal depth image of the coal mine shaft as the X-axis and the vertical direction of the internal color image and the internal depth image of the coal mine shaft as the Y-axis, the scale of the coordinate axes of the two-dimensional rectangular coordinate system of the internal color image of the coal mine shaft and the two-dimensional rectangular coordinate system of the internal depth image of the coal mine shaft represents the same length; S32. Based on the speckle detection algorithm, perform obstacle feature extraction processing on the internal color image of the coal mine shaft to determine the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information; S33. The intelligent inspection terminal performs distance analysis on the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information to determine the obstacle position information; S34. The intelligent inspection terminal measures the obstacle shape information in the internal color image of the coal mine shaft to obtain the reduced size information of the obstacle; S35. The intelligent inspection terminal performs reduction calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information.

4. The method for autonomous inspection of underground coal mine drones according to claim 3, characterized in that, In step S33, the distance analysis of the internal depth image of the coal mine shaft specifically includes the following steps: S331. The intelligent inspection terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional rectangular coordinate system of the depth image; S332. The intelligent inspection terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each position of the obstacle in the two-dimensional rectangular coordinate system of the depth image to determine the pixel value of each position of the obstacle in the depth image; S333. The intelligent inspection terminal performs conversion processing on the pixel value of each position of the obstacle in the depth image according to the shooting parameters and coding method of the depth camera to determine the obstacle position information.

5. A method for autonomous inspection of underground coal mine drones according to claim 1, characterized in that, In step S4, the verification processing of the obstacle shape information and the obstacle size information specifically includes the following steps: S41. The intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the obstacle standard shape information and the obstacle standard size information; S42. The intelligent inspection terminal respectively compares the obstacle shape information and the obstacle size information with the obstacle standard shape information and the obstacle standard size information; S43. If the obstacle standard shape information is consistent with the obstacle shape information and the obstacle standard size information is consistent with the obstacle size information, set the obstacle shape information as the obstacle actual shape information and set the obstacle size information as the obstacle actual size information; S44. If the obstacle standard shape information is inconsistent with the obstacle shape information and the obstacle standard size information is inconsistent with the obstacle size information, set the obstacle standard shape information to the actual obstacle shape information, set the obstacle standard size information to the actual obstacle size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera.

6. The method for autonomous inspection of an underground coal mine drone according to claim 1, characterized in that, The specific steps for performing a risk assessment and analysis on the actual obstacle shape information and the actual obstacle size information in step S5 are as follows: S51. Based on the intelligent inspection terminal, send a data reading request to an external storage device to obtain an obstacle risk reference table. Among them, the obstacle risk reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence; S52. The intelligent inspection terminal performs shape matching processing on the obstacle shape reference table with the actual obstacle shape information as a feature to determine the obstacle type and the obstacle shape risk level; S53. The intelligent inspection terminal performs information matching processing on the obstacle size reference table with the obstacle type as a feature to determine the reference size of the obstacle corresponding to the obstacle type; S54. The intelligent inspection terminal performs size matching processing on the reference size of the obstacle corresponding to the obstacle type with the actual obstacle size information as a feature to determine the obstacle size risk level; S55. The intelligent inspection terminal performs a risk weight analysis on the obstacle shape risk level and the obstacle size risk level to determine the obstacle risk level.

7. The method for autonomous inspection of underground coal mine drones according to claim 6, characterized in that, The specific calculation formula for the risk weight analysis is: ； Wherein, the is the obstacle danger level; is the obstacle shape danger level; is the weight proportion of the obstacle shape danger level; is the obstacle size danger level; is the weight proportion of the obstacle size danger level.

8. A method for autonomous inspection of drones in underground coal mines according to claim 1, characterized in that, The specific steps for performing a calculation and analysis on the minimum edge coordinates of the coal mine shaft at the obstacle position information and the edge information of the obstacle in step S74 are as follows: S741. Based on the intelligent inspection terminal, perform a difference calculation on the minimum edge coordinates of the coal mine shaft at the obstacle position information and the edge information of the obstacle to obtain the passing spacing data; S742. Based on the intelligent inspection terminal, perform a size analysis on the drone to obtain the height data and the length data of the drone; S743. Based on the intelligent inspection terminal, perform a judgment process on the passing spacing data, the height data of the drone, and the length data of the drone to determine whether the drone can pass.

9. A method for autonomous inspection of underground coal mine drones according to claim 8, characterized in that, The specific steps for performing a judgment process on the passing spacing data, the height data of the drone, and the length data of the drone in step S743 are as follows: S7431. If the passing spacing data is the spacing data between the top or bottom of the obstacle and the edge of the coal mine shaft; S7432. Based on the intelligent inspection terminal, perform a judgment process on the height data of the drone and the passing spacing data; S7433. If the height data of the drone is greater than or equal to the passing spacing data, the drone cannot pass; S7434. If the height data of the drone is less than the passing spacing data, the drone can pass and can perform subsequent inspection tasks; S7435. If the passing spacing data is the spacing data between the left or right side of the obstacle and the edge of the coal mine shaft; S7436. Based on the intelligent inspection terminal, perform a judgment process on the length data of the drone and the passing spacing data; S7437. If the length data of the UAV is greater than or equal to the passing spacing data, the UAV cannot pass through. S7438. If the length data of the UAV is less than the passing spacing data, the UAV can pass through and can perform subsequent inspection tasks.

Images