Underground coal mine unmanned aerial vehicle autonomous inspection method

Abstract

The invention discloses a coal mine underground unmanned aerial vehicle autonomous inspection method, and relates to the technical field of image analysis, and the method comprises the steps: S1, importing a to-be-inspected coal mine internal trend map into an intelligent inspection terminal of an unmanned aerial vehicle; and S2, the intelligent inspection terminal performs real-time image acquisition on the interior of the coal mine well according to the to-be-inspected coal mine well interior trend graph. The method comprises the following steps: firstly, carrying out feature analysis on an internal color image and an internal depth image of a coal mine well, and determining obstacle position information, obstacle actual shape information and obstacle actual size information; the method comprises the following steps: acquiring obstacle position information, obstacle actual shape information and obstacle actual size information, performing risk degree analysis on the obstacle position information, the obstacle actual shape information and the obstacle actual size information to determine the obstacle risk level, and finally performing distance analysis on the obstacle and the surrounding well wall of the coal mine well to determine that the unmanned aerial vehicle can cross the obstacle to perform subsequent inspection, thereby shortening the inspection period of the coal mine well, and meanwhile, improving the inspection efficiency. And the potential safety hazard of the inspection personnel is reduced to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a method for autonomous inspection of unmanned aerial vehicles in underground coal mines. Background Art

[0002] A coal mine is an underground or open-pit mine used to mine coal resources. It is the main place for coal production and usually includes structures such as shafts, tunnels, and working faces.

[0003] When the length of the coal mine is long or the depth of the coal mine is deep, the manual inspection cycle is too long, which increases the work intensity of the workers. In addition, with the mining of coal, certain dangers may appear inside the coal mine, and workers may be injured during the inspection. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, a method for autonomous inspection by drones in coal mines is provided. This technical solution solves the problem raised in the above-mentioned background technology that when the length of the coal mine is long or the depth of the coal mine is deep, the manual inspection cycle is too long, which increases the work intensity of the workers. In addition, with the mining of coal, certain dangers may appear inside the coal mine, and workers may be injured during the inspection.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: A method for autonomous inspection of an unmanned aerial vehicle in a coal mine, comprising: S1. Importing the internal trend map of the coal mine to be inspected into the intelligent inspection terminal of the drone, wherein the internal trend map of the coal mine to be inspected is obtained from the database system; It is understandable that the drone conducts inspections based on the internal orientation images of the coal mine shaft, because the orientation in the coal mine shaft is formed according to the distribution of coal, and there may be multiple shafts; S2. The intelligent inspection terminal collects real-time images of the interior of the coal mine according to the internal trend map of the coal mine to be inspected, and obtains an internal image of the coal mine, wherein the internal image of the coal mine includes an internal color image of the coal mine and an internal depth image of the coal mine; It can be understood that the depth image represents the actual distance between the shooting device and the object being shot, and therefore, the position of the obstacle can be determined by shooting the depth image; The internal depth image represents the distance between the depth camera and obstacles inside the coal mine, that is, each pixel value in the depth image represents the distance between a certain position inside the coal mine and the camera; S3. Based on the intelligent inspection terminal, feature analysis and processing are performed on the internal image of the coal mine to determine relevant data of internal obstacles of the coal mine, wherein the relevant data of the internal obstacles of the coal mine includes obstacle position information, obstacle shape information, and obstacle size information; S4. Based on the laser scanner, the obstacle shape information and obstacle size information are verified and processed with the obstacle position information as the feature to obtain the actual shape information and the actual size information of the obstacle; S5. Based on the intelligent inspection terminal, perform risk assessment and analysis on the actual shape information and actual size information of the obstacle to determine the risk level of the obstacle; S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle danger level and obstacle location information, and sends the warning information to each worker's electronic device; It is understandable that when it is determined that an obstacle is dangerous, in order to avoid scratching the worker, the location of the obstacle is sent to the worker's electronic device to increase the worker's vigilance; S7. Based on the intelligent inspection terminal, feature analysis is performed on the obstacle location information and the actual shape information of the obstacle to determine whether the drone can pass.

[0006] Preferably, the real-time image acquisition of the interior of the coal mine in step S2 includes the following steps: S21. The intelligent inspection terminal constructs a three-dimensional rectangular coordinate system according to the internal trend map of the coal mine to be inspected, sets the X axis in the width direction of the coal mine, sets the Y axis in the internal trend of the coal mine, and sets the Z axis in the height direction of the coal mine; S22, the intelligent inspection terminal controls the depth camera to take color images of the interior of the coal mine, obtains the internal color images of the coal mine, and records the shooting coordinates of the depth camera; It can be understood that in order to determine the obstacle position in the depth image by the obstacle position coordinates in the color image, it is necessary to shoot at the same shooting coordinates; S23. The intelligent inspection terminal controls the depth camera to capture a depth image of the interior of the coal mine according to the shooting coordinates of the depth camera, so as to obtain an internal depth image of the coal mine.

[0007] Preferably, the feature analysis processing of the internal image of the coal mine in step S3 specifically includes the following steps: S31. The intelligent inspection terminal constructs a two-dimensional rectangular coordinate system for the internal color image and the internal depth image of the coal mine, respectively, with the horizontal direction of the internal color image and the internal depth image of the coal mine as the X-axis, and the vertical direction of the internal color image and the internal depth image of the coal mine as the Y-axis, and the lengths represented by the coordinate axis scales of the two-dimensional rectangular coordinate system of the internal color image of the coal mine and the two-dimensional rectangular coordinate system of the internal depth image of the coal mine are consistent; It can be understood that the color image and the depth image construct the same rectangular coordinate system in order to subsequently determine the position of the obstacle in the depth image by the position of the obstacle in the color image; S32, based on the spot detection algorithm, performing obstacle feature extraction processing on the internal color image of the coal mine shaft, and determining the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information; S33, the intelligent inspection terminal performs distance analysis on the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information to determine the obstacle position information; S34, the intelligent inspection terminal measures and processes the obstacle shape information in the internal color image of the coal mine shaft to obtain the reduced size information of the obstacle; S35. The intelligent inspection terminal performs restoration calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information.

[0008] Preferably, the distance analysis of the internal depth image of the coal mine in step S33 specifically includes the following steps: S331, the intelligent inspection terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional rectangular coordinate system in the depth image; S332, the intelligent inspection terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each obstacle position in the two-dimensional rectangular coordinate system in the depth image, and determines the pixel value of each obstacle position in the depth image; S333. The intelligent inspection terminal converts the pixel values ​​of each position of the obstacle in the depth image according to the shooting parameters and encoding method of the depth camera to determine the obstacle position information.

[0009] Preferably, the verification process of the obstacle shape information and obstacle size information in step S4 specifically includes the following steps: S41, the intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the standard shape information and standard size information of the obstacle; S42, the intelligent inspection terminal compares the obstacle shape information and the obstacle size information with the obstacle standard shape information and the obstacle standard size information respectively; S43: If the standard obstacle shape information is consistent with the obstacle shape information, and the standard obstacle size information is consistent with the obstacle size information, the obstacle shape information is set as the actual obstacle shape information, and the obstacle size information is set as the actual obstacle size information; S44: If the obstacle standard shape information and the obstacle shape information are inconsistent, and the obstacle standard size information and the obstacle size information are inconsistent, the obstacle standard shape information is set as the obstacle actual shape information, and the obstacle standard size information is set as the obstacle actual size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera.

[0010] Preferably, the risk assessment analysis of the actual shape information and the actual size information of the obstacle in step S5 specifically includes the following steps: S51, based on the intelligent inspection terminal, sending a data reading request to an external storage device to obtain an obstacle hazard level reference table, wherein the obstacle hazard level reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence; S52, the intelligent inspection terminal performs shape matching processing on the obstacle shape reference table based on the actual shape information of the obstacle, and determines the obstacle type and obstacle shape hazard level; S53, the intelligent inspection terminal performs information matching processing on the obstacle size reference table based on the obstacle type as a feature, and determines the obstacle reference size corresponding to the obstacle type; S54, the intelligent inspection terminal performs size matching processing on the obstacle reference size corresponding to the obstacle type based on the actual size information of the obstacle as a feature, and determines the obstacle size danger level; S55. The intelligent inspection terminal performs a hazard weight analysis on the obstacle shape hazard level and the obstacle size hazard level to determine the obstacle hazard level.

[0011] Preferably, the specific calculation formula of the risk weight analysis is:

[0012] In the formula, is the obstacle hazard level; is the obstacle shape hazard level; is the weight ratio of the obstacle shape danger level; is the obstacle size hazard level; It is the weight ratio of obstacle size danger level.

[0013] Preferably, the feature analysis of the obstacle position information and the actual shape information of the obstacle in step S7 specifically includes the following steps: S71. Based on the intelligent inspection terminal, perform feature analysis on the actual shape information of the obstacle to determine edge information of the obstacle, where the edge information of the obstacle is specifically any three or any two or any combination of the left edge position information of the obstacle, the right edge position information of the obstacle, the top edge position information of the obstacle, and the bottom edge position information of the obstacle; It is understandable that the obstacle may appear in different positions, it may appear at the top of the coal mine, it may appear at the bottom of the coal mine, or it may completely block the top and bottom of the coal mine, leaving only the left and right sides; S72, the intelligent inspection terminal controls the laser scanner to collect information around the obstacle location information, and obtains the edge coordinates of the coal mine in the obstacle location information, where the edge coordinates of the coal mine in the obstacle location information are three-dimensional coordinates; S73, the intelligent inspection terminal compares and processes the edge coordinates of the coal mine shaft at the obstacle location information to determine the minimum edge coordinates of the coal mine shaft at the obstacle location information; S74. Based on the intelligent inspection terminal, the minimum edge coordinates of the obstacle location information and the edge information of the obstacle in the coal mine are calculated and analyzed to determine whether the drone can pass.

[0014] Preferably, the step S74 of calculating and analyzing the minimum edge coordinates of the obstacle position information of the coal mine and the edge information of the obstacle specifically includes the following steps: S741. Based on the intelligent inspection terminal, the minimum edge coordinates of the obstacle position information of the coal mine and the edge information of the obstacle are calculated to obtain the clearance data; S742. Based on the intelligent inspection terminal, perform size analysis on the drone to obtain the height data and length data of the drone; S743. Based on the intelligent inspection terminal, the clearance distance data, the height data of the drone, and the length data of the drone are judged and processed to determine whether the drone can pass.

[0015] Preferably, the judgment and processing of the traffic spacing data, the height data of the drone, and the length data of the drone in step S743 specifically includes the following steps: S7431. If the clearance data is the clearance data between the top of the obstacle or the bottom of the obstacle and the edge of the coal mine; S7432, based on the intelligent inspection terminal, judge and process the altitude data and traffic spacing data of the drone; S7433: If the altitude data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7434: If the altitude data of the drone is less than the clearance data, the drone can pass and can carry out subsequent inspection tasks; S7435, if the clearance data is the clearance data between the left side of the obstacle or the right side of the obstacle and the edge of the coal mine; S7436. Based on the intelligent inspection terminal, the length data and the clearance data of the drone are judged and processed; S7437: If the length data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7438. If the length data of the drone is less than the clearance data, the drone can pass and can carry out subsequent inspection tasks.

[0016] Furthermore, a coal mine underground unmanned aerial vehicle autonomous inspection system is proposed, which is used to implement the above-mentioned coal mine underground unmanned aerial vehicle autonomous inspection method, including: An intelligent inspection terminal, which performs image acquisition, image analysis, and obstacle feature calculation based on the internal trend map of the coal mine to be inspected, determines the obstacle danger level, and determines whether the drone can pass; A database system, the database system being used to store an internal trend map of a coal mine to be inspected; Among them, the internal integration of the intelligent inspection terminal includes: Central control module, which is used to control data transmission and information interaction between various modules; A depth camera, which is used to collect color images and depth images of the interior of the coal mine; An image analysis module, the image analysis module is used to analyze the position of obstacles, calculate the size of obstacles, and analyze the shape of obstacles on the internal color image of the coal mine and the internal depth image of the coal mine; A feature verification module, which controls a laser scanner to perform laser scanning on the obstacle position information and verifies the obstacle shape information and obstacle size information through the scanned data; A hazard level assessment module, the hazard level assessment module is used to perform hazard weight analysis on the obstacle shape hazard level and the obstacle size hazard level to determine the obstacle hazard level; A warning information generation module, wherein the warning information generation module generates corresponding warning information according to the obstacle danger level and obstacle location information to ensure the construction safety of workers; The spacing judgment module is used to judge and process the passing spacing data, the height data of the drone and the length data of the drone to determine whether the drone can pass through and complete the subsequent inspection tasks.

[0017] Furthermore, a storage medium is proposed, on which a computer program is stored. When the computer program is called and run, it executes the above-mentioned method for autonomous inspection of underground coal mine by unmanned aerial vehicles.

[0018] Compared with the prior art, the present invention provides a method for autonomous inspection of coal mines by drones, which has the following beneficial effects: The present invention first performs feature analysis on the internal color image and the internal depth image of the coal mine shaft to determine the obstacle position information, the actual shape information of the obstacle and the actual size information of the obstacle, then performs a hazard analysis on the obstacle position information, the actual shape information of the obstacle and the actual size information of the obstacle to determine the obstacle hazard level, and finally, performs a spacing analysis on the obstacle and the surrounding walls of the coal mine shaft to determine that the drone can cross the obstacle for subsequent inspections, thereby shortening the inspection cycle of the coal mine shaft and, at the same time, reducing the safety hazards of the inspection personnel to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of steps S1-S7 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 2 This is a flow chart of steps S21-S23 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 3 This is a flow chart of steps S31-S35 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 4 This is a flow chart of steps S331-S333 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 5 This is a flow chart of steps S41-S44 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 6 This is a flow chart of steps S51-S55 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 7 This is a flow chart of steps S71-S74 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Figure 8 This is a flow chart of steps S741-S743 in a method for autonomous inspection of underground coal mines by unmanned aerial vehicles proposed by the present invention; Fig. 9 This is a flow chart of steps S7431-S7438 in a method for autonomous inspection of underground coal mine drones proposed by the present invention. DETAILED DESCRIPTION

[0020] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0021] Reference Figure 1 As shown, a method for autonomous inspection of an underground coal mine by a UAV includes: S1. Importing the internal trend map of the coal mine to be inspected into the intelligent inspection terminal of the drone, wherein the internal trend map of the coal mine to be inspected is obtained from the database system; S2. The intelligent inspection terminal collects real-time images of the interior of the coal mine according to the internal trend map of the coal mine to be inspected, and obtains an internal image of the coal mine, wherein the internal image of the coal mine includes an internal color image of the coal mine and an internal depth image of the coal mine; S3. Based on the intelligent inspection terminal, feature analysis and processing are performed on the internal image of the coal mine to determine relevant data of internal obstacles of the coal mine, wherein the relevant data of the internal obstacles of the coal mine includes obstacle position information, obstacle shape information, and obstacle size information; S4. Based on the laser scanner, the obstacle shape information and obstacle size information are verified and processed with the obstacle position information as the feature to obtain the actual shape information and the actual size information of the obstacle; S5. Based on the intelligent inspection terminal, perform risk assessment and analysis on the actual shape information and actual size information of the obstacle to determine the risk level of the obstacle; S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle danger level and obstacle location information, and sends the warning information to each worker's electronic device; S7. Based on the intelligent inspection terminal, feature analysis is performed on the obstacle location information and the actual shape information of the obstacle to determine whether the drone can pass; It is understandable to those skilled in the art that with excessive mining of coal, the depth of the coal mine will be too deep and the lateral extension of the coal mine will become very long. If the coal mine that is too deep and too long is inspected manually, a lot of time will be wasted and the progress of coal mining will be delayed. Therefore, the interior of the coal mine is imaged by a drone, and then feature analysis is performed on the acquired image to determine the location information of obstacles in the coal mine, the actual shape information of the obstacles, and the actual size information of the obstacles. The above information is evaluated for danger and the danger level of the obstacles is determined. Finally, warning information is generated based on the danger level of the obstacles and the location information of the obstacles. Workers avoid obstacles based on the warning information, or maintenance personnel clear obstacles based on the warning information, thereby avoiding injuries to workers caused by obstacles. In addition, the spacing between the obstacles and the surrounding walls of the coal mine is analyzed to determine whether the drone can cross the obstacles to complete subsequent inspections.

[0022] Reference Figure 2 As shown, the real-time image acquisition of the interior of the coal mine in step S2 includes the following steps: S21. The intelligent inspection terminal constructs a three-dimensional rectangular coordinate system according to the internal trend map of the coal mine to be inspected, sets the X axis in the width direction of the coal mine, sets the Y axis in the internal trend of the coal mine, and sets the Z axis in the height direction of the coal mine; S22, the intelligent inspection terminal controls the depth camera to take color images of the interior of the coal mine, obtains the internal color images of the coal mine, and records the shooting coordinates of the depth camera; S23, the intelligent inspection terminal controls the depth camera to capture the depth image of the interior of the coal mine according to the shooting coordinates of the depth camera, and obtains the internal depth image of the coal mine; In this embodiment, in order to more accurately determine the location of the obstacle and the shooting position of the drone, a three-dimensional rectangular coordinate system is constructed for the interior of the coal mine, and the depth camera of the drone can not only take color images but also take depth images. The purpose of taking color images is to better determine whether there are obstacles in the coal mine, and the depth image can represent the distance between the depth camera and the obstacle, that is, the obstacle location information.

[0023] Reference Figure 3 As shown, the feature analysis processing of the internal image of the coal mine in step S3 specifically includes the following steps: S31. The intelligent inspection terminal constructs a two-dimensional rectangular coordinate system for the internal color image and the internal depth image of the coal mine, respectively, with the horizontal direction of the internal color image and the internal depth image of the coal mine as the X-axis, and the vertical direction of the internal color image and the internal depth image of the coal mine as the Y-axis, and the lengths represented by the coordinate axis scales of the two-dimensional rectangular coordinate system of the internal color image of the coal mine and the two-dimensional rectangular coordinate system of the internal depth image of the coal mine are consistent; S32, based on the spot detection algorithm, performing obstacle feature extraction processing on the internal color image of the coal mine shaft, and determining the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information; S33, the intelligent inspection terminal performs distance analysis on the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information to determine the obstacle position information; S34, the intelligent inspection terminal measures and processes the obstacle shape information in the internal color image of the coal mine shaft to obtain the reduced size information of the obstacle; S35. The intelligent inspection terminal performs restoration calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information; In this embodiment, the obstacle shape information in the color image is measured to determine the size of the obstacle in the color image, and then, combined with the scaling ratio of the depth camera, the obstacle size information, that is, the actual size of the obstacle in the coal mine, can be calculated.

[0024] Reference Figure 4 As shown, the distance analysis of the internal depth image of the coal mine in step S33 specifically includes the following steps: S331, the intelligent inspection terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional rectangular coordinate system in the depth image; S332, the intelligent inspection terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each obstacle position in the two-dimensional rectangular coordinate system in the depth image, and determines the pixel value of each obstacle position in the depth image; S333, the intelligent inspection terminal converts the pixel value of each position of the obstacle in the depth image according to the shooting parameters and encoding method of the depth camera to determine the obstacle position information; In this embodiment, when the existence of an obstacle is determined in the color image, the position of the obstacle in the depth image can be determined according to the position of the obstacle in the color image, because the color image and the depth image are shot at the same position, and the color image and the depth image shot with the same shooting parameters have the same size. Therefore, by constructing the same two-dimensional rectangular coordinate system for the color image and the depth image, the position of the obstacle in the depth image can be determined by the position of the obstacle in the color image. Because the depth image represents the distance between the shooting device and the object, the position of the obstacle in the depth image is determined by the coordinates, and then the obstacle position information is obtained.

[0025] Reference Figure 5 As shown, the verification process of obstacle shape information and obstacle size information in step S4 specifically includes the following steps: S41, the intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the standard shape information and standard size information of the obstacle; S42, the intelligent inspection terminal compares the obstacle shape information and the obstacle size information with the obstacle standard shape information and the obstacle standard size information respectively; S43: If the standard obstacle shape information is consistent with the obstacle shape information, and the standard obstacle size information is consistent with the obstacle size information, the obstacle shape information is set as the actual obstacle shape information, and the obstacle size information is set as the actual obstacle size information; S44. If the obstacle standard shape information and the obstacle shape information are inconsistent, and the obstacle standard size information and the obstacle size information are inconsistent, the obstacle standard shape information is set as the obstacle actual shape information, and the obstacle standard size information is set as the obstacle actual size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera; In this embodiment, the image may be disturbed by external factors during the shooting process, resulting in blurred images, which may further cause errors between the shape of the obstacle photographed and the actual shape of the obstacle. If the error is too large, the drone cannot cross the obstacle, but an instruction is issued that the drone can cross the obstacle, which may cause the drone to collide with the obstacle and then crash, increasing the inspection cost. Therefore, the obstacle position is scanned by a laser scanner to determine the standard shape information and standard size information of the obstacle. The reason why the laser scanner is not used for inspection all the time is because the energy consumption of the laser scanner is too high. If it is used all the time, the drone's energy consumption will be too fast, and more energy will be needed to inspect the coal mine. It also takes time for the drone to replace the energy, which indirectly increases the inspection time.

[0026] Reference Figure 6As shown, the risk assessment analysis of the actual shape information and the actual size information of the obstacle in step S5 specifically includes the following steps: S51, based on the intelligent inspection terminal, sending a data reading request to an external storage device to obtain an obstacle hazard level reference table, wherein the obstacle hazard level reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence; S52, the intelligent inspection terminal performs shape matching processing on the obstacle shape reference table based on the actual shape information of the obstacle, and determines the obstacle type and obstacle shape hazard level; S53, the intelligent inspection terminal performs information matching processing on the obstacle size reference table based on the obstacle type as a feature, and determines the obstacle reference size corresponding to the obstacle type; S54, the intelligent inspection terminal performs size matching processing on the obstacle reference size corresponding to the obstacle type based on the actual size information of the obstacle as a feature, and determines the obstacle size danger level; S55. The intelligent inspection terminal performs a hazard weight analysis on the obstacle shape hazard level and the obstacle size hazard level to determine the obstacle hazard level; The specific calculation formula for risk weight analysis is:

[0027] In the formula, is the obstacle hazard level; is the obstacle shape hazard level; is the weight ratio of the obstacle shape danger level; is the obstacle size hazard level; is the weight ratio of the obstacle size danger level; In this embodiment, obstacles of different shapes and sizes will cause a certain degree of harm to workers. For example, a sharp obstacle, if the length of the sharp obstacle is also relatively long, will cause a larger wound to the worker. Therefore, the actual shape information and the actual size information of the obstacle are evaluated to determine the obstacle hazard level. Then, warning information is generated based on the obstacle hazard level and the obstacle location information. Workers can avoid obstacles based on the warning information. When the obstacle hazard level is too high, maintenance personnel can remove the obstacle.

[0028] Reference Figure 7 As shown, the feature analysis of the obstacle position information and the actual shape information of the obstacle in step S7 specifically includes the following steps: S71. Based on the intelligent inspection terminal, perform feature analysis on the actual shape information of the obstacle to determine edge information of the obstacle, where the edge information of the obstacle is specifically any three or any two or any combination of the left edge position information of the obstacle, the right edge position information of the obstacle, the top edge position information of the obstacle, and the bottom edge position information of the obstacle; S72, the intelligent inspection terminal controls the laser scanner to collect information around the obstacle location information, and obtains the edge coordinates of the coal mine in the obstacle location information, where the edge coordinates of the coal mine in the obstacle location information are three-dimensional coordinates; S73, the intelligent inspection terminal compares and processes the edge coordinates of the coal mine shaft at the obstacle location information to determine the minimum edge coordinates of the coal mine shaft at the obstacle location information; S74, based on the intelligent inspection terminal, the minimum edge coordinates of the obstacle position information of the coal mine and the edge information of the obstacle are calculated and analyzed to determine whether the drone can pass; In this embodiment, the obstacle appears in the coal mine, and there must be a certain distance between it and the surrounding walls of the coal mine. When the distance is too large, the drone can pass through and carry out subsequent inspection tasks. When the distance is too small, the drone cannot pass through and cannot carry out subsequent inspection tasks. Therefore, it is necessary to determine the minimum edge coordinates of the coal mine in the obstacle location information.

[0029] Reference Figure 8 As shown, the calculation and analysis of the minimum edge coordinates of the obstacle position information of the coal mine and the edge information of the obstacle in step S74 specifically includes the following steps: S741. Based on the intelligent inspection terminal, the minimum edge coordinates of the obstacle position information of the coal mine and the edge information of the obstacle are calculated to obtain the clearance data; S742. Based on the intelligent inspection terminal, perform size analysis on the drone to obtain the height data and length data of the drone; S743. Based on the intelligent inspection terminal, the clearance distance data, the height data of the drone, and the length data of the drone are judged and processed to determine whether the drone can pass.

[0030] Reference Fig. 9 As shown, the judgment and processing of the traffic spacing data, the height data of the drone, and the length data of the drone in step S743 specifically includes the following steps: S7431. If the clearance data is the clearance data between the top of the obstacle or the bottom of the obstacle and the edge of the coal mine; S7432, based on the intelligent inspection terminal, judge and process the altitude data and traffic spacing data of the drone; S7433: If the altitude data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7434: If the altitude data of the drone is less than the clearance data, the drone can pass and can carry out subsequent inspection tasks; S7435, if the clearance data is the clearance data between the left side of the obstacle or the right side of the obstacle and the edge of the coal mine; S7436. Based on the intelligent inspection terminal, the length data and the clearance data of the drone are judged and processed; S7437: If the length data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7438: If the length data of the drone is less than the clearance distance data, the drone can pass and can carry out subsequent inspection tasks; In this embodiment, the position of the obstacle is different, and the clearance data formed will be different. For example, when the obstacle is too large, it will block part of the coal mine. For example, the obstacle completely blocks the top and bottom of the coal mine, leaving only partial clearance on the left and right sides. At this time, if the length of the clearance is greater than the length of the drone, the drone can pass through the obstacle and conduct subsequent inspections. If the obstacle completely blocks the left and right sides of the coal mine, leaving only partial clearance at the top, at this time, it is necessary to judge the width of the drone and the clearance data to determine whether the drone can cross the obstacle for subsequent inspections.

[0031] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for autonomous inspection of coal mines by drones, characterized in that: include: S1. Importing the internal trend map of the coal mine to be inspected into the intelligent inspection terminal of the drone, wherein the internal trend map of the coal mine to be inspected is obtained from the database system; S2. The intelligent inspection terminal collects real-time images of the interior of the coal mine according to the internal trend map of the coal mine to be inspected, and obtains an internal image of the coal mine, wherein the internal image of the coal mine includes an internal color image of the coal mine and an internal depth image of the coal mine; S3. Based on the intelligent inspection terminal, feature analysis and processing are performed on the internal image of the coal mine to determine relevant data of internal obstacles of the coal mine, wherein the relevant data of the internal obstacles of the coal mine includes obstacle position information, obstacle shape information, and obstacle size information; S4. Based on the laser scanner, the obstacle shape information and obstacle size information are verified and processed with the obstacle position information as the feature to obtain the actual shape information and the actual size information of the obstacle; S5. Based on the intelligent inspection terminal, perform risk assessment and analysis on the actual shape information and actual size information of the obstacle to determine the risk level of the obstacle; S6. The intelligent inspection terminal generates corresponding warning information according to the obstacle danger level and obstacle location information, and sends the warning information to each worker's electronic device; S7. Based on the intelligent inspection terminal, feature analysis is performed on the obstacle location information and the actual shape information of the obstacle to determine whether the drone can pass.

2. The method for autonomous inspection of underground coal mines by drones according to claim 1, characterized in that: The real-time image acquisition of the interior of the coal mine in step S2 includes the following steps: S21. The intelligent inspection terminal constructs a three-dimensional rectangular coordinate system according to the internal trend map of the coal mine to be inspected, sets the X axis in the width direction of the coal mine, sets the Y axis in the internal trend of the coal mine, and sets the Z axis in the height direction of the coal mine; S22, the intelligent inspection terminal controls the depth camera to take color images of the interior of the coal mine, obtains the internal color images of the coal mine, and records the shooting coordinates of the depth camera; S23. The intelligent inspection terminal controls the depth camera to capture a depth image of the interior of the coal mine according to the shooting coordinates of the depth camera, so as to obtain an internal depth image of the coal mine.

3. The method for autonomous inspection of underground coal mines by drones according to claim 1, characterized in that: The feature analysis processing of the internal image of the coal mine in step S3 specifically includes the following steps: S31. The intelligent inspection terminal constructs a two-dimensional rectangular coordinate system for the internal color image and the internal depth image of the coal mine, respectively, with the horizontal direction of the internal color image and the internal depth image of the coal mine as the X-axis, and the vertical direction of the internal color image and the internal depth image of the coal mine as the Y-axis, and the lengths represented by the coordinate axis scales of the two-dimensional rectangular coordinate system of the internal color image of the coal mine and the two-dimensional rectangular coordinate system of the internal depth image of the coal mine are consistent; S32, based on the spot detection algorithm, performing obstacle feature extraction processing on the internal color image of the coal mine shaft, and determining the position of the obstacle in the two-dimensional rectangular coordinate system of the color image and the obstacle shape information; S33, the intelligent inspection terminal performs distance analysis on the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information to determine the obstacle position information; S34, the intelligent inspection terminal measures and processes the obstacle shape information in the internal color image of the coal mine shaft to obtain the reduced size information of the obstacle; S35. The intelligent inspection terminal performs restoration calculation processing on the reduced size information of the obstacle according to the zoom ratio of the depth camera to obtain the obstacle size information.

4. The method for autonomous inspection of underground coal mines by drones according to claim 3 is characterized in that: The step S33 of performing distance analysis on the internal depth image of the coal mine specifically includes the following steps: S331, the intelligent inspection terminal determines the position of the internal depth image of the coal mine shaft according to the position of the obstacle in the two-dimensional rectangular coordinate system in the color image and the obstacle shape information, and obtains the position of each position of the obstacle in the two-dimensional rectangular coordinate system in the depth image; S332, the intelligent inspection terminal performs pixel value analysis on the internal depth image of the coal mine shaft according to the position of each obstacle position in the two-dimensional rectangular coordinate system in the depth image, and determines the pixel value of each obstacle position in the depth image; S333. The intelligent inspection terminal converts the pixel values ​​of each position of the obstacle in the depth image according to the shooting parameters and encoding method of the depth camera to determine the obstacle position information.

5. The method for autonomous inspection of underground coal mines by drones according to claim 1, characterized in that: The verification process of the obstacle shape information and obstacle size information in step S4 specifically includes the following steps: S41, the intelligent inspection terminal controls the laser scanner to perform laser scanning on the obstacle position information to obtain the standard shape information and standard size information of the obstacle; S42, the intelligent inspection terminal compares the obstacle shape information and the obstacle size information with the obstacle standard shape information and the obstacle standard size information respectively; S43: If the standard obstacle shape information is consistent with the obstacle shape information, and the standard obstacle size information is consistent with the obstacle size information, the obstacle shape information is set as the actual obstacle shape information, and the obstacle size information is set as the actual obstacle size information; S44: If the obstacle standard shape information and the obstacle shape information are inconsistent, and the obstacle standard size information and the obstacle size information are inconsistent, the obstacle standard shape information is set as the obstacle actual shape information, and the obstacle standard size information is set as the obstacle actual size information, and the intelligent inspection terminal adjusts the shooting parameters of the depth camera.

6. The method for autonomous inspection of underground coal mines by drones according to claim 1, characterized in that: The step S5 of performing risk assessment analysis on the actual shape information and the actual size information of the obstacle specifically includes the following steps: S51, based on the intelligent inspection terminal, sending a data reading request to an external storage device to obtain an obstacle hazard level reference table, wherein the obstacle hazard level reference table includes an obstacle shape reference table and an obstacle size reference table, and the obstacle shape reference table and the obstacle size reference table are in one-to-one correspondence; S52, the intelligent inspection terminal performs shape matching processing on the obstacle shape reference table based on the actual shape information of the obstacle, and determines the obstacle type and obstacle shape hazard level; S53, the intelligent inspection terminal performs information matching processing on the obstacle size reference table based on the obstacle type as a feature, and determines the obstacle reference size corresponding to the obstacle type; S54, the intelligent inspection terminal performs size matching processing on the obstacle reference size corresponding to the obstacle type based on the actual size information of the obstacle as a feature, and determines the obstacle size hazard level; S55. The intelligent inspection terminal performs a hazard weight analysis on the obstacle shape hazard level and the obstacle size hazard level to determine the obstacle hazard level.

7. The method for autonomous inspection of underground coal mines by drones according to claim 6, characterized in that: The specific calculation formula for the risk weight analysis is: ； In the formula, is the obstacle hazard level; is the obstacle shape hazard level; is the weight ratio of the obstacle shape danger level; is the obstacle size hazard level; It is the weight ratio of obstacle size danger level.

8. The method for autonomous inspection of underground coal mines by drones according to claim 1, characterized in that: The feature analysis of the obstacle position information and the actual shape information of the obstacle in step S7 specifically includes the following steps: S71. Based on the intelligent inspection terminal, perform feature analysis on the actual shape information of the obstacle to determine edge information of the obstacle, where the edge information of the obstacle is specifically any three or any two or any combination of the left edge position information of the obstacle, the right edge position information of the obstacle, the top edge position information of the obstacle, and the bottom edge position information of the obstacle; S72, the intelligent inspection terminal controls the laser scanner to collect information around the obstacle location information, and obtains the edge coordinates of the coal mine in the obstacle location information, where the edge coordinates of the coal mine in the obstacle location information are three-dimensional coordinates; S73, the intelligent inspection terminal compares and processes the edge coordinates of the coal mine shaft at the obstacle location information to determine the minimum edge coordinates of the coal mine shaft at the obstacle location information; S74. Based on the intelligent inspection terminal, the minimum edge coordinates of the obstacle location information and the edge information of the obstacle in the coal mine are calculated and analyzed to determine whether the drone can pass.

9. The method for autonomous inspection of underground coal mines by drones according to claim 8, characterized in that: The step S74 of calculating and analyzing the minimum edge coordinates of the coal mine at the obstacle position information and the edge information of the obstacle specifically includes the following steps: S741. Based on the intelligent inspection terminal, the minimum edge coordinates of the coal mine at the obstacle position information and the edge information of the obstacle are calculated to obtain the clearance data; S742. Based on the intelligent inspection terminal, perform size analysis on the drone to obtain the height data and length data of the drone; S743. Based on the intelligent inspection terminal, the clearance distance data, the height data of the drone, and the length data of the drone are judged and processed to determine whether the drone can pass.

10. The method for autonomous inspection of underground coal mines by drones according to claim 8, characterized in that: The step S743 of determining and processing the traffic spacing data, the height data of the drone, and the length data of the drone specifically includes the following steps: S7431. If the clearance data is the clearance data between the top of the obstacle or the bottom of the obstacle and the edge of the coal mine; S7432, based on the intelligent inspection terminal, judge and process the altitude data and traffic spacing data of the drone; S7433: If the altitude data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7434: If the altitude data of the drone is less than the clearance data, the drone can pass and can carry out subsequent inspection tasks; S7435, if the clearance data is the clearance data between the left side of the obstacle or the right side of the obstacle and the edge of the coal mine; S7436. Based on the intelligent inspection terminal, the length data and the clearance data of the drone are judged and processed; S7437: If the length data of the drone is greater than or equal to the clearance data, the drone cannot pass; S7438. If the length data of the drone is less than the clearance data, the drone can pass and can carry out subsequent inspection tasks.

Images