Mining area environment real-time monitoring and alarming system based on unmanned aerial vehicle cruising

Abstract

According to the mining area environment real-time monitoring and alarming system based on unmanned aerial vehicle cruising, the route and the shooting point of unmanned aerial vehicle cruising are determined based on the position information and the landform height of the mining area environment, aerial photography can be carried out at any time according to needs, change detection between aerial photography images at the same position at different times is achieved, and the real-time monitoring and alarming of the mining area environment is achieved. According to the method, invalid aerial photography of the environment outside the area is avoided, mining area environment image shooting is carried out based on a shooting point, the mining area environment image is transmitted back to a ground computer, the mining area environment image is synthesized to obtain an aerial panoramic image, the aerial panoramic image is detected and recognized to obtain a construction violation recognition result, and the construction violation recognition result is obtained. By utilizing the super-strong computing capability and storage capability of the ground computer, rapid and accurate construction violation identification is realized, prompt and early warning information is obtained based on a construction violation identification result and is sent to a manager, the construction violation position is rapidly detected, and the construction violation behavior is stopped in time.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental monitoring, and in particular to a real-time monitoring and alarm system for a mining area environment based on unmanned aerial vehicle (UAV) cruising. Background Art

[0002] As an important means of acquiring spatial data, drone aerial photography technology has the advantages of being small and light, with few restrictions on take-off and landing sites, stable flight, a wide field of view, high clarity, real-time transmission at any time, and intelligence. It is a powerful supplement to satellite remote sensing and manned aerial remote sensing, and is widely used in inspection, security, reconnaissance, and environmental monitoring in various complex terrains.

[0003] In the field of environmental monitoring and inspection, monitoring changes in the status of above-ground structures is a common practice, particularly in mining areas. Because house demolition in mining areas significantly impacts the economic well-being of local farmers, timely and effective regulation of illegal construction has become a major challenge for local governments. Currently, monitoring illegal construction in mining areas relies primarily on manual inspections, which are labor-intensive, inefficient, and prone to omissions. Manual drone inspections are also used, requiring manual evaluation of captured images or videos, resulting in low efficiency.

[0004] Currently, in addition to manual inspections, mining area environmental monitoring mainly relies on satellite-based remote sensing images of mining areas. Change detection is performed on remote sensing images acquired at different times to obtain change information and locate illegal buildings in the mining area. However, monitoring using satellite-based remote sensing images of mining areas has the following drawbacks:

[0005] Satellite remote sensing images have a fixed height and viewing angle. For scenes with large ups and downs and complex terrain such as mining areas, blind spots are easily found, which can hide some buildings and make it impossible to fully detect the specific conditions of the mining area.

[0006] Satellite remote sensing images have poor real-time updating performance and cannot update data in real time. However, for illegal construction in mining areas, simple houses may be built in a few days, which obviously cannot effectively stop illegal construction in a timely manner.

[0007] The resolution of satellite remote sensing images is not high, and the illegal buildings are relatively small. It is difficult to extract effective feature points from satellite remote sensing images, which will directly affect the detection effect.

[0008] In order to solve the above problems, the present invention proposes to use drone fixed-point detection to detect illegal buildings in mining areas. Summary of the Invention

[0009] The present invention provides a real-time monitoring and alarm system for mining environment based on unmanned aerial vehicle (UAV) cruise, which is used to solve the problems raised in the background technology.

[0010] A real-time monitoring and alarm system for mining environment based on drone patrol, including:

[0011] The shooting determination module is used to determine the drone's cruising route and shooting points based on the location information and terrain height of the mining environment;

[0012] An image transmission module is used to capture images of the mining area environment based on the shooting points and transmit the images of the mining area environment back to the ground computer;

[0013] The image detection module is used to synthesize the mining area environment image to obtain an aerial panoramic image, detect and identify the aerial panoramic image, and obtain the illegal construction identification result;

[0014] The result warning module is used to obtain warning information based on the illegal building identification results and send it to the manager.

[0015] Preferably, the shooting determination module includes:

[0016] A range determination unit, configured to locate and determine location information of the mining environment based on map software, and determine a flight range of the UAV cruise based on the location information;

[0017] A route determination unit is used to determine the route and altitude of the UAV cruise based on the flight range of the UAV cruise and the terrain altitude of the mining environment;

[0018] The shooting point determination unit is used to determine the shooting point of the drone cruise based on the route and altitude of the drone cruise and the regional characteristics of the mining environment.

[0019] Preferably, the image transmission module includes:

[0020] A marking unit, used to mark the shooting points based on their distribution characteristics to obtain position marking results;

[0021] A shooting unit, configured to shoot an image of the mining area environment at a shooting point using a drone to obtain an initial shot image;

[0022] The marking unit further marks the initial captured image based on the position marking result to obtain a mining area environment image;

[0023] The transmission unit is used to transmit mining environment images back to the ground computer based on the 5G module carried by the drone.

[0024] Preferably, the image detection module includes:

[0025] An image integration unit is used to determine the integration order of the mining environment images based on the position marks in the mining environment images to obtain an initial integrated image, and to fuse the edge portions of adjacent mining environment images in the initial integrated image to obtain an aerial panoramic image;

[0026] The illegal building identification unit is used to detect and identify the aerial panoramic image based on the feature point extraction and matching method to obtain the illegal building identification result.

[0027] Preferably, the result warning module includes:

[0028] A prompt unit, configured to, when the illegal building identification result is that there is no illegal building, determine that the prompt information is normal for the cruise, and send it to the administrator through the 5G module;

[0029] The early warning unit is used to determine that the early warning information is a cruise abnormality when the illegal construction identification result is that there is an illegal construction phenomenon, and map the aerial panoramic image to the map of the mining environment, mark the location of the illegal construction on the map, and send the map containing the illegal construction location and early warning information to the manager through the 5G module.

[0030] Preferably, the range determination unit includes:

[0031] An environmental marking unit is used to determine a mining monitoring area based on the location information of the mining environment, and mark the landmark environment and complex environment in the mining monitoring area respectively to obtain a first marking point and a second marking point;

[0032] A range determination unit is configured to establish range verification information based on the first marking point, and to establish an initial flight range of the UAV cruise that satisfies the range verification information based on the primary monitoring range of the UAV and in combination with the mining area monitoring area;

[0033] The range supplement unit is used to determine the cruising flight range for obtaining the omnidirectional information of the second marking point, and integrate the initial flight range with the cruising flight range to obtain the cruising flight range of the UAV.

[0034] Preferably, the route determination unit includes:

[0035] An altitude determination unit, used to determine the altitude of the drone's cruising based on the shooting recognition accuracy requirements;

[0036] The path design unit is used to divide the mining environment into multiple regular areas, and use 90% of the UAV's coverage width as the regular path spacing. Based on the path spacing and full coverage as the standard combined with the flight range, the UAV's initial cruising path for each regular area is determined;

[0037] A specific analyzing unit is configured to determine a monitoring difficulty of the regular area based on the number of second marking points in the regular area, set a path density in the regular area based on the monitoring difficulty, and adjust the initial cruise path based on the path density to obtain a specific cruise path;

[0038] A path determination unit is used to obtain the cruise path of the entire area based on the specific cruise paths of all regular areas, and to obtain the initial cruise path of the UAV from takeoff based on the altitude at which the UAV is cruising;

[0039] a route determination unit, configured to preliminarily establish a cruising sequence of an initial cruising path based on the distribution structure of all regular areas to obtain an initial cruising route, and optimize the initial cruising route based on predicted tailwind and headwind characteristics to obtain a first optimized route;

[0040] a route optimization unit, configured to perform secondary optimization on the first optimized route based on a principle of minimum route turns and a route optimization algorithm to obtain a second optimized route;

[0041] A flight endurance verification unit, configured to determine a flight endurance requirement for completing the second optimized route and to judge whether the flight endurance of the UAV meets the said flight endurance requirement;

[0042] If so, the second optimized route is used as the final drone cruising route;

[0043] Otherwise, the flight distance difference caused by the drone's endurance requirements is obtained, and the specific cruise path is simplified and subsequently optimized on the basis of ensuring the initial cruise path in the regular area. The degree of simplification is determined according to the flight distance difference, and the latest optimized route is obtained as the final drone cruise route.

[0044] Preferably, the shooting point determination unit includes:

[0045] a determination unit, configured to set 90% of the coverage length of the drone's filming as the shooting distance, and a position above the second mark point before the preset distance as the designated shooting point;

[0046] The acquisition unit is used to acquire shooting points according to the shooting distance on the route of the drone cruise, and use the determined shooting points and the designated shooting points as the shooting points of the drone cruise.

[0047] Preferably, the illegal building identification unit includes:

[0048] an image processing unit, configured to perform Gaussian filtering on the aerial panoramic image to obtain a filtered image, perform matrix transformation on the filtered image to obtain a target matrix, divide the target matrix into regions to obtain a plurality of sub-matrices of the same specification, obtain points with maximum values in the sub-matrices as a key point set, and obtain a marked image based on the key point set;

[0049] a scale transformation unit, configured to spatially construct the aerial panoramic image using a first scale of a box filter to obtain a first scale space, and spatially construct the aerial panoramic image using a second scale of a box filter to obtain a second scale space;

[0050] A feature point determination unit is used to compare the marked image with 26 points in the neighborhood of the key points in the first scale space and the second scale space respectively to obtain feature points;

[0051] A direction determination unit is used to obtain the brightness difference characteristics of each 60-degree sector in the circular neighborhood of the feature point, and select the direction of the sector with the largest brightness difference characteristic as the main direction of the feature point;

[0052] An area acquisition unit is configured to acquire a rectangular area block of a preset size in four directions: the horizontal direction of the main direction, the vertical direction of the main direction, the absolute value direction of the horizontal direction of the main direction, and the absolute value direction of the vertical direction of the main direction;

[0053] The illegal building judgment unit is used to match the brightness difference characteristics of the four rectangular area blocks with the standard brightness difference characteristics of the pre-extracted illegal building image to obtain a matching degree. If the matching degree is greater than the preset matching degree, the illegal building identification result is determined to be the presence of an illegal building; otherwise, the illegal building identification result is determined to be the absence of an illegal building.

[0054] Preferably, the transmission unit includes:

[0055] A compression unit, configured to compress the mining area environment image according to a preset compression ratio to obtain a compressed image;

[0056] a sequence determination unit, configured to sort the compressed images based on acquisition time to obtain an initial transmission sequence, perform a preliminary comparison between the compressed images and historical compressed images at corresponding positions to obtain image differences, and adjust the initial transmission sequence in descending order of the image differences to obtain a target transmission sequence;

[0057] a level marking unit, configured to mark the compressed image for transmission level based on image differences;

[0058] a scoring design unit, configured to design a first score for the 5G transmission channel based on the channel occupancy, design a second score for the 5G transmission channel based on the channel transmission rate, design a third score for the 5G transmission channel based on the channel's historical transmission stability, design different transmission requirements based on different transmission levels, and design weights for the first score, the second score, and the third score according to the transmission requirements;

[0059] The channel selection unit determines the transmission score values of all 5G transmission channels for the current compressed image in the target transmission sequence based on the first score, the second score, the third score and their corresponding weight values, selects the 5G transmission channel with the largest transmission score value to transmit the current compressed image, and transmits it back to the ground computer.

[0060] Compared with the prior art, the present invention has achieved the following beneficial effects:

[0061] Based on the location information and terrain height of the mining environment, the drone's cruise route and shooting points are determined, so that aerial photography can be carried out at any time as needed, and change detection between aerial images of the same location at different times is achieved, avoiding invalid aerial photography of the environment outside the area. Mining environment images are captured based on the shooting points and transmitted back to the ground computer. By combining the drone's long-range endurance with 5G technology, remote intelligent environmental monitoring without human intervention can be achieved, greatly improving monitoring efficiency. The mining environment images are synthesized to obtain aerial panoramic images, and detection and identification based on the aerial panoramic images are performed to obtain illegal construction identification results. The super computing power and storage capacity of the ground computer are used to achieve fast and accurate illegal construction identification. Based on the illegal construction identification results, prompt warning information is sent to managers to quickly detect the location of illegal construction and stop illegal construction behavior in time.

[0062] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0063] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 This is a structural diagram of a real-time monitoring and alarm system for mining environment based on drone cruising in an embodiment of the present invention;

[0066] Figure 2 is a structural diagram of the shooting determination module in an embodiment of the present invention;

[0067] Figure 3 is a structural diagram of the image detection module in an embodiment of the present invention;

[0068] Figure 4A flowchart of path determination in an embodiment of the present invention;

[0069] Figure 5 This is a flow chart of illegal building identification in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] Example 1:

[0072] The embodiment of the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruise, such as Figure 1 Shown, including:

[0073] The shooting determination module is used to determine the drone's cruising route and shooting points based on the location information and terrain height of the mining environment;

[0074] An image transmission module is used to capture images of the mining area environment based on the shooting points and transmit the images of the mining area environment back to the ground computer;

[0075] The image detection module is used to synthesize the mining area environment image to obtain an aerial panoramic image, detect and identify the aerial panoramic image, and obtain the illegal construction identification result;

[0076] The result warning module is used to obtain warning information based on the illegal building identification results and send it to the manager.

[0077] In this embodiment, the drone returns home after taking all images.

[0078] In this embodiment, the warning information is sent to the administrator via the 5G module.

[0079] In this embodiment, the location information and landform height of the mining environment are obtained based on map software.

[0080] In this embodiment, a prompt message is sent when the illegal building identification result is normal, and an early warning message is sent when it is abnormal.

[0081] In this embodiment, the frequency of drone aerial photography is determined by the administrator based on actual conditions. It can be once a day or once every few days. For example, in seasons suitable for building houses, the frequency can be higher, and in seasons not suitable for building houses, the frequency can be lower.

[0082] The beneficial effects of the above design scheme are: by determining the route and shooting points of the drone cruise based on the location information and terrain height of the mining environment, aerial photography can be carried out at any time as needed, and change detection between aerial images of the same location at different times can be achieved, avoiding invalid aerial photography of the environment outside the area, and shooting images of the mining environment based on the shooting points, and transmitting the mining environment images back to the ground computer. By combining the long-range endurance of the drone with 5G technology, there is no need to manually take off and land the drone on site, and remote intelligent environmental monitoring without human participation can be achieved, greatly improving the monitoring efficiency. The mining environment images are synthesized to obtain aerial panoramic images, and the aerial panoramic images are detected and identified to obtain illegal construction identification results. The super computing power and storage capacity of the ground computer are used to achieve fast and accurate illegal construction identification, and prompt warning information based on the illegal construction identification results is sent to the manager, so as to quickly detect the location of the illegal construction and stop the illegal construction behavior in time.

[0083] Example 2:

[0084] Based on Example 1, the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruise, such as Figure 2 As shown, the shooting determination module includes:

[0085] A range determination unit, configured to locate and determine location information of the mining environment based on map software, and determine a flight range of the UAV cruise based on the location information;

[0086] A route determination unit is used to determine the route and altitude of the UAV cruise based on the flight range of the UAV cruise and the terrain altitude of the mining environment;

[0087] The shooting point determination unit is used to determine the shooting point of the drone cruise based on the route and altitude of the drone cruise and the regional characteristics of the mining environment.

[0088] In this embodiment, the flight range based on the drone cruise can achieve full coverage of the mining environment.

[0089] In this embodiment, the route and altitude of the drone cruise ensure that there are no blind spots during the cruise process.

[0090] In this embodiment, the shooting points of the drone cruise ensure that there are no blind spots during the cruise.

[0091] The beneficial effects of the above design scheme are: by locating and determining the location information of the mining environment based on map software, determining the flight range of the drone cruise based on the location information, determining the route and altitude of the drone cruise based on the flight range of the drone cruise, combined with the topographic height of the mining environment, determining the shooting point of the drone cruise based on the route and altitude of the drone cruise, combined with the regional characteristics of the mining environment, and realizing aerial photography of only the designated area, avoiding invalid aerial photography of the environment outside the area, so that the shape and boundary of each aerial image are fixed, reducing the difficulty of the detection algorithm, shortening the detection time, and also realizing change detection between aerial images of the same location at different times, reducing the difficulty of the environmental change detection algorithm, and shortening the detection time.

[0092] Example 3:

[0093] Based on Example 1, this embodiment of the present invention provides a real-time monitoring and alarm system for a mining area environment based on drone cruising, wherein the image transmission module includes:

[0094] A marking unit, used to mark the shooting points based on their distribution characteristics to obtain position marking results;

[0095] A shooting unit, configured to shoot an image of the mining area environment at a shooting point using a drone to obtain an initial shot image;

[0096] The marking unit further marks the initial captured image based on the position marking result to obtain a mining area environment image;

[0097] The transmission unit is used to transmit mining environment images back to the ground computer based on the 5G module carried by the drone.

[0098] In this embodiment, the initial captured image is marked based on the position marking result to obtain a mining environment image, which can clarify the area where the image was captured and provide a clear image information basis for subsequent image analysis and processing.

[0099] The beneficial effects of the above design scheme are: by using drones to shoot images of the mining area environment at the shooting point, the initial shooting images are obtained, the image shooting based on the drone is realized, the clarity of the shooting images is guaranteed, and the mining area environment images are transmitted back to the ground computer based on the 5G module carried by the drone. By combining the long-range endurance of the drone with 5G technology, there is no need to manually take off and land the drone on site, and remote intelligent environmental monitoring without human intervention can be realized, greatly improving the monitoring efficiency.

[0100] Example 4:

[0101] Based on Example 1, the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruise, such as Figure 3As shown, the image detection module includes:

[0102] An image integration unit is used to determine the integration order of the mining environment images based on the position marks in the mining environment images to obtain an initial integrated image, and to fuse the edge portions of adjacent mining environment images in the initial integrated image to obtain an aerial panoramic image;

[0103] The illegal building identification unit is used to detect and identify the aerial panoramic image based on the feature point extraction and matching method to obtain the illegal building identification result.

[0104] In this embodiment, the feature point extraction and matching method is to extract feature points of the aerial panoramic image and match them with feature points of the standard image to perform construction recognition.

[0105] The beneficial effects of the above design scheme are: by determining the integration order of the mining environment images based on the position marks in the mining environment images, an initial integrated image is obtained, and the edge parts of the adjacent mining environment images in the initial integrated image are fused to obtain an aerial panoramic image, thereby realizing the integration of multi-region images and providing a high-quality image basis for further analysis and detection. The aerial panoramic images are detected and identified based on the feature point extraction and matching method to obtain the illegal construction identification results, and the super computing power and storage capacity of the ground computer are utilized to achieve fast and accurate illegal construction identification.

[0106] Example 5:

[0107] Based on Example 1, this embodiment of the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruising, wherein the result warning module includes:

[0108] A prompt unit, configured to, when the illegal building identification result is that there is no illegal building, determine that the prompt information is normal for the cruise, and send it to the administrator through the 5G module;

[0109] The early warning unit is used to determine that the early warning information is a cruise abnormality when the illegal construction identification result is that there is an illegal construction phenomenon, and map the aerial panoramic image to the map of the mining environment, mark the location of the illegal construction on the map, and send the map containing the illegal construction location and early warning information to the manager through the 5G module.

[0110] The beneficial effect of the above design scheme is: by sending prompt warning information based on the illegal construction identification results to the manager, the location of the illegal construction can be quickly detected and the illegal construction behavior can be stopped in time.

[0111] Example 6:

[0112] Based on Example 2, this embodiment of the present invention provides a real-time monitoring and alarm system for a mining area environment based on drone cruising, wherein the range determination unit includes:

[0113] An environmental marking unit is used to determine a mining monitoring area based on the location information of the mining environment, and mark the landmark environment and complex environment in the mining monitoring area respectively to obtain a first marking point and a second marking point;

[0114] A range determination unit is configured to establish range verification information based on the first marking point, and to establish an initial flight range of the UAV cruise that satisfies the range verification information based on the primary monitoring range of the UAV and in combination with the mining area monitoring area;

[0115] The range supplement unit is used to determine the cruising flight range for obtaining the omnidirectional information of the second marking point, and integrate the initial flight range with the cruising flight range to obtain the cruising flight range of the UAV.

[0116] In this embodiment, the iconic environment is, for example, an iconic building, an iconic landform, etc.

[0117] In this embodiment, the complex environment is, for example, a complex terrain.

[0118] In this embodiment, the initial flight range can monitor the entire mining area.

[0119] In this embodiment, the cruising flight range can achieve all-round monitoring of complex environments.

[0120] The beneficial effects of the above design scheme are: by determining the mining monitoring area based on the location information of the mining environment, and marking the landmark environment and complex environment in the mining monitoring area respectively, the first marking point and the second marking point are obtained, which provide a basis for determining the flight range, and the range verification information is established based on the first marking point. Based on the one-time monitoring range of the UAV, combined with the mining monitoring area, the initial flight range of the UAV cruise that meets the range verification information is established, and the cruising flight range for obtaining the all-round information of the second marking point is determined. The initial flight range is integrated with the cruising flight range to obtain the initial flight range of the UAV cruise, ensuring that the determined UAV cruise flight range can fully cover the mining area and can achieve all-round monitoring of the complex environment, thereby ensuring the UAV cruise effect.

[0121] Example 7:

[0122] Based on Example 6, this embodiment of the present invention provides a real-time monitoring and alarm system for a mining area environment based on drone cruising, wherein the route determination unit includes:

[0123] An altitude determination unit, used to determine the altitude of the drone's cruising based on the shooting recognition accuracy requirements;

[0124] The path design unit is used to divide the mining environment into multiple regular areas, and use 90% of the UAV's coverage width as the regular path spacing. Based on the path spacing and full coverage as the standard combined with the flight range, the UAV's initial cruising path for each regular area is determined;

[0125] A specific analyzing unit is configured to determine a monitoring difficulty of the regular area based on the number of second marking points in the regular area, set a path density in the regular area based on the monitoring difficulty, and adjust the initial cruise path based on the path density to obtain a specific cruise path;

[0126] A path determination unit is used to obtain the cruise path of the entire area based on the specific cruise paths of all regular areas, and to obtain the initial cruise path of the UAV from takeoff based on the altitude at which the UAV is cruising;

[0127] a route determination unit, configured to preliminarily establish a cruising sequence of an initial cruising path based on the distribution structure of all regular areas to obtain an initial cruising route, and optimize the initial cruising route based on predicted tailwind and headwind characteristics to obtain a first optimized route;

[0128] a route optimization unit, configured to perform secondary optimization on the first optimized route based on a principle of minimum route turns and a route optimization algorithm to obtain a second optimized route;

[0129] A flight endurance verification unit, configured to determine a flight endurance requirement for completing the second optimized route and to judge whether the flight endurance of the UAV meets the said flight endurance requirement;

[0130] If so, the second optimized route is used as the final drone cruising route;

[0131] Otherwise, the flight distance difference caused by the drone's endurance requirements is obtained, and the specific cruise path is simplified and subsequently optimized on the basis of ensuring the initial cruise path in the regular area. The degree of simplification is determined according to the flight distance difference, and the latest optimized route is obtained as the final drone cruise route.

[0132] In this embodiment, Figure 4 As shown in the figure, the specific process of route determination is as follows:

[0133] Determine the cruising altitude of the drone based on the shooting and recognition accuracy requirements;

[0134] The mining environment is divided into multiple regular areas, and 90% of the UAV coverage width is used as the regular path spacing. The initial cruising path of the UAV for each regular area is determined based on the path spacing and full coverage combined with the flight range.

[0135] determining a monitoring difficulty of the regular area based on the number of second marking points in the regular area, setting a path density in the regular area based on the monitoring difficulty, and adjusting the initial cruise path based on the path density to obtain a specific cruise path;

[0136] The cruising path of the entire area is obtained based on the specific cruising paths of all regular areas. Combined with the cruising altitude of the UAV, the initial cruising path of the UAV from takeoff is obtained.

[0137] Based on the distribution structure of all regular areas, a cruising sequence of the initial cruising path is preliminarily established to obtain an initial cruising route. The initial cruising route is optimized based on the predicted tailwind and headwind characteristics to obtain a first optimized route.

[0138] Based on the principle of minimum route turns and the route optimization algorithm, the first optimized route is optimized twice to obtain a second optimized route;

[0139] Determining the endurance requirement for completing the second optimized route, and judging whether the endurance of the drone meets the endurance requirement;

[0140] If so, the second optimized route is used as the final drone cruising route;

[0141] Otherwise, the flight distance difference caused by the drone's endurance requirements is obtained, and the specific cruise path is simplified and subsequently optimized on the basis of ensuring the initial cruise path in the regular area. The degree of simplification is determined according to the flight distance difference, and the latest optimized route is obtained as the final drone cruise route.

[0142] In this embodiment, paths have no directions and routes include directions.

[0143] In this embodiment, the route optimization algorithm is, for example, a greedy algorithm or a genetic algorithm. A greedy algorithm is an algorithm strategy that takes the best (ie, most favorable) option under the current state in each selection step, hoping to eventually lead to a global optimal result.

[0144] In this embodiment, for example, if the route optimization algorithm is a genetic algorithm, the second optimization of the first optimized route is performed based on the principle of minimum route turns and the route optimization algorithm as follows:

[0145] Encoding: Represent the first optimized route as a sequence of nodes, where each node represents a position in the route;

[0146] Initialize the population: randomly generate a set of routes as the initial population;

[0147] Fitness function: Calculate the number of turns for each route. The fewer the number of turns, the higher the fitness.

[0148] Selection operation: select excellent individuals to enter the next generation based on fitness;

[0149] Crossover operation: Perform crossover operation on selected individuals to generate new individuals;

[0150] Mutation operation: perform mutation operations on new individuals to increase the diversity of the population;

[0151] Iterative update: Repeat the selection, crossover, and mutation operations until the termination condition is met.

[0152] In this embodiment, for example, if the route optimization algorithm is a greedy algorithm, the first optimized route is optimized twice based on the principle of minimum route turns and the route optimization algorithm. Specifically, the map layout is first determined, and an adjacency list is used to represent the map, where each node corresponds to a list containing nodes adjacent to the node. The current direction of travel of the first optimized route is recorded, and the greedy algorithm is used to select a node from the adjacent nodes of the current node at each step that can minimize the number of turns as the next node.

[0153] In this embodiment, the first optimized route ensures flying as far downwind as possible to reduce resistance.

[0154] In this embodiment, the second optimized route ensures the least number of turns.

[0155] In this embodiment, the greater the difference in flight distance, the greater the degree of simplification.

[0156] In this embodiment, a greater number of second marking points corresponds to a greater monitoring difficulty and a more complex path.

[0157] In this embodiment, 90% of the coverage width of the drone shooting is used as the normal path spacing to ensure a certain degree of overlap.

[0158] The beneficial effects of the above design scheme are as follows: by dividing the mining environment into regular areas to obtain multiple regular areas, and taking 90% of the drone shooting coverage width as the regular path spacing, the drone's initial cruise path for each regular area is determined according to the path spacing with full coverage as the standard combined with the flight range, and the path planning is performed by area division to ensure the accuracy of the path, the monitoring difficulty of the regular area is determined based on the number of second marking points in the regular area, the path density is set in the regular area based on the monitoring difficulty, and the initial cruise path is adjusted based on the path density to obtain a specific cruise path, so as to achieve different cruise paths for different regional characteristics and ensure the targeted nature of the cruise path, and then optimize the route by two aspects: headwind and turn number, to ensure that the energy consumption of the second optimized route is minimized, and finally the route is verified and simplified according to the drone's own endurance, to ensure that the designed route can be completed by the drone, and only aerial photography of the specified area is achieved, avoiding invalid aerial photography of the environment outside the area, so that the shape and boundary of each aerial image are fixed, reducing the difficulty of the detection algorithm and shortening the detection time.

[0159] Example 8:

[0160] Based on Example 6, this embodiment of the present invention provides a real-time monitoring and alarm system for a mining area environment based on drone cruising, wherein the shooting point determination unit includes:

[0161] a determination unit, configured to set 90% of the coverage length of the drone's filming as the shooting distance, and a position above the second mark point before the preset distance as the designated shooting point;

[0162] The acquisition unit is used to acquire shooting points according to the shooting distance on the route of the drone cruise, and use the determined shooting points and the designated shooting points as the shooting points of the drone cruise.

[0163] The beneficial effect of the above design scheme is: by taking 90% of the drone's shooting coverage length as the shooting distance, and the position before the preset distance above the second mark point as the designated shooting point, the shooting points are obtained according to the shooting distance on the drone's cruising route, and the determined shooting points and the designated shooting points are used as the drone's cruising shooting points to ensure that the obtained shooting points can be fully and accurately covered.

[0164] Example 9:

[0165] Based on Example 4, this embodiment of the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruising, wherein the illegal construction identification unit includes:

[0166] an image processing unit, configured to perform Gaussian filtering on the aerial panoramic image to obtain a filtered image, perform matrix transformation on the filtered image to obtain a target matrix, divide the target matrix into regions to obtain a plurality of sub-matrices of the same specification, obtain points with maximum values in the sub-matrices as a key point set, and obtain a marked image based on the key point set;

[0167] a scale transformation unit, configured to spatially construct the aerial panoramic image using a first scale of a box filter to obtain a first scale space, and spatially construct the aerial panoramic image using a second scale of a box filter to obtain a second scale space;

[0168] A feature point determination unit is used to compare the marked image with 26 points in the neighborhood of the key points in the first scale space and the second scale space respectively to obtain feature points;

[0169] A direction determination unit is used to obtain the brightness difference characteristics of each 60-degree sector in the circular neighborhood of the feature point, and select the direction of the sector with the largest brightness difference characteristic as the main direction of the feature point;

[0170] An area acquisition unit is configured to acquire a rectangular area block of a preset size in four directions: the horizontal direction of the main direction, the vertical direction of the main direction, the absolute value direction of the horizontal direction of the main direction, and the absolute value direction of the vertical direction of the main direction;

[0171] The illegal building judgment unit is used to match the brightness difference characteristics of the four rectangular area blocks with the standard brightness difference characteristics of the pre-extracted illegal building image to obtain a matching degree. If the matching degree is greater than the preset matching degree, the illegal building identification result is determined to be the presence of an illegal building; otherwise, the illegal building identification result is determined to be the absence of an illegal building.

[0172] In this embodiment, Figure 5 The specific process of illegal building identification is as follows:

[0173] Performing Gaussian filtering on the aerial panoramic image to obtain a filtered image, performing matrix transformation on the filtered image to obtain a target matrix, dividing the target matrix into regions to obtain multiple sub-matrices of the same specification, obtaining the points with maximum values in the sub-matrices as a key point set, and obtaining a marked image based on the key point set;

[0174] Performing spatial construction on the aerial panoramic image using a first-scale box filter to obtain a first-scale space, and performing spatial construction on the aerial panoramic image using a second-scale box filter to obtain a second-scale space;

[0175] Compare the marked image with 26 points in the neighborhood of key points in the first scale space and the second scale space respectively to obtain feature points;

[0176] Obtain the brightness difference feature of each 60-degree sector within the circular neighborhood of the feature point, and select the direction of the sector with the largest brightness difference feature as the main direction of the feature point;

[0177] Obtain a rectangular area block of a preset size in four directions: the horizontal direction of the main direction, the vertical direction of the main direction, the absolute value direction of the horizontal direction of the main direction, and the absolute value direction of the vertical direction of the main direction;

[0178] The brightness difference features of the four rectangular area blocks are matched with the standard brightness difference features of the pre-extracted illegal building image to obtain a matching degree. If the matching degree is greater than the preset matching degree, the illegal building identification result is determined to be the presence of an illegal building; otherwise, the illegal building identification result is determined to be the absence of an illegal building.

[0179] In this embodiment, the target matrix is a square matrix consisting of second-order partial derivatives of multivariate functions.

[0180] In this embodiment, the marked image is compared with 26 points in the neighborhood of the key point in the first scale space and the second scale space respectively, and points with large differences compared with the 26 points in the neighborhood of the key point are selected as feature points.

[0181] The beneficial effects of the above design scheme are: by detecting and identifying aerial panoramic images based on feature point extraction and matching, illegal building identification results are obtained, and the super computing power and storage capacity of ground computers are utilized to achieve fast and accurate illegal building identification.

[0182] Example 10:

[0183] Based on Example 3, this embodiment of the present invention provides a real-time monitoring and alarm system for mining environment based on drone cruising, wherein the transmission unit includes:

[0184] A compression unit, configured to compress the mining area environment image according to a preset compression ratio to obtain a compressed image;

[0185] a sequence determination unit, configured to sort the compressed images based on acquisition time to obtain an initial transmission sequence, perform a preliminary comparison between the compressed images and historical compressed images at corresponding positions to obtain image differences, and adjust the initial transmission sequence in descending order of the image differences to obtain a target transmission sequence;

[0186] a level marking unit, configured to mark the compressed image for transmission level based on image differences;

[0187] a scoring design unit, configured to design a first score for the 5G transmission channel based on the channel occupancy, design a second score for the 5G transmission channel based on the channel transmission rate, design a third score for the 5G transmission channel based on the channel's historical transmission stability, design different transmission requirements based on different transmission levels, and design weights for the first score, the second score, and the third score according to the transmission requirements;

[0188] The channel selection unit determines the transmission score values of all 5G transmission channels for the current compressed image in the target transmission sequence based on the first score, the second score, the third score and their corresponding weight values, selects the 5G transmission channel with the largest transmission score value to transmit the current compressed image, and transmits it back to the ground computer.

[0189] In this embodiment, different levels of compressed images correspond to different transmission requirements.

[0190] In this embodiment, the more urgent the transmission, the higher the corresponding transmission level, and the transmission requirement is mainly fast, while the transmission requirement of a low transmission level is mainly stable.

[0191] The beneficial effects of the above design scheme are: by compressing the mining area environment image according to a preset compression ratio to obtain a compressed image, the image transmission volume is reduced to improve the transmission efficiency, the compressed image is sorted based on the acquisition time to obtain an initial transmission sequence, and the compressed image is preliminarily compared with the historical compressed image of the corresponding position to obtain the image difference, and the initial transmission sequence is adjusted in order from large to small according to the image difference to obtain the target transmission sequence, so as to realize priority transmission of images with large changes and improve the speed of staff to discover anomalies, and mark the transmission level of the compressed image based on the image difference, and determine the transmission score value of all 5G transmission channels for the current compressed image in the target transmission sequence based on the first score, the second score and the third score and their corresponding weight values, and select the 5G transmission channel with the largest transmission score value to transmit the current compressed image and transmit it back to the ground computer. By selecting different transmission strategies for different image situations, it is ensured that the image transmission better meets the actual requirements and the efficiency of real-time monitoring and alarm of the area environment is guaranteed from the transmission aspect.

[0192] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of this application document and its equivalents, the present invention is intended to include these modifications and variations.

Claims

1. A real-time monitoring and alarm system for mining area environment based on drone cruise, characterized in that: include: The shooting determination module is used to determine the drone's cruising route and shooting points based on the location information and terrain height of the mining environment; An image transmission module is used to capture images of the mining area environment based on the shooting points and transmit the images of the mining area environment back to the ground computer; The image detection module is used to synthesize the mining area environment image to obtain an aerial panoramic image, detect and identify the aerial panoramic image, and obtain the illegal construction identification result; The result warning module is used to obtain warning information based on the illegal building identification results and send it to the manager.

2. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 1 is characterized in that: The shooting determination module includes: A range determination unit, configured to locate and determine location information of the mining environment based on map software, and determine a flight range of the UAV cruise based on the location information; A route determination unit is used to determine the route and altitude of the UAV cruise based on the flight range of the UAV cruise and the terrain altitude of the mining environment; The shooting point determination unit is used to determine the shooting point of the drone cruise based on the route and altitude of the drone cruise and the regional characteristics of the mining environment.

3. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 1 is characterized in that: The image transmission module includes: A marking unit, used to mark the shooting points based on their distribution characteristics to obtain position marking results; A shooting unit, configured to shoot an image of the mining area environment at a shooting point using a drone to obtain an initial shot image; The marking unit further marks the initial captured image based on the position marking result to obtain a mining area environment image; The transmission unit is used to transmit mining environment images back to the ground computer based on the 5G module carried by the drone.

4. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 1 is characterized in that: The image detection module includes: An image integration unit is used to determine the integration order of the mining environment images based on the position marks in the mining environment images to obtain an initial integrated image, and to fuse the edge portions of adjacent mining environment images in the initial integrated image to obtain an aerial panoramic image; The illegal building identification unit is used to detect and identify the aerial panoramic image based on the feature point extraction and matching method to obtain the illegal building identification result.

5. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 1 is characterized in that: The result warning module includes: A prompt unit, configured to, when the illegal building identification result is that there is no illegal building, determine that the prompt information is normal for the cruise, and send it to the administrator through the 5G module; The early warning unit is used to determine that the early warning information is a cruise abnormality when the illegal construction identification result is that there is an illegal construction phenomenon, and map the aerial panoramic image to the map of the mining environment, mark the location of the illegal construction on the map, and send the map containing the illegal construction location and early warning information to the manager through the 5G module.

6. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 2 is characterized in that: The range determination unit includes: An environmental marking unit is used to determine a mining monitoring area based on the location information of the mining environment, and mark the landmark environment and complex environment in the mining monitoring area respectively to obtain a first marking point and a second marking point; A range determination unit is configured to establish range verification information based on the first marking point, and to establish an initial flight range of the UAV cruise that satisfies the range verification information based on the primary monitoring range of the UAV and in combination with the mining area monitoring area; The range supplement unit is used to determine the cruising flight range for obtaining the omnidirectional information of the second marking point, and integrate the initial flight range with the cruising flight range to obtain the cruising flight range of the UAV.

7. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 6 is characterized in that: The route determination unit includes: An altitude determination unit, used to determine the cruising altitude of the drone based on the shooting recognition accuracy requirements; The path design unit is used to divide the mining environment into multiple regular areas, and use 90% of the UAV's coverage width as the regular path spacing. Based on the path spacing and full coverage as the standard combined with the flight range, the UAV's initial cruising path for each regular area is determined; A specific analyzing unit is configured to determine a monitoring difficulty of the regular area based on the number of second marking points in the regular area, set a path density in the regular area based on the monitoring difficulty, and adjust the initial cruise path based on the path density to obtain a specific cruise path; A path determination unit is used to obtain the cruise path of the entire area based on the specific cruise paths of all regular areas, and to obtain the initial cruise path of the UAV from takeoff based on the altitude at which the UAV is cruising; a route determination unit, configured to preliminarily establish a cruising sequence of an initial cruising path based on the distribution structure of all regular areas to obtain an initial cruising route, and optimize the initial cruising route based on predicted tailwind and headwind characteristics to obtain a first optimized route; a route optimization unit, configured to perform secondary optimization on the first optimized route based on a principle of minimum route turns and a route optimization algorithm to obtain a second optimized route; A flight endurance verification unit, configured to determine a flight endurance requirement for completing the second optimized route and to judge whether the flight endurance of the UAV meets the said flight endurance requirement; If so, the second optimized route is used as the final drone cruising route; Otherwise, the flight distance difference caused by the drone's endurance requirements is obtained, and the specific cruise path is simplified and subsequently optimized on the basis of ensuring the initial cruise path in the regular area. The degree of simplification is determined according to the flight distance difference, and the latest optimized route is obtained as the final drone cruise route.

8. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 6 is characterized in that: The shooting point determination unit includes: a determination unit, configured to set 90% of the coverage length of the drone's filming as the shooting distance, and a position above the second mark point before the preset distance as the designated shooting point; The acquisition unit is used to acquire shooting points according to the shooting distance on the route of the drone cruise, and use the determined shooting points and the designated shooting points as the shooting points of the drone cruise.

9. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 4 is characterized in that: The illegal building identification unit includes: an image processing unit, configured to perform Gaussian filtering on the aerial panoramic image to obtain a filtered image, perform matrix transformation on the filtered image to obtain a target matrix, divide the target matrix into regions to obtain a plurality of sub-matrices of the same specification, obtain points with maximum values in the sub-matrices as a key point set, and obtain a marked image based on the key point set; a scale transformation unit, configured to spatially construct the aerial panoramic image using a first scale of a box filter to obtain a first scale space, and spatially construct the aerial panoramic image using a second scale of a box filter to obtain a second scale space; A feature point determination unit is used to compare the marked image with 26 points in the neighborhood of the key points in the first scale space and the second scale space respectively to obtain feature points; A direction determination unit is used to obtain the brightness difference feature of each 60-degree sector in the circular neighborhood of the feature point, and select the direction of the sector with the largest brightness difference feature as the main direction of the feature point; An area acquisition unit is configured to acquire a rectangular area block of a preset size in four directions: the horizontal direction of the main direction, the vertical direction of the main direction, the absolute value direction of the horizontal direction of the main direction, and the absolute value direction of the vertical direction of the main direction; The illegal building judgment unit is used to match the brightness difference characteristics of the four rectangular area blocks with the standard brightness difference characteristics of the pre-extracted illegal building image to obtain a matching degree. If the matching degree is greater than the preset matching degree, the illegal building identification result is determined to be the presence of an illegal building; otherwise, the illegal building identification result is determined to be the absence of an illegal building.

10. The real-time monitoring and alarm system for mining environment based on drone cruise according to claim 3 is characterized in that: The transmission unit includes: A compression unit, configured to compress the mining area environment image according to a preset compression ratio to obtain a compressed image; a sequence determination unit, configured to sort the compressed images based on acquisition time to obtain an initial transmission sequence, perform a preliminary comparison between the compressed images and historical compressed images at corresponding positions to obtain image differences, and adjust the initial transmission sequence in descending order of the image differences to obtain a target transmission sequence; a level marking unit, configured to mark the compressed image for transmission level based on image differences; a scoring design unit, configured to design a first score for the 5G transmission channel based on the channel occupancy, design a second score for the 5G transmission channel based on the channel transmission rate, design a third score for the 5G transmission channel based on the channel's historical transmission stability, design different transmission requirements based on different transmission levels, and design weights for the first score, the second score, and the third score according to the transmission requirements; The channel selection unit determines the transmission score values of all 5G transmission channels for the current compressed image in the target transmission sequence based on the first score, the second score, the third score and their corresponding weight values, selects the 5G transmission channel with the largest transmission score value to transmit the current compressed image, and transmits it back to the ground computer.

Images