Space-Air-Ground Monitoring System and Method for Dangerous Rock Collapse Based on Pure Visual Perception

Abstract

The present invention provides an air-space-ground dangerous rock collapse monitoring system and method based on pure vision perception, belonging to the technical field of dangerous rock collapse early warning. The invention solves the problems of cumbersome procedures, low efficiency and high danger in the on-site layout of monitoring equipment by means of a cruising unmanned aerial vehicle (UAV), an RTK positioning module, a vision sensing device, an air-dropped target layout device and a real-time modeling workstation. At the same time, the position of the UAV is real-time positioned by the RTK positioning module in combination with a public reference station, and the image data is corrected according to the air-dropped disc target, solving the problems of poor accuracy and susceptibility to interference of the GNSS positioning technology. Finally, the displacement and inclination of the dangerous rock are calculated by the real-time three-dimensional model, and the motion trend and evaluation model are generated, and the multi-index comprehensive weight method is used for hierarchical early warning. The invention takes the UAV as the main equipment, avoiding repeated investment in hardware, and is suitable for large-scale popularization and application in the dangerous rock area on the water-facing side of the reservoir bank where people cannot enter.

Description

Technical Field

[0001] The present invention relates to the technical field of dangerous rock collapse early warning, and in particular to an air-space-ground dangerous rock collapse monitoring system and method based on pure vision perception. Background Art

[0002] The geological environment in the Three Gorges Reservoir area is complex and fragile, with frequent water storage and precipitation regulation. Coupled with the influence of extreme weather such as heavy rain and floods and human activities, it has always been a high-incidence area and frequent-occurrence area of geological disasters. Now, with the increasing deterioration of the rock mass in the drawdown zone, the problem of high-position dangerous rocks has become increasingly prominent. Especially the hydraulic erosion caused by the rise and fall of the water storage level has formed a drawdown zone (water level fluctuation zone) up to 30 m high on both sides of the reservoir area, forming local high-steep and overhanging rock masses, posing great risks and potential hazards. After investigation, there are many potential hazards of dangerous rock collapse in the Three Gorges Reservoir area. Once the dangerous rock cracks and becomes unstable and collapses into the river, it will pose a serious threat to shipping and the safety of the reservoir area. Monitoring and early warning of the damage evolution process of dangerous rock masses is one of the effective means to mitigate and prevent disasters, but the early warning system depends on effective monitoring equipment and technical solutions.

[0003] However, since the reservoir bank slope has a water-facing side, the available working environment and equipment installation space are limited, making it difficult to install monitoring equipment on-site. Moreover, the process of manually climbing and deploying monitoring devices in the wild is highly risky, resulting in the inapplicability of conventional contact monitoring schemes. At the same time, the traditional on-site deployment of monitoring devices has high costs, low efficiency, great danger, and problems such as poor accuracy, low timeliness, and susceptibility to interference of the GNSS (Global Navigation Satellite System) positioning technology. Summary of the Invention

[0004] To solve the above problems in the prior art, the present invention provides an air-space-ground dangerous rock collapse monitoring system and method based on pure vision perception. The invention solves the problems of high cost, low efficiency, and great danger in on-site deployment of monitoring devices through a cruise unmanned aerial vehicle, an RTK (Real Time Kinematic) positioning module, a vision sensing device, an air-drop target deployment device, and a real-time modeling workstation. In addition, the RTK positioning module uses a common reference station and satellite positioning to solve the problems of poor accuracy and susceptibility to interference of the GNSS positioning technology. Finally, the three-dimensional model calculates the motion posture, displacement, and inclination of the dangerous rock and evaluates the dangerous rock mass, solving the problems of poor accuracy and low timeliness of traditional methods. To achieve the above object, the technical solution is as follows:

[0005] On the one hand, the present invention provides an air-space-ground dangerous rock collapse monitoring system based on pure vision perception, the system comprising:

[0006] Cruise UAV, used to perform autonomous flight missions and cover the monitoring area;

[0007] RTK positioning module, used for the cruise UAV to perform high-precision positioning during flight and obtain the accurate position information of the cruise UAV;

[0008] Vision sensing device, used for multi-directional photography of dangerous rock masses to obtain image data of multiple angles of the dangerous rock masses;

[0009] Air-drop target layout device, used to accurately layout targets in the monitoring area, correct the image data, and improve the visual positioning and modeling accuracy;

[0010] Real-time modeling workstation, used to generate 3D models, calculate the displacement, inclination of dangerous rocks and generate the motion trend, and evaluate and give early warnings for the motion trend of the dangerous rock mass.

[0011] Optionally, the RTK positioning module adopts multi-frequency anti-interference technology and multi-step adaptive filtering technology;

[0012] The RTK positioning module, according to the relative positioning principle, uses a common reference station and satellites to calculate the three-dimensional coordinates of the cruise UAV in real time, which is used to assist the calculation process of the image data and improve the calculation speed and accuracy.

[0013] Optionally, the air-drop target layout device includes:

[0014] Carrier UAV, used to carry and drop the target to a predetermined position in the monitoring area;

[0015] Disk target, used to provide a visual reference point with high contrast;

[0016] Drop-type bracket, used to stabilize and release the disk target.

[0017] Optionally, the drop-type bracket includes:

[0018] Conical legs, used to provide stable support and positioning for the drop-type bracket;

[0019] Triangular prism-shaped iron block, used to increase the weight of the drop-type bracket to ensure that the drop-type bracket lands stably after being dropped;

[0020] Bolts, used to fix the disk target;

[0021] Lifting ropes, used to assist the carrier UAV to carry the drop-type bracket and the disk target;

[0022] Magnets, used to assist the temporary adsorption of the carrier UAV and the drop-type bracket to ensure the accurate release of the drop-type bracket;

[0023] The conical legs include:

[0024] A silicone film, used for containing glue. The silicone film ruptures after landing, causing the glue to flow out, thereby increasing the stability of the drop-type bracket.

[0025] A thimble, used for piercing the silicone film and enhancing the grip of the drop-type bracket on the ground, preventing the drop-type bracket from moving or tilting after being deployed.

[0026] Optionally, the aerial target deployment device is deployed inside and outside the deformation area of the dangerous rock mass, and the inclination directions of the deployed disc targets cover the four directions of east, west, south, and north.

[0027] Optionally, the real-time modeling workstation includes:

[0028] A wireless receiver, used for receiving data from the cruising drone;

[0029] A PC terminal, used for processing and displaying the received data;

[0030] Point cloud synthesis software, used for generating a three-dimensional model;

[0031] An image processing device, used for calculating the displacement and inclination of the dangerous rock and generating the motion trend of the dangerous rock mass, and for evaluating and warning the dangerous rock mass.

[0032] Optionally, generating a three-dimensional model includes:

[0033] Obtaining an image of the disc target based on the precise position information of the cruising drone and the image data of multiple angles of the dangerous rock mass;

[0034] Performing two-dimensional image correction based on the image of the disc target to obtain feature points in multiple image sequences;

[0035] Based on the feature points in the multiple image sequences, eliminating invalid feature points according to a set threshold to obtain feature points in valid image sequences;

[0036] Based on the feature points in the valid image sequences, obtaining two-dimensional images with multi-dimensional associations through feature point matching and perspective changes;

[0037] Based on the two-dimensional images with multi-dimensional associations, synthesizing the two-dimensional images with multi-dimensional associations through SFM (Structure from Motion) three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model.

[0038] Optionally, calculating the motion attitude, displacement, and inclination of the dangerous rock and evaluating and warning the dangerous rock mass includes:

[0039] Based on the three-dimensional model, compare the feature points in the three-dimensional model, and obtain the boundary of the dangerous rock mass through the displacement change amount;

[0040] Based on the three-dimensional model and the boundary of the dangerous rock mass, calculate and evaluate the volume, boundary surface and inclination direction of the dangerous rock mass to obtain the displacement, inclination degree and movement trend of the dangerous rock mass;

[0041] Based on the displacement, inclination degree and movement trend of the dangerous rock mass, obtain the evaluation model of the dangerous rock mass;

[0042] Based on the evaluation model of the dangerous rock mass, obtain the danger level and give an early warning to the dangerous rock mass.

[0043] Optionally, obtaining the danger level and giving an early warning to the dangerous rock mass based on the evaluation model of the dangerous rock mass includes:

[0044] Based on the evaluation model of the dangerous rock mass, obtain the speed of the dangerous rock mass , the acceleration of the dangerous rock mass , the displacement of the dangerous rock mass , the inclination angle of the dangerous rock mass and the shooting time ;

[0045] Based on the speed of the dangerous rock mass and the shooting time , calculate through formula (1) to obtain the speed change rate between two adjacent times;

[0046] (1)

[0047] In the formula, is the speed change rate, is the speed change amount at time t + 1, is the speed change amount at time t, is the speed at time t + 1, is the speed at time t, is the speed at time t - 1;

[0048] Based on the acceleration of the dangerous rock mass and the shooting time , calculate through formula (2) to obtain the acceleration change rate between two adjacent times;

[0049] (2)

[0050] In the formula, is the acceleration change rate, is the acceleration change amount at time t + 1, is the acceleration change amount at time is the acceleration at time t+1, is the acceleration at time is the acceleration at time t-1;

[0051] According to the displacement of the dangerous rock mass and the shooting time , by calculating with formula (3), the displacement change rate between two adjacent times is obtained;

[0052] (3)

[0053] In the formula, is the displacement change rate, is the displacement change at time t+1, is the displacement change at time is the displacement at time t+1, is the displacement at time t, is the displacement at time t-1;

[0054] According to the inclination angle of the dangerous rock mass and the shooting time , by calculating with formula (4), the inclination angle change rate between two adjacent times is obtained;

[0055] (4)

[0056] In the formula, is the inclination angle change rate, is the inclination angle change at time t+1, is the inclination angle change at time is the inclination angle at time t+1, is the inclination angle at time is the inclination angle at time t-1;

[0057] According to the evaluation model of the dangerous rock mass, the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle are obtained, and the relationship between the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle conforms to formula (5);

[0058] (5)

[0059] In the formula, is the weight factor of velocity, is the weight factor of acceleration, is the weight factor of displacement, is the weight factor of the tilt angle;

[0060] According to the rate of change of velocity, the rate of change of acceleration, the rate of change of displacement, and the rate of change of tilt angle for two adjacent times, as well as the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement, and the weight factor of tilt angle, the comprehensive weight is obtained through formula (6). ;

[0061] (6)

[0062] According to the comprehensive weight and a preset warning threshold are compared, and according to the warning rules, the danger level is obtained and a warning is issued for the dangerous rock mass. The warning rules are as follows:

[0063] Rule 1: When the warning level is level IV, a blue warning;

[0064] Rule 2: When the warning level is level III, a yellow warning;

[0065] Rule 3: When the warning level is level II, an orange warning;

[0066] Rule 4: When the warning level is level I, a red warning;

[0067] When the warning level reaches orange warning or red warning, the on-site personnel need to evacuate.

[0068] On the other hand, the present invention provides a pure vision perception-based air-space-ground dangerous rock collapse monitoring method, which is implemented by a pure vision perception-based air-space-ground dangerous rock collapse monitoring system. The method includes:

[0069] S1. Install the RTK positioning module and the vision sensing device on the cruise unmanned aerial vehicle, and obtain the regional range of the dangerous rock mass and design the target point layout plan through the initial flight of the cruise unmanned aerial vehicle;

[0070] S2. Place the thorn needle into the slideway of the conical leg, place the silicone film into the upper cavity of the conical leg, modulate the glue, pour it into the silicone film, and tie it up. Then screw the 3 conical legs into the screw holes at the three corners of the triangular prism-shaped iron block. Finally, connect one end of the 3 suspension ropes to the three corners of the triangular prism-shaped iron block and the other end to the magnet to obtain the drop-type support;

[0071] S3. According to the target layout plan, use a transport drone to drop the disc target and the drop-type bracket to the pre-designed target position, obtaining the area of the dangerous rock mass with the disc target;

[0072] S4. According to the area of the dangerous rock mass with the disc target, set the flight area and route of the cruise drone, and start the cruise drone to take pictures of the area of the dangerous rock mass with the disc target in the automatic cruise state, obtaining the accurate position information of the cruise drone and the image data of multiple angles of the dangerous rock mass, and transmitting the accurate position information of the cruise drone and the image data of multiple angles of the dangerous rock mass to the real-time modeling workstation;

[0073] S6. According to the image data of multiple angles of the dangerous rock mass, use the disc target to correct the image data of multiple angles of the dangerous rock mass, obtaining the corrected image data;

[0074] S9. According to the three-dimensional model and the boundary of the dangerous rock mass, calculate and evaluate the volume, boundary surface and inclination direction of the dangerous rock mass, obtaining the displacement, inclination degree of the dangerous rock and generating the motion trend;

[0075] S7. According to the three-dimensional model, obtain the typical point displacement change field through the distance change of the feature points;

[0076] S8. According to the typical point displacement change field, obtain the boundary of the dangerous rock mass through the displacement change amount;

[0077] S9. According to the three-dimensional model and the boundary of the dangerous rock mass, calculate and evaluate the volume, boundary surface and inclination direction of the dangerous rock mass, obtaining the displacement, inclination degree of the dangerous rock and generating the motion trend;

[0078] S10. According to the displacement, inclination degree and motion trend of the dangerous rock, evaluate the dangerous rock mass, generate an evaluation model. If the evaluation model reaches the danger threshold, send out the cruise drone to fly close to the dangerous part to identify the internal features of the dangerous part;

[0079] S11. According to the internal features of the dangerous part, through more detailed and accurate research and judgment, obtain the warning level of the dangerous part. When the warning level reaches the orange warning or red warning, the on-site personnel need to be evacuated.

[0080] The technical solution of the present invention has at least the following beneficial effects compared with the prior art:

[0081] On the one hand, the above solution solves the problems of cumbersome procedures, low efficiency, and high danger in the on-site layout of monitoring equipment through a cruise drone, an RTK positioning module, a visual sensing device, an aerial target layout device, and a real-time modeling workstation in a non-contact pure visual perception manner. On the other hand, the RTK positioning module combines with a public reference station to real-time locate the position of the drone, and then corrects the image data according to the aerial disk target, solving the problems of poor accuracy and susceptibility to interference of the GNSS positioning technology. Finally, the displacement and inclination of the dangerous rock are calculated through the real-time three-dimensional model, and the motion trend and evaluation model are generated, and a multi-index comprehensive weight method is used for hierarchical early warning. The invention uses the drone as the main device, avoiding repeated investment in hardware, and is suitable for large-scale popularization and application in the dangerous rock area on the water-facing side of the reservoir bank where people cannot enter. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required in the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0083] Figure 1 It is a schematic diagram of the system architecture of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0084] Figure 2 It is a system block diagram of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0085] Figure 3 It is a top view of the target layout scheme of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0086] Figure 4 It is a side view of the target layout scheme of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0087] Figure 5 It is a schematic diagram of the aerial target layout device of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0088] Figure 6 It is a schematic diagram of the conical leg of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0089] Figure 7 It is a flowchart of generating a three-dimensional model of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0090] Figure 8 It is a flowchart for calculating the displacement and inclination of dangerous rocks, generating the motion state, evaluating and warning the dangerous rock mass in an embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention;

[0091] Figure 9 It is a flowchart for obtaining the danger level through the evaluation model of the dangerous rock mass and warning the dangerous rock mass in an embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention;

[0092] Figure 10 It is a flowchart of an embodiment of the air-space-ground dangerous rock collapse monitoring method based on pure vision perception of the present invention.

[0093] Explanation of reference numerals in the figure: Cruise UAV 1, RTK positioning module 2, vision sensing device 3, air-drop target layout device 4, real-time modeling workstation 5, remote controller 6, carrier UAV 7, disc target 701, triangular prism-shaped iron block 702, conical leg 703, silica gel film 704, thorn needle 705, bolt 706, suspension rope 707, magnet 708. Detailed implementation manners

[0094] The technical solutions in the present invention will be described below with reference to the accompanying drawings.

[0095] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two can be selected.

[0096] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0097] As Figure 1 shown in the schematic diagram of the system architecture of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention and as Figure 2 shown in the system block diagram of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention, the present invention provides an air-space-ground dangerous rock collapse monitoring system based on pure vision perception, which can implement an air-space-ground dangerous rock collapse monitoring method based on pure vision perception. The system includes: a cruise UAV 1, an RTK (Real-Time Kinematic) positioning module 2, a vision sensing device 3, an air-drop target layout device 4, and a real-time modeling workstation 5;

[0098] Cruise UAV 1, used to perform autonomous flight missions and cover the monitoring area;

[0099] Specifically, the cruise UAV 1 is provided with a fixed frame, which can mount the RTK positioning module 2 and the vision sensing device 3. The RTK positioning module 2 is detachably arranged above the cruise UAV 1, and the vision sensing device 3 is detachably arranged below the cruise UAV 1;

[0100] The cruise UAV 1 reserves a data interaction interface for receiving data information from the RTK positioning module 2 and the vision sensing device 3;

[0101] The cruise UAV 1 includes a supporting remote controller 6. The remote controller 6 can set the flight range and route of the cruise UAV 1; the remote controller 6 can display information such as the flight position and images in real time;

[0102] The cruise UAV 1 is equipped with a wireless communication module, which can send data to the remote controller 6 and can also send data to a real-time modeling workstation 5 several kilometers away.

[0103] RTK positioning module 2, used for the cruise UAV 1 to perform high-precision positioning during flight to ensure obtaining accurate position information of the cruise UAV 1;

[0104] Specifically, the RTK positioning module 2 adopts multi-frequency anti-interference technology and multi-step adaptive filtering technology;

[0105] The RTK positioning module 2, according to the relative positioning principle, uses a common reference station and satellites to calculate the three-dimensional coordinates of the cruise UAV 1 in real time, which is used to assist the image data calculation process and improve the calculation speed and accuracy. The positioning refresh frequency of the RTK positioning module 2 is greater than 20 Hz, the time error is less than 20 ns, the baud rate is 115200 bps, the coordinate error is less than 5 mm, and it has 1408 channels and supports the new frequency points B1C, B2a, and B2b of the Beidou-3 satellite for calculation.

[0106] Vision sensing device 3, used to perform multi-directional photography on dangerous rock masses to obtain image data of multiple angles of dangerous rock masses;

[0107] Specifically, the vision sensing device 3 includes a photography sensor for photographing dangerous rock masses. During the flight of the cruise UAV 1, image data of multiple points and multiple directions can be photographed through the vision sensing device 3.

[0108] Air-drop target layout device 4, used to accurately layout targets in the monitoring area to correct image data and improve visual positioning and modeling accuracy;

[0109] Specifically, as Figure 5Schematic diagram of the aerial target deployment device of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention as shown. The aerial target deployment device 4 includes:

[0110] A carrier drone 7, which is used to carry and drop the target to a predetermined position in the monitoring area. The predetermined position in the monitoring area can be expressed as, for example, Figure 3 The top view of the target layout plan of the air-space-ground dangerous rock collapse monitoring system embodiment based on pure vision perception of the present invention as shown and the side view of the target layout plan of the air-space-ground dangerous rock collapse monitoring system embodiment based on pure vision perception of the present invention as shown. Figure 4

[0111] Furthermore, a camera is arranged below the carrier drone 7. The camera can display the position of the drop-type bracket in real time. After the carrier drone 7 transports the drop-type bracket to the designated position, it slowly descends. After the drop-type bracket touches the ground, the magnetic attraction device is powered off to release the drop-type bracket.

[0112] A disc target 701, which is used to provide a visual reference point with high contrast.

[0113] The disc target 701 is a stainless steel disc with a diameter of 100 mm, and its surface is sprayed red, having a good reflective effect.

[0114] A drop-type bracket, which is used to stabilize and release the disc target.

[0115] The drop-type bracket includes:

[0116] Conical legs 703, which are used to provide stable support and positioning for the drop-type bracket.

[0117] Furthermore, as shown in the schematic diagram of the conical leg of the air-space-ground dangerous rock collapse monitoring system embodiment based on pure vision perception of the present invention, the conical leg 703 includes: Figure 6

[0118] A silica gel film 704, which is used to hold glue. The silica gel film 704 breaks after landing, allowing the glue to flow out, increasing the stability of the drop-type bracket.

[0119] A thorn needle 705, which is used to pierce the silica gel film and enhance the attachment force of the drop-type bracket to the ground, preventing the drop-type bracket from moving or tilting after deployment.

[0120] ​​The conical legs 703 are used to stabilize the drop-type bracket and release the internal glue; the conical legs 703 are composed of a cylindrical part at the upper part and a conical part at the lower part. Inside the conical legs 703 are two cylindrical cavity structures, the upper cylindrical cavity structure is used to store the silicone film 704 and the internal glue, and the lower cylindrical cavity structure serves as the slideway for the thorn needle 705. There are 4 glue outlet holes arranged around the upper cylindrical cavity structure;

[0121] The silicone film 704 is about 1 mm thick. After pouring the glue, tie the bag mouth and put it into the upper cylindrical cavity. When the conical legs 703 touch the ground, the internal silicone film 704 is punctured by the thorn needle 705, releasing the internal glue into the upper cylindrical cavity, and then flowing out to the outside through the glue outlet holes on the upper cylindrical cavity;

[0122] The setting time of the glue is greater than 0.5 hour and less than 3 hours, ensuring that it does not solidify before being dropped to the designated position, and its fluidity begins to decrease during the gradual outflow after being dropped, so that the peripheral area of the conical legs 703 can fully contact the glue;

[0123] The thorn needle 705 is a cylindrical steel needle, with a solid glue arranged in the middle. The solid glue is blocked in the lower cylindrical cavity to prevent the thorn needle 705 from falling during transportation;

[0124] The triangular prism-shaped iron block 702 is used to increase the weight of the drop-type bracket, ensuring that the drop-type bracket lands stably after being dropped;

[0125] The triangular prism-shaped iron block 702 is the counterweight iron for the disc target 701, with a weight of about 5 kg. There are 3 screw holes drilled at the three lower corners of the triangular prism-shaped iron block 702 for fixing the conical legs 703, and a bolt 706 is fixed on the top;

[0126] The bolt 706 is used to fix the disc target 701;

[0127] The lifting rope 707 is used to assist the carrying drone 7 to carry the drop-type bracket and the disc target 701;

[0128] The magnet 708 is used to assist the temporary adsorption of the carrying drone 7 and the drop-type bracket, ensuring the accurate release of the drop-type bracket;

[0129] Furthermore, the magnetic suction force of the magnet is greater than 10 kg;

[0130] Specifically, the aerial target deployment device 4 is dropped within and outside the deformation area of the dangerous rock mass, and the inclination directions of the dropped disc targets 701 cover the four directions of east, west, south, and north.

[0131] The real-time modeling workstation 5 is used to generate a three-dimensional model, calculate the movement attitude, displacement and inclination of the dangerous rock mass, and evaluate and give early warnings about the movement trend of the dangerous rock mass.

[0132] Specifically, the real-time modeling workstation 5 includes:

[0133] A wireless receiver for receiving data from the cruise UAV;

[0134] A PC (Personal Computer) for processing and displaying the received data;

[0135] Point cloud synthesis software for generating a three-dimensional model;

[0136] An image processing device for calculating the displacement and inclination of the dangerous rock, generating the motion trend of the dangerous rock mass, and evaluating and warning the dangerous rock mass.

[0137] Specifically, as Figure 7 shown in the flowchart of generating a three-dimensional model of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention, generating a three-dimensional model includes:

[0138] According to the precise position information of the cruise UAV 1 and the image data of multiple angles of the dangerous rock mass, an image of the disc target 701 is obtained;

[0139] According to the image of the disc target 701, two-dimensional image correction is performed to obtain feature points in multiple image sequences;

[0140] According to the feature points in the multiple image sequences, invalid feature points are removed according to a set threshold to obtain feature points in the effective image sequences;

[0141] According to the feature points in the effective image sequences, through feature point matching and perspective change, a multi-dimensionally associated two-dimensional image is obtained;

[0142] According to the multi-dimensionally associated two-dimensional image, the multi-dimensionally associated two-dimensional image is synthesized by SFM (Structure from Motion) three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model.

[0143] Specifically, as Figure 8 shown is the flowchart of calculating the displacement and inclination of the dangerous rock, generating the motion trend, and evaluating and warning the dangerous rock mass in the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention. The calculation of the displacement and inclination of the dangerous rock, generating the motion trend, and evaluating and warning the dangerous rock mass includes:

[0144] According to the three-dimensional model, the feature points in the three-dimensional model are compared, and the boundary of the dangerous rock mass is obtained through the displacement change amount;

[0145] Based on the three-dimensional model and the boundary of the dangerous rock mass, calculate and evaluate the volume, boundary surface, and inclination direction of the dangerous rock mass to obtain the displacement, inclination degree, and movement trend of the dangerous rock mass;

[0146] Based on the displacement, inclination degree, and movement trend of the dangerous rock mass, obtain an evaluation model of the dangerous rock mass;

[0147] Based on the evaluation model of the dangerous rock mass, obtain the danger level and give an early warning to the said dangerous rock mass.

[0148] Specifically, as Figure 9 is the flowchart of obtaining the danger level and giving an early warning to the said dangerous rock mass through the evaluation model of the dangerous rock mass in the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure vision perception of the present invention shown. Based on the evaluation model of the dangerous rock mass, obtaining the danger level and giving an early warning to the said dangerous rock mass includes:

[0149] Based on the evaluation model of the dangerous rock mass, obtain the velocity of the dangerous rock mass , the acceleration of the dangerous rock mass , the displacement of the dangerous rock mass , the inclination angle of the dangerous rock mass and the shooting time ;

[0150] Based on the velocity of the dangerous rock mass and the shooting time , calculate through formula (1) to obtain the rate of change of velocity between two adjacent times;

[0151] (1)

[0152] In the formula, is the rate of change of velocity, is the change in velocity at time t + 1, is the change in velocity at time t, is the velocity at time t + 1, is the velocity at time t, is the velocity at time t - 1;

[0153] Based on the acceleration of the dangerous rock mass and the shooting time , calculate through formula (2) to obtain the rate of change of acceleration between two adjacent times;

[0154] (2)

[0155] In the formula, is the rate of change of acceleration, is the change in acceleration at time t + 1, is The change in acceleration at a moment is the acceleration at time t + 1 is the acceleration at a moment is the acceleration at time t - 1;

[0156] According to the displacement of the dangerous rock mass and the shooting time , calculate through formula (3) to obtain the displacement change rate between two adjacent times;

[0157] (3)

[0158] In the formula, is the displacement change rate, is the displacement change at time t + 1, is the displacement change at a moment is the displacement at time t + 1, is the displacement at time t, is the displacement at time t - 1;

[0159] According to the inclination angle of the dangerous rock mass and the shooting time , calculate through formula (4) to obtain the inclination angle change rate between two adjacent times;

[0160] (4)

[0161] In the formula, is the inclination angle change rate, is the inclination angle change at time t + 1, is the inclination angle change at a moment is the inclination angle at time t + 1, is the inclination angle at a moment is the inclination angle at time t - 1;

[0162] According to the evaluation model of the dangerous rock mass, obtain the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement, and the weight factor of inclination angle. The relationship between the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement, and the weight factor of inclination angle conforms to formula (5);

[0163] (5)

[0164] In the formula, is the weight factor of velocity, is the weight factor of acceleration, is the weight factor of displacement, is the weight factor of the inclination angle;

[0165] According to the velocity change rates of two adjacent times, the acceleration change rates of two adjacent times, the displacement change rates of two adjacent times, the inclination angle change rates of two adjacent times, the weight factor of the velocity, the weight factor of the acceleration, the weight factor of the displacement, and the weight factor of the inclination angle, the comprehensive weight is obtained through formula (6). ;

[0166] (6)

[0167] According to the comprehensive weight and the preset warning threshold are compared. According to the warning rules, the danger level is obtained and a warning is issued for the dangerous rock mass. The warning rules are as follows:

[0168] Rule 1: When the warning level is level IV, which is a blue warning;

[0169] Rule 2: When the warning level is level III, which is a yellow warning;

[0170] Rule 3: When the warning level is level II, which is an orange warning;

[0171] Rule 4: When the warning level is level I, which is a red warning;

[0172] When the warning level reaches orange warning or red warning, the on-site personnel need to evacuate.

[0173] As Figure 10 shown in the flowchart of the embodiment of the method for monitoring dangerous rock collapse based on pure vision perception of the present invention, the present invention provides a method for monitoring dangerous rock collapse based on pure vision perception. This method is implemented by a system for monitoring dangerous rock collapse based on pure vision perception. This method includes:

[0174] S1. Install the RTK positioning module 2 and the vision sensing device 3 on the cruise unmanned aerial vehicle 1. By the way of the initial flight of the cruise unmanned aerial vehicle 1, the regional range of the dangerous rock mass is obtained and a target point layout plan is designed;

[0175] S2. Place the lancet 705 into the slideway of the conical leg 703, place the silica gel membrane 704 into the upper cavity of the conical leg 703, prepare the glue, pour it into the silica gel membrane 704 and tie it up. Then screw the 3 conical legs 703 into the screw holes at the three corners of the triangular prism-shaped iron block 702. Finally, connect one end of the 3 suspension ropes 707 to the three corners of the triangular prism-shaped iron block 702 and the other end to the magnet 708 to obtain the drop-type bracket.

[0176] S3. According to the target layout plan, use the transport drone 7 to drop the disc target 701 and the drop-type bracket to the pre-designed target position to obtain the area of the dangerous rock mass with the disc target 701.

[0177] S4. According to the area of the dangerous rock mass with the disc target 701, set the flight area and route of the cruise drone 1, and start the cruise drone 1 to photograph the area of the dangerous rock mass with the disc target 701 in the automatic cruise state to obtain the accurate position information of the cruise drone 1 and the image data of the dangerous rock mass from multiple angles, and transmit the accurate position information of the cruise drone 1 and the image data of the dangerous rock mass from multiple angles to the real-time modeling workstation 5.

[0178] S5. According to the image data of the dangerous rock mass from multiple angles, use the disc target 701 to correct the image data of the dangerous rock mass from multiple angles to obtain the corrected image data.

[0179] S6. According to the accurate position information of the cruise drone 1 and the corrected image data, synthesize the corrected image data through the SFM three-dimensional reconstruction technology and the point cloud synthesis software to obtain a three-dimensional model.

[0180] S7. According to the three-dimensional model, obtain the typical point displacement change field through the distance change of the feature points.

[0181] S8. According to the typical point displacement change field, obtain the boundary of the dangerous rock mass through the displacement change amount.

[0182] S9. According to the three-dimensional model and the boundary of the dangerous rock mass, calculate and evaluate the volume, boundary surface and inclination direction of the dangerous rock mass to obtain the displacement, inclination degree of the dangerous rock and generate the motion trend.

[0183] S10. According to the displacement, inclination degree and motion trend of the dangerous rock, evaluate the dangerous rock mass to generate an evaluation model. If the evaluation model reaches the danger threshold, send out the cruise drone 1 to fly close to the dangerous part to identify the internal characteristics of the dangerous part.

[0184] S11. Based on the internal characteristics of the dangerous part, through more detailed and accurate judgment, obtain the warning level of the dangerous part. When the warning level reaches the orange warning or red warning, the on-site personnel need to be evacuated.

[0185] The present invention provides an air-space-ground dangerous rock collapse monitoring system and method based on pure vision perception. The invention solves the problems of high cost, low efficiency and great danger in on-site deployment of monitoring devices through a cruise unmanned aerial vehicle 1, an RTK positioning module 2, a vision sensing device 3, an air-drop target layout device 4 and a real-time modeling workstation 5. On the other hand, the RTK positioning module 2 uses a public reference station and satellite positioning to solve the problems of poor accuracy and susceptibility to interference of the GNSS (Global Navigation Satellite System) positioning technology. Finally, by calculating the movement attitude, displacement and inclination of the dangerous rock through a three-dimensional model and evaluating the dangerous rock mass, the problems of poor accuracy and low timeliness of traditional methods are solved.

[0186] It can be understood that the present invention is described through the above embodiments and should not be construed as a limitation on the implementation mode and scope of the present invention. Those skilled in the art know that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception is characterized by: The system comprises: Cruise drones, used to perform autonomous flight missions and cover surveillance areas; The RTK positioning module is used to perform high-precision positioning of the cruise UAV during flight to obtain accurate position information of the cruise UAV; Visual sensing device, used to take multi-directional photographs of dangerous rock bodies and obtain image data of dangerous rock bodies at multiple angles; An airdrop target placement device is used to accurately place targets in the monitoring area, correct image data, and improve visual positioning and modeling accuracy; A real-time modeling workstation is used to generate a three-dimensional model, calculate the displacement and inclination of dangerous rocks, generate movement trends, and evaluate and warn of the dangerous rock mass; The calculation of the displacement and inclination of the dangerous rock and the generation of the movement situation, and the evaluation and early warning of the dangerous rock body include: According to the three-dimensional model, the characteristic points in the three-dimensional model are compared to obtain the displacement of the dangerous rock mass. , and obtain the boundary of the dangerous rock mass through the displacement variation; Calculating and evaluating the volume, boundary surface and inclination direction of the dangerous rock mass according to the three-dimensional model and the boundary of the dangerous rock mass to obtain the movement status of the dangerous rock mass; According to the movement situation of the dangerous rock mass, the danger level is obtained and an early warning is given to the dangerous rock mass, including: According to different shooting time The movement state of the dangerous rock mass is obtained by , acceleration of dangerous rock mass , the inclination angle of the dangerous rock mass ; According to the speed of the dangerous rock mass and shooting time , calculated by formula (1), the velocity change rate of two adjacent times is obtained; （1） In the formula, is the speed change rate, is the velocity change at time t+1, is the velocity change at time t, is the speed at time t+1, is the speed at time t, is the speed at time t-1; According to the acceleration of the dangerous rock mass and shooting time , calculated by formula (2), the acceleration change rate between two adjacent times is obtained; （2） In the formula, is the rate of change of acceleration, is the acceleration change at time t+1, is the acceleration change at time t, is the acceleration at time t+1, is the acceleration at time t, is the acceleration at time t-1; According to the displacement of the dangerous rock mass and shooting time , calculated by formula (3), the displacement change rate between two adjacent times is obtained; （3） In the formula, is the displacement change rate, is the displacement change at time t+1, is the displacement change at time t, is the displacement at time t+1, is the displacement at time t, is the displacement at time t-1; According to the inclination angle of the dangerous rock mass and shooting time , calculated by formula (4), the rate of change of the tilt angle between two adjacent times is obtained; （4） In the formula, is the rate of change of the tilt angle, is the change in the tilt angle at time t+1, is the change in the tilt angle at time t, is the tilt angle at time t+1, is the tilt angle at time t, is the inclination angle at time t-1; The relationship between the weight factor of the velocity, the weight factor of the acceleration, the weight factor of the displacement and the weight factor of the tilt angle conforms to formula (5); （5） In the formula, is the weight factor of speed, is the weight factor of acceleration, is the weight factor of the displacement, is the weight factor of the tilt angle; According to the velocity change rate of two adjacent times, the acceleration change rate of two adjacent times, the displacement change rate of two adjacent times, the inclination angle change rate of two adjacent times, the velocity weight factor, the acceleration weight factor, the displacement weight factor and the inclination angle weight factor, the comprehensive weight is obtained by formula (6): ; （6） According to the comprehensive weight and pre-set warning thresholds Compare and obtain the danger level according to the early warning rules and issue an early warning for the dangerous rock mass. The early warning rules are as follows: Rule 1: When The warning level is level IV, which is a blue warning; Rule 2: When The warning level is level III, which is a yellow warning; Rule 3: When The warning level is Level II, which is an orange warning; Rule 4: When The warning level is level Ⅰ, which is a red warning; When the warning level reaches the orange warning or the red warning, on-site personnel need to evacuate.

2. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The RTK positioning module adopts multi-frequency anti-interference technology and multi-step adaptive filtering technology; The RTK positioning module uses public reference stations and satellites based on the principle of relative positioning to calculate the three-dimensional coordinates of the cruise drone in real time, which is used to assist the image data calculation process and improve the calculation speed and accuracy.

3. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The airdrop target deployment device comprises: A carrier drone, used to carry and drop the target to a predetermined location in the monitoring area; Disc targets to provide high-contrast visual reference points; A drop-type bracket is used for stabilizing and releasing the disc target.

4. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 3 is characterized in that: The drop-type bracket comprises: Conical legs for providing stable support and positioning of the drop-type stand; The triangular prism-shaped iron block is used to increase the weight of the drop-type bracket to ensure that the drop-type bracket is firmly grounded after being dropped; Bolts, used to fix the disc target; A sling, used to assist the carrier drone in carrying the drop-type bracket and the disc target; A magnet, used to assist the temporary adsorption of the carrier drone and the drop-type bracket to ensure accurate release of the drop-type bracket; The tapered leg comprises: A silicone film is used to contain glue, and the silicone film breaks after falling to the ground, allowing the glue to flow out, thereby increasing the stability of the drop-type bracket; The puncture needle is used to puncture the silicone membrane and enhance the grip of the drop-type bracket on the ground to prevent the drop-type bracket from moving or tilting after being deployed.

5. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 3 is characterized in that: The airdrop target placement device is placed inside and outside the deformation area of ​​the dangerous rock mass, and the tilt directions of the placed disc targets cover four directions: east, west, south, and north.

6. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The real-time modeling workstation comprises: A wireless receiver, used to receive data from the cruise drone; PC side, used to process and display the received data; Point cloud synthesis software for generating 3D models; The image processing device is used to calculate the displacement and inclination of dangerous rocks and generate the movement status of dangerous rock bodies, and to evaluate and warn the dangerous rock bodies.

7. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 6 is characterized in that: The generating of the three-dimensional model comprises: Obtaining an image of the disc target according to the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles; Performing two-dimensional image correction according to the image of the disk target to obtain feature points in a plurality of image sequences; According to the feature points in the plurality of image sequences, invalid feature points are eliminated according to a set threshold value to obtain feature points in a valid image sequence; According to the feature points in the effective image sequence, a two-dimensional image with multi-dimensional association is obtained through feature point matching and perspective change; According to the multi-dimensionally associated two-dimensional images, the multi-dimensionally associated two-dimensional images are synthesized by using the SFM three-dimensional reconstruction technology and the point cloud synthesis software to obtain a three-dimensional model.

8. An air-air-ground dangerous rock collapse monitoring method based on pure visual perception, the air-air-ground dangerous rock collapse monitoring method based on pure visual perception is implemented by the air-air-ground dangerous rock collapse monitoring system based on pure visual perception according to any one of claims 1 to 7, characterized in that: The method comprises: S1. Installing an RTK positioning module and a visual sensor device on a cruise drone, obtaining the area of ​​the dangerous rock mass and designing a target layout plan through a preliminary flight of the cruise drone; S2, placing the needle into the slideway of the conical leg, placing the silicone membrane into the upper cavity of the conical leg, preparing glue, pouring it into the silicone membrane, and tying it, then screwing the three conical legs into the screw holes at the three corners of the triangular prism-shaped iron block, and finally connecting one end of the three hanging ropes to the three corners of the triangular prism-shaped iron block and the other end to the magnet to obtain a drop-type bracket; S3, according to the target arrangement plan, the disc target and the delivery bracket are delivered to the pre-designed target position by a carrier drone to obtain the area of ​​the dangerous rock mass with the disc target; S4, according to the area of ​​the dangerous rock body with the disc target, set the flight area and route of the cruise drone, start the cruise drone to take photos of the area of ​​the dangerous rock body with the disc target in an automatic cruise state, obtain the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles, and transmit the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles to the real-time modeling workstation; S5, according to the image data of the dangerous rock body at multiple angles, using the disk target to correct the image data of the dangerous rock body at multiple angles to obtain corrected image data; S6. According to the precise position information of the cruise drone and the corrected image data, synthesize the corrected image data by using SFM three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model; S7, obtaining a typical point displacement change field through a distance change of feature points according to the three-dimensional model; S8. According to the displacement change field of the typical points, the boundary of the dangerous rock mass is obtained through the displacement change amount; S9, calculating and evaluating the volume, boundary surface and inclination direction of the dangerous rock body according to the three-dimensional model and the boundary of the dangerous rock body, and obtaining the movement state of the dangerous rock body; S10, evaluating the dangerous rock body according to the movement situation of the dangerous rock body to obtain a danger level, and when the danger level reaches a danger threshold, dispatching the cruise drone to fly close to the dangerous part to identify the internal features of the dangerous part; S11. According to the internal characteristics of the dangerous area, through more detailed and accurate analysis, the warning level of the dangerous area is obtained. When the warning level reaches orange warning or red warning, on-site personnel need to be evacuated.

Images