Air-sky-ground dangerous rock collapse monitoring system and method based on pure visual perception

Abstract

The invention provides an air-sky-ground dangerous rock collapse monitoring system and method based on pure visual perception, and belongs to the technical field of dangerous rock collapse early warning. The problems of tedious working procedures, low efficiency, high risk and the like of on-site monitoring equipment arrangement are solved in a non-contact pure visual perception mode, meanwhile, the position of the unmanned aerial vehicle is positioned in real time through an RTK positioning module in combination with a public base station, image data are corrected according to an air-dropped disc target, and the accuracy of the unmanned aerial vehicle is improved. The method solves the problems of poor precision, easy interference and the like based on a GNSS positioning technology, finally solves the displacement and inclination of the dangerous rock through a real-time three-dimensional model, generates a motion situation and an evaluation model, and carries out graded early warning through a multi-index comprehensive weight method. The method is suitable for large-scale popularization and application in reservoir bank upstream face dangerous rock areas where people cannot enter.

Description

Technical Field

[0001] The present invention relates to the technical field of dangerous rock collapse early warning, and in particular to an air-space-ground dangerous rock collapse monitoring system and method based on pure visual perception. Background Art

[0002] The geological environment of the Three Gorges Reservoir is complex and fragile, with frequent water storage and precipitation scheduling, coupled with the influence of extreme weather such as rainstorms and floods and human activities, it has always been a high-incidence and frequent geological disaster area. Today, with the increasing degradation of the rock mass in the drawdown zone, the problem of high-level dangerous rocks has become increasingly prominent, especially the hydraulic erosion caused by the rise and fall of the water level, which has formed a drawdown zone (water level change zone) as high as 30 m on both sides of the reservoir, forming a local high and steep inverted rock mass, which poses great risks and hidden dangers. According to the investigation, there are many hidden dangers of dangerous rock collapse in the Three Gorges Reservoir area. Once the dangerous rock cracks and becomes unstable and collapses into the river, it will pose a serious threat to shipping and the safety of the reservoir area. Monitoring and early warning of the destruction evolution process of dangerous rock masses is one of the effective means to mitigate and prevent disasters, but the early warning system relies on effective monitoring equipment and technical solutions.

[0003] However, due to the water-facing surface of the reservoir slope, the working environment and equipment placement space are limited, making it difficult to install monitoring equipment on site. The process of manually climbing and deploying monitoring devices in the field is also risky, making conventional contact monitoring solutions often difficult to apply. At the same time, the traditional on-site deployment of monitoring devices is costly, inefficient, and dangerous. The GNSS (Global Navigation Satellite System) positioning technology is easily affected by weather, resulting in poor accuracy, low timeliness, and susceptibility to interference. Summary of the invention

[0004] In order to solve the problems in the above-mentioned prior art, the present invention provides an air-space-ground dangerous rock collapse monitoring system and method based on pure visual perception. The invention solves the problems of high cost, low efficiency and high danger of on-site deployment of monitoring devices through cruise drones, RTK (Real Time Kinematic, real-time dynamic measurement) positioning modules, visual sensing devices, airdrop target deployment devices and real-time modeling workstations. In addition, the RTK positioning module uses public reference stations and satellite positioning to solve the problems of poor accuracy and susceptibility to interference based on GNSS positioning technology. Finally, the three-dimensional model is used to calculate the movement posture, displacement and inclination of the dangerous rock and evaluate the dangerous rock body, solving the problems of poor accuracy and low timeliness of traditional methods. To achieve the above purpose, the technical solution is as follows:

[0005] On the one hand, the present invention provides an air-space-ground dangerous rock collapse monitoring system based on pure visual perception, the system comprising:

[0006] Cruise drones, used to perform autonomous flight missions and cover surveillance areas;

[0007] RTK positioning module, used for high-precision positioning of the cruise UAV during flight to obtain the precise location information of the cruise UAV;

[0008] Visual sensing device, used to take multi-directional photographs of dangerous rock bodies and obtain image data of dangerous rock bodies at multiple angles;

[0009] Airdrop target placement device, used to accurately place targets in the monitoring area, correct image data, and improve visual positioning and modeling accuracy;

[0010] The real-time modeling workstation is used to generate three-dimensional models, calculate the displacement and inclination of dangerous rocks, generate movement trends, and evaluate and warn the movement trends of the dangerous rock mass.

[0011] Optionally, the RTK positioning module adopts multi-frequency anti-interference technology and multi-step adaptive filtering technology;

[0012] Based on the principle of relative positioning, the RTK positioning module uses public base stations and satellites to calculate the three-dimensional coordinates of the cruise drone in real time, which is used to assist the image data solution process and improve the solution speed and accuracy.

[0013] Optionally, the airdrop target deployment device comprises:

[0014] A carrier drone is used to carry and drop targets to predetermined locations in the surveillance area;

[0015] Disc targets to provide high-contrast visual reference points;

[0016] Drop-type stand for stabilizing and releasing disc targets.

[0017] Optionally, the drop-type bracket comprises:

[0018] Conical legs for providing stable support and positioning of the drop-type stand;

[0019] The triangular prism-shaped iron block is used to increase the weight of the drop-type bracket to ensure that the drop-type bracket is firmly grounded after being dropped;

[0020] A bolt, used for fixing the disc target;

[0021] A sling, used to assist the carrier drone in carrying the drop-type bracket and the disc target;

[0022] A magnet is used to assist the temporary adsorption of the carrier drone and the drop-type bracket to ensure accurate release of the drop-type bracket;

[0023] The tapered legs include:

[0024] A silicone film is used to contain glue. The silicone film breaks after falling to the ground, allowing the glue to flow out, thereby increasing the stability of the drop-type bracket;

[0025] The puncture needle is used to puncture the silicone film and enhance the gripping force of the drop-type bracket on the ground to prevent the drop-type bracket from moving or tilting after being deployed.

[0026] Optionally, the airdrop target placement device is placed inside and outside the deformation area of ​​the dangerous rock body and the tilt directions of the placed disc targets cover four directions of east, west, south and north.

[0027] Optionally, the real-time modeling workstation comprises:

[0028] A wireless receiver, used for receiving data from the cruising drone;

[0029] PC side, used to process and display the received data;

[0030] Point cloud synthesis software for generating 3D models;

[0031] The image processing device is used to calculate the displacement and inclination of dangerous rocks and generate the movement status of dangerous rock bodies, and to evaluate and warn the dangerous rock bodies.

[0032] Optionally, generating a three-dimensional model includes:

[0033] According to the precise position information of the cruise UAV and the image data of the dangerous rock body at multiple angles, an image of the disc target is obtained;

[0034] According to the image of the disk target, two-dimensional image correction is performed to obtain feature points in a plurality of image sequences;

[0035] According to the feature points in the plurality of image sequences, invalid feature points are eliminated according to a set threshold value to obtain feature points in a valid image sequence;

[0036] According to the feature points in the effective image sequence, a two-dimensional image with multi-dimensional association is obtained through feature point matching and perspective change;

[0037] According to the multi-dimensionally associated two-dimensional images, the multi-dimensionally associated two-dimensional images are synthesized by using SFM (Structure from Motion, structured optical flow method) three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model.

[0038] Optionally, the calculation of the movement posture, displacement and inclination of the dangerous rock and the evaluation and early warning of the dangerous rock body include:

[0039] According to the three-dimensional model, characteristic points in the three-dimensional model are compared, and the boundary of the dangerous rock mass is obtained through displacement variation;

[0040] According to the three-dimensional model and the boundary of the dangerous rock mass, the volume, boundary surface and inclination direction of the dangerous rock mass are calculated and evaluated to obtain the displacement, inclination and movement state of the dangerous rock mass;

[0041] According to the displacement, inclination and movement status of the dangerous rock mass, an assessment model of the dangerous rock mass is obtained;

[0042] According to the evaluation model of the dangerous rock mass, the danger level is obtained and an early warning is issued for the dangerous rock mass.

[0043] Optionally, obtaining a danger level and issuing an early warning for the dangerous rock mass according to the evaluation model of the dangerous rock mass includes:

[0044] According to the evaluation model of the dangerous rock mass, the velocity of the dangerous rock mass is obtained. , acceleration of dangerous rock mass , displacement of dangerous rock mass , the inclination angle of the dangerous rock mass and shooting time ;

[0045] According to the speed of the dangerous rock and shooting time , calculated by formula (1), the velocity change rate of two adjacent times is obtained;

[0046] (1)

[0047] In the formula, is the speed change rate, is the velocity change at time t+1, is the velocity change at time t, is the speed at time t+1, is the speed at time t, is the speed at time t-1;

[0048] According to the acceleration of the dangerous rock mass and shooting time , calculated by formula (2), the acceleration change rate between two adjacent times is obtained;

[0049] (2)

[0050] In the formula, is the rate of change of acceleration, is the acceleration change at time t+1, for The acceleration change at time, is the acceleration at time t+1, for The acceleration of time, is the acceleration at time t-1;

[0051] According to the displacement of the dangerous rock mass and shooting time , calculated by formula (3), the displacement change rate of two adjacent times is obtained;

[0052] (3)

[0053] In the formula, is the displacement change rate, is the displacement change at time t+1, for The displacement change at time, is the displacement at time t+1, is the displacement at time t, is the displacement at time t-1;

[0054] According to the inclination angle of the dangerous rock mass and shooting time , calculated by formula (4), the rate of change of the tilt angle between two adjacent times is obtained;

[0055] (4)

[0056] In the formula, is the rate of change of the tilt angle, is the change in the tilt angle at time t+1, for The change in tilt angle at time, is the tilt angle at time t+1, for The tilt angle at the moment, is the inclination angle at time t-1;

[0057] According to the evaluation model of the dangerous rock mass, the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle are obtained, and the relationship among the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle conforms to formula (5);

[0058] (5)

[0059] In the formula, is the weight factor of speed, is the weight factor of acceleration, is the weight factor of the displacement, is the weight factor of the tilt angle;

[0060] According to the velocity change rate of the two adjacent times, the acceleration change rate of the two adjacent times, the displacement change rate of the two adjacent times, the tilt angle change rate of the two adjacent times, the weight factor of the velocity, the weight factor of the acceleration, the weight factor of the displacement and the weight factor of the tilt angle, the comprehensive weight is obtained by formula (6): ;

[0061] (6)

[0062] According to the comprehensive weight and pre-set warning thresholds Compare and obtain the danger level according to the early warning rules and issue an early warning for the dangerous rock mass. The early warning rules are as follows:

[0063] Rule 1: When The warning level is level IV, which is a blue warning;

[0064] Rule 2: When The warning level is level III, which is a yellow warning;

[0065] Rule 3: When The warning level is Level II, which is an orange warning;

[0066] Rule 4: When The warning level is level Ⅰ, which is a red warning;

[0067] When the warning level reaches orange or red, on-site personnel need to evacuate.

[0068] On the other hand, the present invention provides an air-space-ground dangerous rock collapse monitoring method based on pure visual perception, which is implemented by an air-space-ground dangerous rock collapse monitoring system based on pure visual perception, and the method includes:

[0069] S1. Install the RTK positioning module and the visual sensor device on the cruise drone, obtain the area range of the dangerous rock mass and design the target layout plan through the preliminary flight of the cruise drone;

[0070] S2, put the needle into the slide of the conical leg, put the silicone membrane into the upper cavity of the conical leg, prepare glue, pour it into the silicone membrane, and tie it, then screw the three conical legs into the screw holes at the three corners of the triangular prism iron block, and finally connect one end of the three hanging ropes to the three corners of the triangular prism iron block and the other end to the magnet to obtain a drop-type bracket;

[0071] S3. According to the target point arrangement plan, the disc target and the delivery bracket are delivered to the pre-designed target point position by a carrier drone to obtain the area of ​​the dangerous rock mass with the disc target;

[0072] S4, according to the area of ​​the dangerous rock body with the disc target, set the flight area and route of the cruise drone, start the cruise drone to take photos of the area of ​​the dangerous rock body with the disc target in an automatic cruise state, obtain the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles, and transmit the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles to the real-time modeling workstation;

[0073] S5, according to the image data of the dangerous rock body at multiple angles, using the disk target to correct the image data of the dangerous rock body at multiple angles to obtain corrected image data;

[0074] S6. According to the precise position information of the cruise drone and the corrected image data, the corrected image data is synthesized by using SFM three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model;

[0075] S7, according to the three-dimensional model, by changing the distance of the feature points, obtaining a typical point displacement change field;

[0076] S8. According to the displacement change field of the typical point, the boundary of the dangerous rock mass is obtained through the displacement change amount;

[0077] S9, calculating and evaluating the volume, boundary surface and inclination direction of the dangerous rock mass according to the three-dimensional model and the boundary of the dangerous rock mass, obtaining the displacement and inclination of the dangerous rock mass and generating a movement situation;

[0078] S10, evaluating the dangerous rock mass according to the displacement, inclination and movement state of the dangerous rock, generating an evaluation model, and if the evaluation model reaches the danger threshold, dispatching the cruise drone to fly close to the dangerous part to identify the internal features of the dangerous part,

[0079] S11. Based on the internal characteristics of the dangerous area, a more detailed and accurate assessment is made to determine the warning level of the dangerous area. When the warning level reaches orange or red warning, on-site personnel must be evacuated.

[0080] Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:

[0081] On the one hand, the above scheme uses cruise drones, RTK positioning modules, visual sensing devices, airdrop target deployment devices and real-time modeling workstations to solve the problems of complicated procedures, low efficiency and high risk of on-site deployment of monitoring equipment in a non-contact pure visual perception manner. On the other hand, the RTK positioning module is combined with a public base station to locate the drone's position in real time, and then the image data is corrected according to the airdropped disc target, which solves the problems of poor accuracy and susceptibility to interference based on GNSS positioning technology. Finally, the real-time three-dimensional model is used to calculate the displacement and inclination of dangerous rocks and generate motion status and evaluation models, and a multi-indicator comprehensive weighting method is used for graded early warning. The invention uses drones as the main equipment, avoids repeated investment in hardware, and is suitable for large-scale promotion and application in dangerous rock areas on the water side of the reservoir bank where personnel cannot enter. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0083] Figure 1 Schematic diagram of the system architecture of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0084] Figure 2 It is a system block diagram of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0085] Figure 3 It is a top view of a target point arrangement scheme of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0086] Figure 4 It is a side view of a target point arrangement scheme of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0087] Figure 5 It is a schematic diagram of an airdrop target deployment device of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0088] Figure 6 is a schematic diagram of a tapered leg of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0089] Figure 7 It is a flow chart of generating a three-dimensional model of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention;

[0090] Figure 8 It is a flow chart of an embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention, which calculates the displacement and inclination of dangerous rocks and generates movement trends, and evaluates and warns the dangerous rock bodies;

[0091] Fig. 9 It is a flow chart of an embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention, which obtains the danger level through an evaluation model of a dangerous rock body and issues an early warning for the dangerous rock body;

[0092] Fig.10 It is a flow chart of an embodiment of the air-space-ground dangerous rock collapse monitoring method based on pure visual perception of the present invention.

[0093] Explanation of the numbers in the figure: cruise drone 1, RTK positioning module 2, visual sensor device 3, airdrop target deployment device 4, real-time modeling workstation 5, remote control 6, carrier drone 7, disc target 701, triangular prism iron block 702, conical leg 703, silicone film 704, needle 705, bolt 706, lifting rope 707, magnet 708. DETAILED DESCRIPTION

[0094] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0095] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0096] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0097] like Figure 1 The system architecture diagram of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is shown in FIG. Figure 2 The system block diagram of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is shown. The present invention provides an air-space-ground dangerous rock collapse monitoring system based on pure visual perception. The system can implement an air-space-ground dangerous rock collapse monitoring method based on pure visual perception. The system includes: a cruise drone 1, an RTK (Real Time Kinematic, real-time dynamic measurement) positioning module 2, a visual sensor device 3, an airdrop target deployment device 4 and a real-time modeling workstation 5;

[0098] Cruise UAV 1, used to perform autonomous flight missions and cover the monitoring area;

[0099] Specifically, the cruise drone 1 is provided with a fixed frame, which can mount the RTK positioning module 2 and the visual sensor device 3, the RTK positioning module 2 is detachably arranged above the cruise drone 1, and the visual sensor device 3 is detachably arranged below the cruise drone 1;

[0100] The cruise drone 1 reserves a data interaction interface for receiving data information from the RTK positioning module 2 and the visual sensor device 3;

[0101] The cruise drone 1 includes a matching remote controller 6, which can set the flight range and route of the cruise drone 1; the remote controller 6 can display the flight position, image and other information in real time;

[0102] The cruise drone 1 is equipped with a wireless communication module, which can send data to a remote controller 6 and a real-time modeling workstation 5 several kilometers away.

[0103] The RTK positioning module 2 is used for high-precision positioning of the cruise drone 1 during flight to ensure accurate position information of the cruise drone 1;

[0104] Specifically, the RTK positioning 2 module uses multi-frequency anti-interference technology and multi-step adaptive filtering technology;

[0105] The RTK positioning module 2 uses the public reference station and satellites based on the relative positioning principle to solve the three-dimensional coordinates of the cruise drone 1 in real time, which is used to assist the image data solution process and improve the solution speed and accuracy. The RTK positioning module 2 has a positioning refresh frequency greater than 20 Hz, a time error less than 20 ns, a baud rate of 115200 bps, a coordinate error less than 5 mm, 1408 channels, and supports the solution of the new frequency points B1C, B2a, and B2b of the Beidou-3 satellite.

[0106] The visual sensor device 3 is used to take multi-directional photographs of the dangerous rock mass to obtain image data of the dangerous rock mass at multiple angles;

[0107] Specifically, the visual sensing device 3 includes a photographic sensor for photographing dangerous rock bodies. The cruise drone 1 can capture image data at multiple points and in multiple directions through the visual sensing device 3 during flight.

[0108] The airdrop target placement device 4 is used to accurately place targets in the monitoring area, correct image data, and improve visual positioning and modeling accuracy;

[0109] Specifically, Figure 5The schematic diagram of the airdrop target point arrangement device of the air-space-ground dangerous rock collapse monitoring system embodiment of the present invention based on pure visual perception is shown, and the airdrop target point arrangement device 4 includes:

[0110] The carrier drone 7 is used to carry and drop the target to the predetermined position of the monitoring area. The predetermined position of the monitoring area can be expressed as follows: Figure 3 The top view of the target point arrangement scheme of the air-sky-ground dangerous rock collapse monitoring system embodiment of the present invention based on pure visual perception is shown in FIG. Figure 4 A side view of a target point arrangement scheme of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is shown;

[0111] Furthermore, a camera is arranged under the carrier drone 7, which can display the position of the drop-type bracket in real time. The carrier drone 7 transports the drop-type bracket to the designated position and then slowly descends. After the drop-type bracket touches the ground, the magnetic suction device is powered off to release the drop-type bracket.

[0112] A disk target 701 for providing a high-contrast visual reference point;

[0113] The disc target 701 is a stainless steel disc with a diameter of 100 mm, and the surface is sprayed red, which has a good reflective effect;

[0114] A drop-type bracket is used for stabilizing and releasing the disc target.

[0115] The drop-type bracket comprises:

[0116] Conical legs 703, used to provide stable support and positioning of the drop-type bracket;

[0117] Furthermore, if Figure 6 The schematic diagram of the conical leg of the embodiment of the air-sky-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is shown, and the conical leg 703 includes:

[0118] The silicone film 704 is used to contain glue. The silicone film 704 breaks after falling to the ground, allowing the glue to flow out, thereby increasing the stability of the drop-type bracket;

[0119] The puncture needle 705 is used to puncture the silicone membrane and enhance the grip of the drop-type bracket on the ground to prevent the drop-type bracket from moving or tilting after deployment.

[0120] The conical leg 703 is used to stabilize the drop-type bracket and release the glue inside. The conical leg 703 is composed of an upper cylinder and a lower cone. The conical leg 703 has two upper and lower cylindrical cavity structures inside. The upper cylindrical cavity structure is used to store the silicone film 704 and the glue inside. The lower cylindrical cavity structure serves as a slideway for the needle 705. Four glue outlet holes are arranged around the upper cylindrical cavity structure.

[0121] The silicone film 704 is about 1 mm thick. After pouring glue into it, the bag mouth is tied and placed into the upper cylindrical cavity. When the conical leg 703 is grounded, the inner silicone film 704 is pierced by the puncture needle 705, releasing the inner glue into the upper cylindrical cavity, and then flowing out from the glue outlet hole on the upper cylindrical cavity;

[0122] The setting time of the glue is greater than 0.5 hours and less than 3 hours, ensuring that it does not set before being placed at the designated position. The fluidity begins to decrease during the gradual outflow process after placement, so that the peripheral area of ​​the tapered leg 703 is fully in contact with the glue;

[0123] The needle 705 is a cylindrical steel needle with a piece of solid glue arranged in the middle. The solid glue is blocked in the lower cylindrical cavity to prevent the needle 705 from falling during transportation.

[0124] The triangular prism iron block 702 is used to increase the weight of the drop-type bracket to ensure that the drop-type bracket is firmly grounded after being dropped;

[0125] The triangular prism iron block 702 is the counterweight iron of the disc target 701, and weighs about 5 kg. Three screw holes are punched at the three corners below the triangular prism iron block 702 to fix the tapered leg 703, and a bolt 706 is fixed on the top;

[0126] Bolt 706, used to fix the disc target 701;

[0127] A sling 707, used to assist the carrier drone 7 in carrying the drop-type bracket and the disc target 701;

[0128] The magnet 708 is used to assist the temporary adsorption of the carrier drone 7 and the drop-type bracket to ensure the accurate release of the drop-type bracket;

[0129] Furthermore, the magnetic attraction force of the magnet is greater than 10 kg;

[0130] Specifically, the airdrop target placement device 4 is placed inside and outside the deformation area of ​​the dangerous rock body, and the tilt directions of the placed disc targets 701 cover four directions of east, west, south and north.

[0131] The real-time modeling workstation 5 is used to generate a three-dimensional model, calculate the movement posture, displacement and inclination of the dangerous rock, and evaluate and warn the movement status of the dangerous rock body.

[0132] Specifically, the real-time modeling workstation 5 comprises:

[0133] A wireless receiver, used to receive data from the cruise drone;

[0134] PC (Personal Computer) side, used to process and display the received data;

[0135] Point cloud synthesis software for generating 3D models;

[0136] The image processing device is used to calculate the displacement and inclination of dangerous rocks and generate the movement status of dangerous rock bodies, and to evaluate and warn the dangerous rock bodies.

[0137] Specifically, Figure 7 The flowchart of generating a three-dimensional model of an embodiment of an air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is shown, and generating a three-dimensional model includes:

[0138] According to the precise position information of the cruising drone 1 and the image data of the dangerous rock mass at multiple angles, an image of the disc target 701 is obtained;

[0139] According to the image of the disk target 701, two-dimensional image correction is performed to obtain feature points in a plurality of image sequences;

[0140] According to the feature points in the plurality of image sequences, invalid feature points are eliminated according to a set threshold value to obtain feature points in a valid image sequence;

[0141] According to the feature points in the effective image sequence, a two-dimensional image with multi-dimensional association is obtained through feature point matching and perspective change;

[0142] According to the multi-dimensionally associated two-dimensional images, the multi-dimensionally associated two-dimensional images are synthesized by using SFM (Structure from Motion, structured optical flow method) three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model.

[0143] Specifically, Figure 8 1 is a flowchart of calculating the displacement and inclination of dangerous rocks and generating movement trends, and evaluating and warning the dangerous rock bodies according to an embodiment of the air-sky-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention, which calculates the displacement and inclination of dangerous rocks and generates movement trends, and evaluates and warns the dangerous rock bodies, including:

[0144] According to the three-dimensional model, characteristic points in the three-dimensional model are compared, and the boundary of the dangerous rock mass is obtained through displacement variation;

[0145] According to the three-dimensional model and the boundary of the dangerous rock mass, the volume, boundary surface and inclination direction of the dangerous rock mass are calculated and evaluated to obtain the displacement, inclination and movement state of the dangerous rock mass;

[0146] According to the displacement, inclination and movement state of the dangerous rock mass, an assessment model of the dangerous rock mass is obtained;

[0147] According to the evaluation model of the dangerous rock mass, the danger level is obtained and an early warning is issued for the dangerous rock mass.

[0148] Specifically, Fig. 9 The flowchart of the embodiment of the air-space-ground dangerous rock collapse monitoring system based on pure visual perception of the present invention is to obtain the danger level through the evaluation model of the dangerous rock body and to give an early warning to the dangerous rock body, and to obtain the danger level and give an early warning to the dangerous rock body according to the evaluation model of the dangerous rock body, including:

[0149] According to the evaluation model of the dangerous rock mass, the velocity of the dangerous rock mass is obtained. , acceleration of dangerous rock mass , displacement of dangerous rock mass , the inclination angle of the dangerous rock mass and shooting time ;

[0150] According to the speed of the dangerous rock mass and shooting time , calculated by formula (1), the velocity change rate of two adjacent times is obtained;

[0151] (1)

[0152] In the formula, is the speed change rate, is the velocity change at time t+1, is the velocity change at time t, is the speed at time t+1, is the speed at time t, is the speed at time t-1;

[0153] According to the acceleration of the dangerous rock mass and shooting time , calculated by formula (2), the acceleration change rate between two adjacent times is obtained;

[0154] (2)

[0155] In the formula, is the rate of change of acceleration, is the acceleration change at time t+1, for The acceleration change at time, is the acceleration at time t+1, for The acceleration of time, is the acceleration at time t-1;

[0156] According to the displacement of the dangerous rock mass and shooting time , calculated by formula (3), the displacement change rate between two adjacent times is obtained;

[0157] (3)

[0158] In the formula, is the displacement change rate, is the displacement change at time t+1, for The displacement change at time, is the displacement at time t+1, is the displacement at time t, is the displacement at time t-1;

[0159] According to the inclination angle of the dangerous rock mass and shooting time , calculated by formula (4), the rate of change of the tilt angle between two adjacent times is obtained;

[0160] (4)

[0161] In the formula, is the rate of change of the tilt angle, is the change in the tilt angle at time t+1, for The change in tilt angle at time, is the tilt angle at time t+1, for The tilt angle at the moment, is the inclination angle at time t-1;

[0162] According to the evaluation model of the dangerous rock mass, the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle are obtained, and the relationship among the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle conforms to formula (5);

[0163] (5)

[0164] In the formula, is the weight factor of speed, is the weight factor of acceleration, is the weight factor of the displacement, is the weight factor of the tilt angle;

[0165] According to the velocity change rate of the two adjacent times, the acceleration change rate of the two adjacent times, the displacement change rate of the two adjacent times, the tilt angle change rate of the two adjacent times, the weight factor of the velocity, the weight factor of the acceleration, the weight factor of the displacement and the weight factor of the tilt angle, the comprehensive weight is obtained by formula (6): ;

[0166] (6)

[0167] According to the comprehensive weight and pre-set warning thresholds Compare and obtain the danger level according to the early warning rules and issue an early warning for the dangerous rock mass. The early warning rules are as follows:

[0168] Rule 1: When The warning level is level IV, which is a blue warning;

[0169] Rule 2: When The warning level is level III, which is a yellow warning;

[0170] Rule 3: When The warning level is Level II, which is an orange warning;

[0171] Rule 4: When The warning level is level Ⅰ, which is a red warning;

[0172] When the warning level reaches orange or red, on-site personnel need to evacuate.

[0173] like Fig.10 The flowchart of the embodiment of the air-air-ground dangerous rock collapse monitoring method based on pure visual perception of the present invention is shown. The present invention provides an air-air-ground dangerous rock collapse monitoring method based on pure visual perception. The method is implemented by an air-air-ground dangerous rock collapse monitoring system based on pure visual perception. The method includes:

[0174] S1, installing the RTK positioning module 2 and the visual sensor device 3 on the cruise drone 1, obtaining the area range of the dangerous rock mass and designing the target layout plan through the preliminary flight of the cruise drone 1;

[0175] S2, put the needle 705 into the slideway of the conical leg 703, put the silicone film 704 into the upper cavity of the conical leg 703, prepare glue, pour it into the silicone film 704, and tie it, then screw the three conical legs 703 into the screw holes at the three corners of the triangular prism iron block 702, and finally connect one end of the three hanging ropes 707 to the three corners of the triangular prism iron block 702, and connect the other end to the magnet 708, to obtain a drop-in bracket;

[0176] S3, according to the target point arrangement plan, the disc target 701 and the delivery bracket are delivered to the pre-designed target point position by the carrier drone 7, so as to obtain the area of ​​the dangerous rock mass with the disc target 701;

[0177] S4, according to the area of ​​the dangerous rock body with the disc target 701, set the flight area and route of the cruise drone 1, start the cruise drone 1 in the automatic cruise state to take pictures of the area of ​​the dangerous rock body with the disc target 701, obtain the precise position information of the cruise drone 1 and the image data of the dangerous rock body at multiple angles, and transmit the precise position information of the cruise drone 1 and the image data of the dangerous rock body at multiple angles to the real-time modeling workstation 5;

[0178] S5, according to the image data of the dangerous rock body at multiple angles, the disk target 701 is used to correct the image data of the dangerous rock body at multiple angles to obtain corrected image data;

[0179] S6. According to the precise position information of the cruise drone 1 and the corrected image data, the corrected image data is synthesized by using SFM three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model;

[0180] S7, according to the three-dimensional model, by changing the distance of the feature points, obtaining a typical point displacement change field;

[0181] S8. According to the displacement change field of the typical point, the boundary of the dangerous rock mass is obtained through the displacement change amount;

[0182] S9, calculating and evaluating the volume, boundary surface and inclination direction of the dangerous rock mass according to the three-dimensional model and the boundary of the dangerous rock mass, obtaining the displacement and inclination of the dangerous rock mass and generating a movement situation;

[0183] S10, evaluating the dangerous rock mass according to its displacement, inclination and movement status, generating an evaluation model, and if the evaluation model reaches a danger threshold, dispatching the cruise drone 1 to fly close to the dangerous part to identify the internal features of the dangerous part,

[0184] S11. Based on the internal characteristics of the dangerous area, a more detailed and accurate assessment is made to determine the warning level of the dangerous area. When the warning level reaches orange or red warning, on-site personnel must be evacuated.

[0185] The present invention provides an air-space-ground dangerous rock collapse monitoring system and method based on pure visual perception. The invention solves the problems of high cost, low efficiency and great danger of on-site deployment of monitoring devices through a cruise drone 1, an RTK positioning module 2, a visual sensor device 3, an airdrop target deployment device 4 and a real-time modeling workstation 5. On the other hand, the RTK positioning module 2 uses a public reference station and satellite positioning to solve the problems of poor accuracy and susceptibility to interference based on GNSS (Global Navigation Satellite System) positioning technology. Finally, the motion posture, displacement and inclination of the dangerous rock are calculated through a three-dimensional model and the dangerous rock body is evaluated, which solves the problems of poor accuracy and low timeliness of traditional methods.

[0186] It is to be understood that the present invention is described by the above embodiments and should not be construed as limiting the embodiments of the present invention and the scope of the present invention. It is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception is characterized by: The system comprises: Cruise drones, used to perform autonomous flight missions and cover surveillance areas; The RTK positioning module is used to perform high-precision positioning of the cruise UAV during flight to obtain accurate position information of the cruise UAV; Visual sensing device, used to take multi-directional photographs of dangerous rock bodies and obtain image data of dangerous rock bodies at multiple angles; An airdrop target placement device is used to accurately place targets in the monitoring area, correct image data, and improve visual positioning and modeling accuracy; The real-time modeling workstation is used to generate a three-dimensional model, calculate the displacement and inclination of dangerous rocks, generate movement trends, and evaluate and warn the movement trends of the dangerous rock bodies.

2. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The RTK positioning module adopts multi-frequency anti-interference technology and multi-step adaptive filtering technology; The RTK positioning module uses public reference stations and satellites based on the principle of relative positioning to calculate the three-dimensional coordinates of the cruise drone in real time, which is used to assist the image data calculation process and improve the calculation speed and accuracy.

3. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The airdrop target deployment device comprises: A carrier drone, used to carry and drop the target to a predetermined location in the monitoring area; Disc targets to provide high-contrast visual reference points; A drop-type bracket is used for stabilizing and releasing the disc target.

4. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 3 is characterized in that: The drop-type bracket comprises: Conical legs for providing stable support and positioning of the drop-type stand; The triangular prism-shaped iron block is used to increase the weight of the drop-type bracket to ensure that the drop-type bracket is firmly grounded after being dropped; Bolts, used to fix the disc target; A sling, used to assist the carrier drone in carrying the drop-type bracket and the disc target; A magnet, used to assist the temporary adsorption of the carrier drone and the drop-type bracket to ensure accurate release of the drop-type bracket; The tapered leg comprises: A silicone film is used to contain glue, and the silicone film breaks after falling to the ground, allowing the glue to flow out, thereby increasing the stability of the drop-type bracket; The puncture needle is used to puncture the silicone membrane and enhance the grip of the drop-type bracket on the ground to prevent the drop-type bracket from moving or tilting after being deployed.

5. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 3 is characterized in that: The airdrop target placement device is placed inside and outside the deformation area of ​​the dangerous rock mass, and the tilt directions of the placed disc targets cover four directions: east, west, south, and north.

6. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 1 is characterized in that: The real-time modeling workstation comprises: A wireless receiver, used to receive data from the cruise drone; PC side, used to process and display the received data; Point cloud synthesis software for generating 3D models; The image processing device is used to calculate the displacement and inclination of dangerous rocks and generate the movement status of dangerous rock bodies, and to evaluate and warn the dangerous rock bodies.

7. The air-space-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 6 is characterized in that: The generating of the three-dimensional model comprises: Obtaining an image of the disc target according to the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles; Performing two-dimensional image correction according to the image of the disk target to obtain feature points in a plurality of image sequences; According to the feature points in the plurality of image sequences, invalid feature points are eliminated according to a set threshold value to obtain feature points in a valid image sequence; According to the feature points in the effective image sequence, a two-dimensional image with multi-dimensional association is obtained through feature point matching and perspective change; According to the multi-dimensionally associated two-dimensional images, the multi-dimensionally associated two-dimensional images are synthesized by using the SFM three-dimensional reconstruction technology and the point cloud synthesis software to obtain a three-dimensional model.

8. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 6 is characterized in that: The calculation of the displacement and inclination of the dangerous rock and the generation of the movement situation, and the evaluation and early warning of the dangerous rock body include: According to the three-dimensional model, characteristic points in the three-dimensional model are compared, and the boundary of the dangerous rock mass is obtained through displacement change; Calculate and evaluate the volume, boundary surface and inclination direction of the dangerous rock mass according to the three-dimensional model and the boundary of the dangerous rock mass, and obtain the displacement, inclination and movement state of the dangerous rock mass; According to the displacement and inclination of the dangerous rock mass and the movement state of the dangerous rock mass, an assessment model of the dangerous rock mass is obtained; According to the evaluation model of the dangerous rock mass, the danger level is obtained and an early warning is issued for the dangerous rock mass.

9. The air-sky-ground dangerous rock collapse monitoring system based on pure visual perception according to claim 8 is characterized in that: According to the evaluation model of the dangerous rock mass, the danger level is obtained and an early warning is given to the dangerous rock mass, including: According to the evaluation model of the dangerous rock mass, the velocity of the dangerous rock mass is obtained. , acceleration of dangerous rock mass , displacement of dangerous rock mass , the inclination angle of the dangerous rock mass and shooting time ; According to the speed of the dangerous rock mass and shooting time , calculated by formula (1), the velocity change rate of two adjacent times is obtained; （1） In the formula, is the speed change rate, is the velocity change at time t+1, is the velocity change at time t, is the speed at time t+1, is the speed at time t, is the speed at time t-1; According to the acceleration of the dangerous rock mass and shooting time , calculated by formula (2), the acceleration change rate between two adjacent times is obtained; （2） In the formula, is the rate of change of acceleration, is the acceleration change at time t+1, for The acceleration change at time, is the acceleration at time t+1, for The acceleration of time, is the acceleration at time t-1; According to the displacement of the dangerous rock mass and shooting time , calculated by formula (3), the displacement change rate between two adjacent times is obtained; （3） In the formula, is the displacement change rate, is the displacement change at time t+1, for The displacement change at time, is the displacement at time t+1, is the displacement at time t, is the displacement at time t-1; According to the inclination angle of the dangerous rock mass and shooting time , calculated by formula (4), the rate of change of the tilt angle between two adjacent times is obtained; （4） In the formula, is the rate of change of the tilt angle, is the change in the tilt angle at time t+1, for The change in tilt angle at time, is the tilt angle at time t+1, for The tilt angle at the moment, is the inclination angle at time t-1; According to the evaluation model of the dangerous rock mass, a weight factor of velocity, a weight factor of acceleration, a weight factor of displacement and a weight factor of inclination angle are obtained, and the relationship between the weight factor of velocity, the weight factor of acceleration, the weight factor of displacement and the weight factor of inclination angle conforms to formula (5); （5） In the formula, is the weight factor of speed, is the weight factor of acceleration, is the weight factor of the displacement, is the weight factor of the tilt angle; According to the velocity change rate of two adjacent times, the acceleration change rate of two adjacent times, the displacement change rate of two adjacent times, the inclination angle change rate of two adjacent times, the velocity weight factor, the acceleration weight factor, the displacement weight factor and the inclination angle weight factor, the comprehensive weight is obtained by formula (6): ; （6） According to the comprehensive weight and pre-set warning thresholds Compare and obtain the danger level according to the early warning rules and issue an early warning for the dangerous rock mass. The early warning rules are as follows: Rule 1: The warning level is level IV, which is a blue warning; Rule 2: When The warning level is level III, which is a yellow warning; Rule 3: When The warning level is Level II, which is an orange warning; Rule 4: When The warning level is level Ⅰ, which is a red warning; When the warning level reaches the orange warning or the red warning, on-site personnel need to evacuate.

10. An air-air-ground dangerous rock collapse monitoring method based on pure visual perception, the air-air-ground dangerous rock collapse monitoring method based on pure visual perception is implemented by the air-air-ground dangerous rock collapse monitoring system based on pure visual perception according to any one of claims 1 to 8, characterized in that: The method comprises: S1. Installing an RTK positioning module and a visual sensor device on a cruise drone, obtaining the area of ​​the dangerous rock mass and designing a target layout plan through a preliminary flight of the cruise drone; S2, placing the needle into the slideway of the conical leg, placing the silicone membrane into the upper cavity of the conical leg, preparing glue, pouring it into the silicone membrane, and tying it, then screwing the three conical legs into the screw holes at the three corners of the triangular prism-shaped iron block, and finally connecting one end of the three hanging ropes to the three corners of the triangular prism-shaped iron block and the other end to the magnet to obtain a drop-type bracket; S3, according to the target arrangement plan, the disc target and the delivery bracket are delivered to the pre-designed target position by a carrier drone to obtain the area of ​​the dangerous rock mass with the disc target; S4, according to the area of ​​the dangerous rock body with the disc target, set the flight area and route of the cruise drone, start the cruise drone to take photos of the area of ​​the dangerous rock body with the disc target in an automatic cruise state, obtain the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles, and transmit the precise position information of the cruise drone and the image data of the dangerous rock body at multiple angles to the real-time modeling workstation; S5, according to the image data of the dangerous rock body at multiple angles, using the disk target to correct the image data of the dangerous rock body at multiple angles to obtain corrected image data; S6. According to the precise position information of the cruise drone and the corrected image data, synthesize the corrected image data by using SFM three-dimensional reconstruction technology and point cloud synthesis software to obtain a three-dimensional model; S7, obtaining a typical point displacement change field through a distance change of feature points according to the three-dimensional model; S8. According to the displacement change field of the typical points, the boundary of the dangerous rock mass is obtained through the displacement change amount; S9, calculating and evaluating the volume, boundary surface and inclination direction of the dangerous rock mass according to the three-dimensional model and the boundary of the dangerous rock mass, obtaining the displacement and inclination of the dangerous rock mass and generating a movement situation; S10, evaluating the dangerous rock mass according to the displacement, inclination and movement state of the dangerous rock, generating an evaluation model, and if the evaluation model reaches a danger threshold, dispatching the cruise drone to fly close to the dangerous part to identify the internal features of the dangerous part; S11. According to the internal characteristics of the dangerous area, through more detailed and accurate analysis, the warning level of the dangerous area is obtained. When the warning level reaches orange warning or red warning, on-site personnel need to be evacuated.

Images