A radar and camera-based method and system for predicting the trajectory of a falling rock on a slope

Abstract

The application discloses a kind of based on radar and camera's side slope rockfall movement trajectory prediction method and system, method includes: S1 any point on slope is obtained with the slope and slope direction information;S2 the slope surface image of slope body is obtained, identifies slope surface feature, according to preset slope surface feature and slope surface coefficient contrast table, initial slope surface coefficient is automatically given to each grid;S3 is based on slope surface coefficient and rockfall kinematics model, the movement trajectory and landing point of rockfall are predicted;S4 real-time position information and corresponding position information in predicted movement trajectory of rockfall are compared;If position difference value is in preset threshold range, then enter S5;If position difference value exceeds preset threshold range, then determine that slope surface environment has changed significantly, return S2;S5 with slope surface coefficient as variable constructs objective function, carries out iterative optimization search, finds optimal slope surface combination, re-executes S3.The application has the advantages of high prediction accuracy.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of rockfall monitoring, and in particular to a slope rockfall motion trajectory prediction method and system based on radar and cameras. BACKGROUND

[0002] The rockfall motion trajectory prediction method can be generally divided into two categories, namely, an empirical method mainly based on experimental research and a method mainly based on theoretical derivation. The former mainly includes field test research and indoor scale model test research. The theoretical derivation method refers to establishing a reasonable mathematical calculation model based on kinematics and dynamics theory to fit the motion trajectory of the rockfall along the slope surface. The rockfall motion trajectory is generally divided into a rolling (sliding) motion stage, a free falling motion stage and a slope surface collision stage. It is generally recognized that the main factors affecting the slope rockfall motion trajectory include the slope shape (slope gradient, slope length), the geomechanical properties of the slope surface (such as the roughness of the slope surface, the vegetation coverage rate of the slope surface, the softness of the soil covering the slope surface, and the hardness of the bare rock of the slope surface), the size and shape of the rolling rock.

[0003] The slope rockfall monitoring radar is a radar system for monitoring slope rockfall, which mainly transmits electromagnetic wave signals to the monitoring area and receives the signals reflected back by the rockfall. The time delay of the reflected echo can be measured to accurately measure the distance of the rockfall, and the signal-to-clutter ratio of the rockfall can be improved through clutter suppression, the velocity of the rockfall can be measured through Doppler processing, and the angle information of the rockfall can be measured through MIMO imaging technology, so that high-precision measurement of the rockfall can be achieved.

[0004] The current technology related to the present patent application is a rock-soil slope arbitrary shape rockfall motion trajectory three-dimensional analysis method, which has the following method steps: a three-dimensional model of an arbitrary shape rockfall and a slope surface is established in a system coordinate system, whether the rockfall contacts the slope surface at each time is judged through a contact search algorithm, and the contact force at the time of contact is calculated through a contact model; a motion equation of the rockfall is established according to the contact force and the action of gravity, and a time step method is used to solve the motion method to obtain the motion trajectory of the rockfall.

[0005] In the existing research method, the slope coefficient is obtained through multiple tests or manually set according to the experience value. Using multiple tests to obtain the slope coefficient requires a large amount of manpower, and after changing the scene, the correct slope coefficient can be obtained only by conducting tests again. Using manual experience value to obtain the slope coefficient requires manual on-site surveying, and the accuracy is not high. Both of the existing methods require a large amount of manpower and have low adaptability.

[0006] In the existing research method, external feedback data is rarely used to correct the slope coefficient. When the adaptability of the slope coefficient to the monitoring scene is not high, the prediction model does not match, resulting in a large difference between the predicted rockfall motion trajectory and the actual motion trajectory.

[0007] In the prior research method, the motion data of rockfall is acquired by a contact sensor, which has a small monitoring range and is only applicable to the slope scene that can be reached by human. SUMMARY

[0008] In view of the technical problems existing in the prior art, the present application provides a radar and camera-based slope rockfall motion trajectory prediction method and system with high prediction accuracy.

[0009] To solve the above technical problems, the technical solution provided by the present application is as follows:

[0010] A radar and camera-based slope rockfall motion trajectory prediction method, comprising the steps of:

[0011] S1. Establishing a radar rectangular coordinate system with the radar as the origin, acquiring a digital elevation model with the radar as the origin, and obtaining the slope and slope direction information of any point on the slope surface through the digital elevation model;

[0012] S2. Acquiring the slope surface image of the monitored slope body through the camera, performing grid processing on the slope surface image, identifying the slope surface features of each grid based on a deep learning algorithm, and automatically assigning an initial slope surface coefficient to each grid according to a preset slope surface feature and slope surface coefficient comparison table;

[0013] S3. Based on the slope and slope direction information obtained in S1 and the slope surface coefficient obtained in S2, combining a rockfall kinematics model, predicting the motion trajectory and landing point of the rockfall;

[0014] S4. Measuring the real-time position information of the rockfall using the radar, comparing the real-time position information with the corresponding position information in the motion trajectory predicted in S3, and if the position difference value is within a preset threshold range, proceeding to S5; if the position difference value exceeds the preset threshold range, determining that the slope environment has changed significantly, and returning to S2 to reacquire the slope surface image and the slope surface coefficient;

[0015] S5. Constructing a target function with the slope surface coefficient as the variable, performing iterative optimization search within a preset parameter range, finding the optimal slope surface coefficient combination that minimizes the target function value, and based on the optimal slope surface coefficient combination, re-executing S3 to update the rockfall motion trajectory prediction result.

[0016] Preferably, the slope surface features in step S2 include one or more of vegetation coverage, vegetation type, soil type and rock surface smoothness.

[0017] Preferably, the slope surface coefficient in step S2 includes a normal restitution coefficient , a tangent restitution coefficient and a friction coefficient .

[0018] Preferably, the rockfall kinematic model of step S3 comprises a sliding motion model, an oblique throwing motion model and a collision motion model.

[0019] The sliding motion model is:

[0020] ;

[0021] wherein a is the acceleration when the rockfall slides, is the slope angle; g is the gravity acceleration, is the friction coefficient, is the end speed of rockfall sliding, is the initial speed of rockfall sliding, is the sliding time;

[0022] The oblique throwing motion model is:

[0023]

[0024] ;

[0025] ;

[0026] wherein is the horizontal velocity component in the process of rockfall oblique throwing, is the horizontal velocity component at the end of rockfall sliding, is the vertical velocity component in the process of rockfall oblique throwing, is the vertical velocity component at the end of rockfall sliding, is the height direction velocity component in the process of rockfall oblique throwing, is the height direction velocity component at the end of rockfall sliding, is the oblique throwing time;

[0027] The collision motion model is:

[0028]

[0029]

[0030] wherein is the resultant velocity of rockfall in the radar XOY plane after collision, vr is the resultant velocity of rockfall in the radar XOY plane before collision, is the velocity of rockfall in the height direction before collision, is the velocity of rockfall in the height direction after collision; is the normal restitution coefficient; is the tangential restitution coefficient.

[0031] Preferably, after the rockfall collides, the normal component of the velocity of the rockfall after collision is and tangential component If the difference is greater than a preset threshold, it is determined that the rockfall is sliding, and the rockfall is determined to continue to be obliquely thrown. If not, the rockfall continues to be obliquely thrown, and the process is repeated until the rockfall stops moving, and the landing point of the rockfall is obtained.

[0032] Preferably, in step S4, the position difference value is the distance between the real-time position information and the corresponding position information in the predicted motion trajectory; and the preset threshold range is 0.5m-2m.

[0033] Preferably, in step S5, the target function is:

[0034]

[0035] wherein is the normal restitution coefficient; is the tangential restitution coefficient; is the friction coefficient; is the real-time position information of the rockfall measured by the radar; is the corresponding position information in the predicted motion trajectory.

[0036] Preferably, the iterative optimization search in step S5 adopts a full-range step search method.

[0037] Preferably, in step S2, the slope surface image of the monitored slope is obtained by a camera After that, the slope surface image information is converted to the radar rectangular coordinate system through a coordinate conversion relationship, to obtain image information After pretreatment, the gridded image is obtained.

[0038] The application further discloses a slope rockfall monitoring and motion trajectory prediction system based on a radar and a camera, which comprises a memory and a processor connected to each other, and the memory stores a computer program, and the computer program performs the steps of the method when the processor is running.

[0039] Compared with the prior art, the application has the following advantages:

[0040] This invention uses a high-resolution camera to capture images of the monitored area, combines image recognition technology to extract slope features in a grid pattern, and automatically sets the slope coefficient based on a slope feature and slope coefficient comparison table. This method can adaptively register initial parameters for different monitoring scenarios without manual adjustment, improving applicability and saving labor costs. Utilizing the high-precision measurement characteristics of radar, real-time radar measurement information is compared with predicted rockfall trajectories. When the difference is significant but within a reasonable range (i.e., within a preset threshold range), an objective function is established for iterative optimization to find the optimal slope coefficient, and a new model is built for trajectory prediction, thereby improving the accuracy of rockfall trajectory prediction. When the difference exceeds a reasonable range (i.e., exceeds the preset threshold range), it indicates a significant change in the slope environment. The camera is then activated to reacquire image information of the monitored slope, and the initial slope coefficient is reloaded to rebuild the model for trajectory prediction. Leveraging the non-contact and wide-range measurement characteristics of radar, it can achieve large-scale slope rockfall monitoring and trajectory prediction, especially suitable for slope scenarios that are difficult for humans to climb. Attached Figure Description

[0041] Figure 1 This is a flowchart of an embodiment of the radar and camera-based slope rockfall trajectory prediction method of the present invention.

[0042] Figure 2 This diagram illustrates the relationship between the radar rectangular coordinate system and the slope coordinate system in this invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0044] like Figure 1 As shown, the slope rockfall trajectory prediction method based on radar and camera provided in this embodiment of the invention includes the following steps:

[0045] S1. Establish a radar rectangular coordinate system with the radar as the origin, obtain a digital elevation model with the radar as the origin, and obtain the slope and aspect information of any point on the slope through the digital elevation model;

[0046] Specifically, a radar rectangular coordinate system XYZ is established with the radar as the origin, and point P is a point on the slope with a velocity of... In the XYZ coordinate system, the coordinates are ; Obtain a digital elevation model with the radar as the origin, and obtain slope and aspect information, such as slope angle and aspect angle, at any point on the slope through the digital elevation model.

[0047] To facilitate the analysis of the trajectory of falling rocks on the slope, a slope coordinate system is established with point P as the origin. , The axis is parallel to the slope aspect, and the slope angle at point P is... The slope angle is Its velocity in the slope coordinate system is The relationship between the two coordinate systems is as follows: Figure 2 As shown.

[0048] S2. Acquire slope surface images of the monitored slope using a camera, perform gridding processing on the slope surface images, and identify the slope surface features of each grid based on a deep learning algorithm. According to a pre-defined slope surface feature and slope coefficient comparison table, automatically assign an initial slope coefficient to each grid; the specific steps are as follows:

[0049] S201. Acquire slope image information of the monitored area using a high-resolution camera. ;

[0050] S202. Transfer slope image information The image information is obtained by transforming the coordinates to a coordinate system with the radar as the reference point. ;

[0051] Specifically, assuming point P is a point on the slope, its position in the radar coordinate system is... The coordinates in the camera are The position in the image coordinate system is ;

[0052] The transformation relationship between the radar coordinate system and the camera coordinate system is shown below:

[0053]

[0054] Where R is a 3x3 rotation matrix and T is a 3x1 translation matrix. The extrinsic parameter matrix of the camera;

[0055] Transformation matrix from camera coordinate system to image coordinate system (The K matrix is ​​the camera intrinsic parameter matrix.) and f is the camera's focal length, and dx and dy are the length and width of a single pixel in the image plane, respectively. The coordinates of the origin of the image coordinate system in the pixel coordinate system.

[0056] The transformation relationship between the camera coordinate system and the image coordinate system is shown below:

[0057]

[0058] In summary, the transformation relationship from the image coordinate system to the radar coordinate system is as follows:

[0059]

[0060]

[0061] in Represents the matrix Take the first 3 rows and 3 columns. ; This represents the position along the z-axis in the camera coordinate system. For intermediate transformation variables;

[0062] S203. Will After preprocessing such as size adjustment and normalization, the YOLOv5 model is used to identify the slope features of each grid (including vegetation coverage and type, soil type, and rock surface smoothness). YOLOv5 adopts a single-stage detection framework, which simultaneously achieves feature extraction, target localization, and classification through an end-to-end convolutional neural network.

[0063] S204. Based on the slope characteristics and slope coefficient comparison tables shown in Tables 1-3, an initial slope coefficient is automatically selected for each grid; the initial slope coefficient includes the normal recovery coefficient. Tangential coefficient of restitution and coefficient of friction ;

[0064] Table 1 Relationship between normal coefficient of restoration and slope characteristics

[0065]

[0066] Table 2 Relationship between tangential restoration coefficient and slope characteristics

[0067]

[0068] Table 3 Relationship between friction coefficient and slope characteristics

[0069]

[0070] S3. Based on the slope and aspect information of any point on the slope obtained in S1 and the slope coefficient obtained in S2, and combined with the rockfall kinematics model, predict the trajectory and impact point of the falling rock; the specific steps are as follows:

[0071] S301. Based on the transformation relationship between the radar coordinate system and the slope coordinate system, the coordinates of the falling rock on the radar are transformed to the slope coordinate system, and the speed measured by the radar is... Velocity converted to slope coordinate system The transformation relationship is as follows:

[0072]

[0073]

[0074] S302. Based on the obtained initial slope coefficient, the trajectory of falling rocks is predicted in conjunction with the rockfall motion equation; the trajectory of falling rocks is generally divided into three states: sliding, oblique projection, and collision.

[0075] The sliding motion model is as follows:

[0076] ;

[0077] Where g is the acceleration due to gravity. Let be the coefficient of friction, and 'a' be the acceleration of the falling rock as it slides. For sliding time, The initial velocity, ; The end speed of the slide. .

[0078] After the sliding motion ends, the object typically undergoes projectile motion, with a projectile time of [time missing]. The projectile motion model is as follows:

[0079]

[0080] ;

[0081] ;

[0082] in The horizontal velocity component at the end of the rockfall slide. The horizontal velocity component during the oblique throwing process of the falling rock. This represents the vertical velocity component at the end of the rockfall slide. This represents the vertical velocity component during the oblique throwing process of the falling rock. The height-axis velocity component represents the velocity at the end of the rockfall slide. This represents the height-axis velocity component during the oblique throwing process of falling rocks.

[0083] After the projectile motion of the falling rock ends, it enters a collision state. The collision motion model is as follows:

[0084]

[0085]

[0086] Where vr is the resultant velocity of the falling rock in the radar XOY plane before the collision. , Let X be the resultant velocity of the falling rock on the radar XOY plane after the collision. The velocity of the falling rock in the height direction before the collision. The velocity of the falling rock in the height direction after the collision;

[0087] Normal component of the velocity of falling rocks after collision and tangential components It is calculated by the following formula:

[0088] ; ;

[0089] Based on the normal component of the velocity of the falling rock after the collision and tangential components To determine whether the falling rock continues to be thrown at an angle or slides, if If the rock stops moving, it will slide; otherwise, continue the oblique throwing motion, repeating the cycle until the rock stops moving, thus obtaining the predicted position information. ;

[0090] S4. Utilize radar to measure real-time information on falling rocks (including real-time location information of the falling rocks). (and speed), real-time location information Position information in the motion trajectory predicted by S3 A comparison is performed between the real-time location information and the predicted location information. If the (location difference value) is within the preset threshold range, proceed to S5; if the location difference value exceeds the preset threshold, it is determined that the slope environment has changed significantly, and return to S2, and re-acquire the slope image and slope coefficient of the monitored area through the camera; the preset threshold range is preferably 0.5m-2m, that is, when the difference value does not exceed 0.5m, no optimization adjustment is required, when the difference value is between 0.5m and 2m, proceed to S5, when the difference value exceeds 2m, it is determined that the slope environment has changed significantly, and return to S2, and re-acquire the slope image and slope coefficient of the monitored area through the camera;

[0091] S5. Construct an objective function using the slope coefficient as a variable, perform iterative optimization search within the preset parameter range to find the optimal combination of slope coefficients that minimizes the objective function value, and re-execute S3 based on the optimal combination of slope coefficients to update the predicted rockfall trajectory.

[0092] Specifically, an objective function is established with slope parameters as variables. Among them, slope parameters The parameter comparison table above has certain limitations; regarding slope parameters... Within the constraints, perform an iterative full-range search with a step size of 0.05 to find the set of slope parameter values ​​that minimizes the objective function. The slope parameters at this point... The value obtained is the optimal value; the optimal value is used to rebuild the prediction model and predict the trajectory of the falling rock.

[0093] This invention uses a high-resolution camera to capture images of the monitored area, combines image recognition technology to extract slope features in a grid pattern, and automatically sets the slope coefficient based on a slope feature and slope coefficient comparison table. This method can adaptively register initial parameters for different monitoring scenarios without manual adjustment, improving applicability and saving labor costs. Utilizing the high-precision measurement characteristics of radar, real-time radar measurement information is compared with predicted rockfall trajectories. When the difference is significant but within a reasonable range (i.e., within a preset threshold range), an objective function is established for iterative optimization to find the optimal slope coefficient, and a new model is built for trajectory prediction, thereby improving the accuracy of rockfall trajectory prediction. When the difference exceeds a reasonable range (i.e., exceeds the preset threshold range), it indicates a significant change in the slope environment. The camera is then activated to reacquire image information of the monitored slope, and the initial slope coefficient is reloaded to rebuild the model for trajectory prediction. Leveraging the non-contact and wide-range measurement characteristics of radar, it can achieve large-scale slope rockfall monitoring and trajectory prediction, especially suitable for slope scenarios that are difficult for humans to climb.

[0094] This invention also discloses a slope rockfall monitoring and trajectory prediction system based on radar and cameras, comprising an interconnected memory and a processor. The memory stores a computer program, which, when run by the processor, executes the steps of the method described above. The prediction system of this invention corresponds to the prediction method described above and also possesses the advantages described above.

[0095] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium includes: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0096] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for predicting the trajectory of falling rocks on slopes based on radar and cameras, characterized in that, Including the following steps: S1. Establish a radar rectangular coordinate system with the radar as the origin, obtain a digital elevation model with the radar as the origin, and obtain the slope and aspect information of any point on the slope through the digital elevation model; S2. Acquire slope images of the monitored slope using a camera, perform gridding processing on the slope images, identify the slope features of each grid based on a deep learning algorithm, and automatically assign an initial slope coefficient to each grid according to a preset slope feature and slope coefficient comparison table. S3. Based on the slope and aspect information obtained in S1 and the slope coefficient obtained in S2, combined with the rockfall kinematics model, the trajectory and landing point of the falling rock are predicted. S4. Utilize radar to measure the real-time location information of the falling rocks, and compare the real-time location information with the corresponding location information in the motion trajectory predicted in S3. If the location difference value is within the preset threshold range, proceed to S5; if the location difference value exceeds the preset threshold range, it is determined that the slope environment has changed significantly, and return to S2 to reacquire the slope image and slope coefficient. S5. Construct an objective function using the slope coefficient as a variable, perform iterative optimization search within the preset parameter range to find the optimal combination of slope coefficients that minimizes the objective function value, and re-execute S3 based on the optimal combination of slope coefficients to update the predicted rockfall trajectory.

2. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1, characterized in that, The slope characteristics mentioned in step S2 include one or more of the following: vegetation coverage, vegetation type, soil type, and rock surface smoothness.

3. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, The slope coefficient mentioned in step S2 includes the normal restoration coefficient. Tangential recovery coefficient and coefficient of friction .

4. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, The rockfall kinematic model in step S3 includes a sliding motion model, a projectile motion model, and a collision motion model. The sliding motion model is as follows: ； Where 'a' is the acceleration of the falling rock as it slides. The slope angle is g; g is the acceleration due to gravity. The coefficient of friction, The velocity at which the rockfall ends. Let the initial velocity of the falling rock be denoted as . The sliding time; The projectile motion model is as follows: ; ; in The horizontal velocity component during the oblique throwing process of the falling rock. The horizontal velocity component at the end of the rockfall slide. This represents the vertical velocity component during the oblique throwing process of the falling rock. This represents the vertical velocity component at the end of the rockfall slide. The height-axis velocity component during the oblique throwing process of the falling rock. The height-axis velocity component represents the velocity at the end of the rockfall slide. For projectile motion; The collision motion model is as follows: in Let vr be the resultant velocity of the falling rock in the radar XOY plane after the collision, and vr be the resultant velocity of the falling rock in the radar XOY plane before the collision. The velocity of the falling rock in the height direction before the collision. The velocity of the falling rock in the height direction after the collision; The normal restitution coefficient; This is the tangential recovery coefficient.

5. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 4, characterized in that, After the rockfall collision, based on the normal component of the rockfall velocity after the collision... and tangential components To determine whether the falling rock continues to be thrown at an angle or slides, if If the rock falls, it will slide; otherwise, continue throwing at an angle, repeating the cycle until the rock stops moving, thus obtaining the landing point of the rock.

6. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S4, the position difference value is the distance between the real-time position information and the corresponding position information in the predicted motion trajectory; The preset threshold range is 0.5m-2m.

7. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S5, the objective function for: in The normal restitution coefficient; The tangential restitution coefficient; The coefficient of friction; Real-time location information of falling rocks as measured by radar; This refers to the corresponding position information in the predicted motion trajectory.

8. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 7, characterized in that, The iterative optimization search in step S5 adopts a full-range step search method.

9. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S2, a slope surface image of the monitored slope is acquired using a camera. Then, the slope image information The image information is obtained by transforming the coordinates to the radar Cartesian coordinate system. After further preprocessing, a gridded image is obtained.

10. A slope rockfall trajectory prediction system based on radar and camera, comprising an interconnected memory and a processor, wherein the memory stores a computer program, characterized in that, The computer program, when run by a processor, performs the steps of the method as described in any one of claims 1-9.

Images