Radar and camera-based slope rockfall movement track prediction method and system

Abstract

The invention discloses a slope rockfall movement track prediction method and system based on a radar and a camera. The method comprises the following steps: S1, acquiring gradient and slope direction information of any point on a slope surface; s2, acquiring a slope image of the slope, identifying slope features, and automatically endowing each grid with an initial slope coefficient according to a preset slope feature and slope coefficient comparison table; s3, on the basis of the slope surface coefficient and a rockfall kinematic model, predicting a movement track and a falling point of the rockfall; s4, comparing the real-time position information of the rockfall with the corresponding position information in the predicted motion trail; if the position difference value is within a preset threshold range, entering S5; if the position difference value exceeds a preset threshold range, judging that the slope environment is remarkably changed, and returning to S2; and S5, constructing an objective function by taking the slope surface coefficient as a variable, performing iterative optimization search, searching an optimal slope surface coefficient combination, and executing the step S3 again. The method has the advantages of high prediction precision and the like.

Description

Technical Field

[0001] This invention mainly relates to the field of rockfall monitoring technology, specifically to a method and system for predicting the trajectory of rockfalls on slopes based on radar and cameras. Background Technology

[0002] Methods for predicting rockfall trajectories can generally be divided into two categories: empirical methods based on experimental research and methods based on theoretical derivation. The former mainly includes field experimental research and indoor scale-scale model experiments. Theoretical derivation methods refer to establishing reasonable mathematical calculation models based on kinematic and dynamic theories to fit the trajectory of falling rocks along the slope. The trajectory of falling rocks is generally divided into three stages: rolling (sliding) motion, free fall motion, and collision with the slope. The main factors currently recognized as influencing the trajectory of falling rocks on slopes are: slope shape (slope gradient, slope length), geomechanical properties of the slope surface (such as slope roughness, slope vegetation coverage, softness of the slope cover soil, and hardness of the exposed bedrock), and the size and shape of the falling rocks.

[0003] A slope rockfall monitoring radar is a radar system used to monitor slope rockfalls. It primarily works by emitting electromagnetic wave signals into the monitoring area and receiving signals reflected back from falling rocks. By measuring the time delay of the reflected echo, the distance to the falling rocks can be accurately measured. Clutter suppression improves the signal-to-clutter ratio, Doppler processing measures the velocity of the falling rocks, and MIMO imaging technology measures the angle of the falling rocks, enabling high-precision measurement of rockfalls.

[0004] The technology currently related to this patent application is a three-dimensional analysis method for the trajectory of rocks of arbitrary shapes falling on soil slopes. The method steps are as follows: a three-dimensional model of rocks of arbitrary shapes and the slope surface is established in the system coordinate system, and a contact search algorithm is used to determine whether the rocks are in contact with the slope surface at each moment, and the contact force at the time of contact is calculated through the contact model; the motion equation of the rocks is established based on the contact force and gravity, and the motion method is used to solve the motion method to obtain the trajectory of the rocks.

[0005] In existing research methods, the slope coefficient is obtained by back-calculation through multiple experiments or by setting it manually based on experience. Using back-calculation through multiple experiments requires a lot of manpower, and the experiment needs to be repeated after changing the scene to obtain the correct slope coefficient. Using manual experience values ​​requires manual on-site surveys and its accuracy is not high. Both of these existing methods require a lot of manpower and have low adaptability.

[0006] Existing research methods rarely use external feedback data to correct the slope coefficient. When the slope coefficient is not well adapted to the monitoring scenario, it will cause a mismatch in the prediction model, resulting in a large difference between the predicted rockfall trajectory and the actual trajectory.

[0007] Existing research methods typically use contact sensors to acquire data on rockfall movement. However, this method has a limited monitoring range and is only applicable to slope scenarios that are accessible to humans. Summary of the Invention

[0008] To address the technical problems existing in the prior art, this invention provides a method and system for predicting the trajectory of falling rocks on slopes using radar and cameras with high accuracy.

[0009] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows: A method for predicting the trajectory of falling rocks on slopes based on radar and cameras includes the following steps: S1. Establish a radar rectangular coordinate system with the radar as the origin, obtain a digital elevation model with the radar as the origin, and obtain the slope and aspect information of any point on the slope through the digital elevation model; S2. Acquire slope images of the monitored slope using a camera, perform gridding processing on the slope images, identify the slope features of each grid based on a deep learning algorithm, and automatically assign an initial slope coefficient to each grid according to a preset slope feature and slope coefficient comparison table. S3. Based on the slope and aspect information obtained in S1 and the slope coefficient obtained in S2, combined with the rockfall kinematics model, the trajectory and landing point of the falling rock are predicted. S4. Utilize radar to measure the real-time location information of the falling rocks, and compare the real-time location information with the corresponding location information in the motion trajectory predicted in S3. If the location difference value is within the preset threshold range, proceed to S5; if the location difference value exceeds the preset threshold range, it is determined that the slope environment has changed significantly, and return to S2 to reacquire the slope image and slope coefficient. S5. Construct an objective function using the slope coefficient as a variable, perform iterative optimization search within the preset parameter range to find the optimal combination of slope coefficients that minimizes the objective function value, and re-execute S3 based on the optimal combination of slope coefficients to update the predicted rockfall trajectory.

[0010] Preferably, the slope characteristics described in step S2 include one or more of vegetation coverage, vegetation type, soil type, and rock surface smoothness.

[0011] Preferably, the slope coefficient in step S2 includes the normal restoration coefficient. Tangential coefficient of restitution and coefficient of friction .

[0012] Preferably, the rockfall kinematic model in step S3 includes a sliding motion model, a projectile motion model, and a collision motion model; The sliding motion model is as follows:

[0013] ; Where 'a' is the acceleration of the falling rock as it slides. The slope angle is g; g is the acceleration due to gravity. The coefficient of friction, The velocity at the end of the rockfall slide. The initial velocity of the sliding rock is... The sliding time; The projectile motion model is as follows:

[0014] ; ; in The horizontal velocity component during the oblique throwing process of the falling rock. The horizontal velocity component at the end of the rockfall slide. This represents the vertical velocity component during the oblique throwing process of the falling rock. This represents the vertical velocity component at the end of the rockfall slide. The height-axis velocity component during the oblique throwing process of the falling rock. The height-axis velocity component represents the velocity at the end of the rockfall slide. For projectile motion; The collision motion model is as follows:

[0015]

[0016] in Let vr be the resultant velocity of the falling rock in the radar XOY plane after the collision, and vr be the resultant velocity of the falling rock in the radar XOY plane before the collision. The velocity of the falling rock in the height direction before the collision. The velocity of the falling rock in the height direction after the collision; The normal restitution coefficient; This is the tangential recovery coefficient. Preferably, after the rockfall collision, the normal component of the rockfall velocity after the collision is used. and tangential components To determine whether the falling rock continues to be thrown at an angle or slides, if If the rock falls, it will slide; otherwise, continue throwing at an angle, repeating the cycle until the rock stops moving, thus obtaining the landing point of the rock.

[0017] Preferably, in step S4, the position difference value is the distance between the real-time position information and the corresponding position information in the predicted motion trajectory; wherein the preset threshold range is 0.5m-2m.

[0018] Preferably, in step S5, the objective function for:

[0019] in The normal restitution coefficient; The tangential restitution coefficient; The coefficient of friction; Real-time location information of falling rocks as measured by radar; This refers to the corresponding position information in the predicted motion trajectory.

[0020] Preferably, the iterative optimization search in step S5 adopts a full-range step search method.

[0021] Preferably, in step S2, a slope surface image of the monitored slope is acquired using a camera. Then, the slope image information The image information is obtained by transforming the coordinates to the radar Cartesian coordinate system. After further preprocessing, a gridded image is obtained.

[0022] The present invention also discloses a slope rockfall monitoring and motion trajectory prediction system based on radar and camera, including a memory and a processor connected to each other. The memory stores a computer program, and the computer program executes the steps of the method described above when run by the processor.

[0023] Compared with the prior art, the advantages of the present invention are as follows: This invention uses a high-resolution camera to capture images of the monitored area, combines image recognition technology to extract slope features in a grid pattern, and automatically sets the slope coefficient based on a slope feature and slope coefficient comparison table. This method can adaptively register initial parameters for different monitoring scenarios without manual adjustment, improving applicability and saving labor costs. Utilizing the high-precision measurement characteristics of radar, real-time radar measurement information is compared with predicted rockfall trajectories. When the difference is significant but within a reasonable range (i.e., within a preset threshold range), an objective function is established for iterative optimization to find the optimal slope coefficient, and a new model is built for trajectory prediction, thereby improving the accuracy of rockfall trajectory prediction. When the difference exceeds a reasonable range (i.e., exceeds the preset threshold range), it indicates a significant change in the slope environment. The camera is then activated to reacquire image information of the monitored slope, and the initial slope coefficient is reloaded to rebuild the model for trajectory prediction. Leveraging the non-contact and wide-range measurement characteristics of radar, it can achieve large-scale slope rockfall monitoring and trajectory prediction, especially suitable for slope scenarios that are difficult for humans to climb. Attached Figure Description

[0024] Figure 1 This is a flowchart of an embodiment of the radar and camera-based slope rockfall trajectory prediction method of the present invention.

[0025] Figure 2 This is a diagram showing the relationship between the radar rectangular coordinate system and the slope coordinate system in this invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown in the figure, the slope rockfall trajectory prediction method based on radar and camera provided in this embodiment of the invention includes the following steps: S1. Establish a radar rectangular coordinate system with the radar as the origin, obtain a digital elevation model with the radar as the origin, and obtain the slope and aspect information of any point on the slope through the digital elevation model; Specifically, a radar rectangular coordinate system XYZ is established with the radar as the origin, and point P is a point on the slope with a velocity of... In the XYZ coordinate system, the coordinates are ; Obtain a digital elevation model with the radar as the origin, and obtain slope and aspect information, such as slope angle and aspect angle, at any point on the slope through the digital elevation model.

[0028] To facilitate the analysis of the trajectory of falling rocks on the slope, a slope coordinate system is established with point P as the origin. , The axis is parallel to the slope aspect, and the slope angle at point P is... The slope angle is Its velocity in the slope coordinate system is The relationship between the two coordinate systems is as follows: Figure 2 As shown.

[0029] S2. Acquire slope surface images of the monitored slope using a camera, perform gridding processing on the slope surface images, and identify the slope surface features of each grid based on a deep learning algorithm. According to a pre-defined slope surface feature and slope coefficient comparison table, automatically assign an initial slope coefficient to each grid; the specific steps are as follows: S201. Acquire slope image information of the monitored area using a high-resolution camera. ; S202. Transfer slope image information The image information is obtained by transforming the coordinates to a coordinate system with the radar as the reference point. ; Specifically, assuming point P is a point on the slope, its position in the radar coordinate system is... The coordinates in the camera are The position in the image coordinate system is ; The transformation relationship between the radar coordinate system and the camera coordinate system is shown below:

[0030] Where R is a 3x3 rotation matrix and T is a 3x1 translation matrix. The extrinsic parameter matrix of the camera; Transformation matrix from camera coordinate system to image coordinate system (The K matrix is ​​the camera intrinsic parameter matrix.) and f is the camera's focal length, and dx and dy are the length and width of a single pixel in the image plane, respectively. The coordinates of the origin of the image coordinate system in the pixel coordinate system.

[0031] The transformation relationship between the camera coordinate system and the image coordinate system is shown below:

[0032] In summary, the transformation relationship from the image coordinate system to the radar coordinate system is as follows:

[0033]

[0034] in Represents the matrix Take the first 3 rows and 3 columns. ; This represents the position along the z-axis in the camera coordinate system. For intermediate transformation variables; S203. Will After preprocessing such as size adjustment and normalization, the YOLOv5 model is used to identify the slope features of each grid (including vegetation coverage and type, soil type, and rock surface smoothness). YOLOv5 adopts a single-stage detection framework, which simultaneously achieves feature extraction, target localization, and classification through an end-to-end convolutional neural network. S204. Based on the slope characteristics and slope coefficient comparison tables shown in Tables 1-3, an initial slope coefficient is automatically selected for each grid; the initial slope coefficient includes the normal recovery coefficient. Tangential coefficient of restitution and coefficient of friction ; Table 1 Relationship between normal coefficient of restoration and slope characteristics

[0035] Table 2 Relationship between tangential restoration coefficient and slope characteristics

[0036] Table 3 Relationship between friction coefficient and slope characteristics

[0037] S3. Based on the slope and aspect information of any point on the slope obtained in S1 and the slope coefficient obtained in S2, and combined with the rockfall kinematics model, predict the trajectory and impact point of the falling rock; the specific steps are as follows: S301. Based on the transformation relationship between the radar coordinate system and the slope coordinate system, the coordinates of the falling rock on the radar are transformed to the slope coordinate system, and the speed measured by the radar is... Velocity converted to slope coordinate system The transformation relationship is as follows:

[0038]

[0039] S302. Based on the obtained initial slope coefficient, the trajectory of falling rocks is predicted in conjunction with the rockfall motion equation; the trajectory of falling rocks is generally divided into three states: sliding, oblique projection, and collision. The sliding motion model is as follows:

[0040] ; Where g is the acceleration due to gravity. Let be the coefficient of friction, and 'a' be the acceleration of the falling rock as it slides. For the sliding time, The initial velocity, ; The end speed of the slide. .

[0041] After the sliding motion ends, the object typically undergoes projectile motion, with a projectile time of [time missing]. The projectile motion model is as follows:

[0042] ; ; in The horizontal velocity component at the end of the rockfall slide. The horizontal velocity component during the oblique throwing process of the falling rock. The vertical velocity component at the end of the rockfall slide. This represents the vertical velocity component during the oblique throwing process of the falling rock. The height-axis velocity component represents the velocity at the end of the rockfall slide. This represents the height-axis velocity component during the oblique throwing process of falling rocks. After the projectile motion of the falling rock ends, it enters a collision state. The collision motion model is as follows:

[0043]

[0044] Where vr is the resultant velocity of the falling rock in the radar XOY plane before the collision. , Let X be the resultant velocity of the falling rock on the radar XOY plane after the collision. The velocity of the falling rock in the height direction before the collision. The velocity of the falling rock in the height direction after the collision; Normal component of the velocity of falling rocks after collision and tangential components It is calculated by the following formula: ; ; Based on the normal component of the velocity of the falling rock after the collision and tangential components To determine whether the falling rock continues to be thrown at an angle or slides, if If the rock stops moving, it will slide; otherwise, continue the oblique throwing motion, repeating the cycle until the rock stops moving, thus obtaining the predicted position information. ; S4. Utilize radar to measure real-time information on falling rocks (including real-time location information of the falling rocks). (and speed), real-time location information Position information in the motion trajectory predicted by S3 A comparison is performed between the real-time location information and the predicted location information. If the (location difference value) is within the preset threshold range, proceed to S5; if the location difference value exceeds the preset threshold, it is determined that the slope environment has changed significantly, and return to S2, and re-acquire the slope image and slope coefficient of the monitored area through the camera; the preset threshold range is preferably 0.5m-2m, that is, when the difference value does not exceed 0.5m, no optimization adjustment is required, when the difference value is between 0.5m and 2m, proceed to S5, when the difference value exceeds 2m, it is determined that the slope environment has changed significantly, and return to S2, and re-acquire the slope image and slope coefficient of the monitored area through the camera; S5. Construct an objective function using the slope coefficient as a variable, perform iterative optimization search within the preset parameter range to find the optimal combination of slope coefficients that minimizes the objective function value, and re-execute S3 based on the optimal combination of slope coefficients to update the predicted rockfall trajectory.

[0045] Specifically, an objective function is established with slope parameters as variables. Among them, slope parameters The parameter comparison table above has certain limitations; regarding slope parameters... Within the constraints, perform an iterative full-range search with a step size of 0.05 to find the set of slope parameter values ​​that minimizes the objective function. The slope parameters at this point... The value obtained is the optimal value; the optimal value is used to rebuild the prediction model and predict the trajectory of the falling rock.

[0046] This invention uses a high-resolution camera to capture images of the monitored area, combines image recognition technology to extract slope features in a grid pattern, and automatically sets the slope coefficient based on a slope feature and slope coefficient comparison table. This method can adaptively register initial parameters for different monitoring scenarios without manual adjustment, improving applicability and saving labor costs. Utilizing the high-precision measurement characteristics of radar, real-time radar measurement information is compared with predicted rockfall trajectories. When the difference is significant but within a reasonable range (i.e., within a preset threshold range), an objective function is established for iterative optimization to find the optimal slope coefficient, and a new model is built for trajectory prediction, thereby improving the accuracy of rockfall trajectory prediction. When the difference exceeds a reasonable range (i.e., exceeds the preset threshold range), it indicates a significant change in the slope environment. The camera is then activated to reacquire image information of the monitored slope, and the initial slope coefficient is reloaded to rebuild the model for trajectory prediction. Leveraging the non-contact and wide-range measurement characteristics of radar, it can achieve large-scale slope rockfall monitoring and trajectory prediction, especially suitable for slope scenarios that are difficult for humans to climb.

[0047] This invention also discloses a slope rockfall monitoring and trajectory prediction system based on radar and cameras, comprising an interconnected memory and a processor. The memory stores a computer program, which, when run by the processor, executes the steps of the method described above. The prediction system of this invention corresponds to the prediction method described above and also possesses the advantages described above.

[0048] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium includes: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0049] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for predicting the trajectory of falling rocks on slopes based on radar and cameras, characterized in that, Including the following steps: S1. Establish a radar rectangular coordinate system with the radar as the origin, obtain a digital elevation model with the radar as the origin, and obtain the slope and aspect information of any point on the slope through the digital elevation model; S2. Acquire slope images of the monitored slope using a camera, perform gridding processing on the slope images, identify the slope features of each grid based on a deep learning algorithm, and automatically assign an initial slope coefficient to each grid according to a preset slope feature and slope coefficient comparison table. S3. Based on the slope and aspect information obtained in S1 and the slope coefficient obtained in S2, combined with the rockfall kinematics model, the trajectory and landing point of the falling rock are predicted. S4. Utilize radar to measure the real-time location information of the falling rocks, and compare the real-time location information with the corresponding location information in the motion trajectory predicted in S3. If the location difference value is within the preset threshold range, proceed to S5; if the location difference value exceeds the preset threshold range, it is determined that the slope environment has changed significantly, and return to S2 to reacquire the slope image and slope coefficient. S5. Construct an objective function using the slope coefficient as a variable, perform iterative optimization search within the preset parameter range to find the optimal combination of slope coefficients that minimizes the objective function value, and re-execute S3 based on the optimal combination of slope coefficients to update the predicted rockfall trajectory.

2. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1, characterized in that, The slope characteristics mentioned in step S2 include one or more of the following: vegetation coverage, vegetation type, soil type, and rock surface smoothness.

3. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, The slope coefficient mentioned in step S2 includes the normal restoration coefficient. Tangential recovery coefficient and coefficient of friction .

4. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, The rockfall kinematic model in step S3 includes a sliding motion model, a projectile motion model, and a collision motion model. The sliding motion model is as follows: ； Where 'a' is the acceleration of the falling rock as it slides. The slope angle is g; g is the acceleration due to gravity. The coefficient of friction, The velocity at the end of the rockfall slide. Let the initial velocity of the falling rock be denoted as . The sliding time; The projectile motion model is as follows: ; ; in The horizontal velocity component during the oblique throwing process of the falling rock. The horizontal velocity component at the end of the rockfall slide. This represents the vertical velocity component during the oblique throwing process of the falling rock. This represents the vertical velocity component at the end of the rockfall slide. The height-axis velocity component during the oblique throwing process of the falling rock. The height-axis velocity component represents the velocity at the end of the rockfall slide. For projectile motion; The collision motion model is as follows: in Let vr be the resultant velocity of the falling rock in the radar XOY plane after the collision, and vr be the resultant velocity of the falling rock in the radar XOY plane before the collision. The velocity of the falling rock in the height direction before the collision. The velocity of the falling rock in the height direction after the collision; The normal restitution coefficient; This is the tangential recovery coefficient.

5. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 4, characterized in that, After the rockfall collision, based on the normal component of the rockfall velocity after the collision... and tangential components To determine whether the falling rock continues to be thrown at an angle or slides, if If the rock falls, it will slide; otherwise, continue throwing at an angle, repeating the cycle until the rock stops moving, thus obtaining the landing point of the rock.

6. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S4, the position difference value is the distance between the real-time position information and the corresponding position information in the predicted motion trajectory; The preset threshold range is 0.5m-2m.

7. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S5, the objective function for: in The normal restitution coefficient; The tangential restitution coefficient; The coefficient of friction; Real-time location information of falling rocks as measured by radar; This refers to the corresponding position information in the predicted motion trajectory.

8. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 7, characterized in that, The iterative optimization search in step S5 adopts a full-range step search method.

9. The method for predicting the trajectory of falling rocks on slopes based on radar and cameras according to claim 1 or 2, characterized in that, In step S2, a slope surface image of the monitored slope is acquired using a camera. Then, the slope image information The image information is obtained by transforming the coordinates to the radar Cartesian coordinate system. After further preprocessing, a gridded image is obtained.

10. A slope rockfall trajectory prediction system based on radar and camera, comprising an interconnected memory and a processor, wherein the memory stores a computer program, characterized in that, The computer program, when run by a processor, performs the steps of the method as described in any one of claims 1-9.

Images