A non-contact measurement system and method for the three-dimensional trajectory of slope rockfalls considering the motion compensation of airborne platforms

Abstract

The present invention discloses a non-contact measurement system and method for the three-dimensional trajectory of slope rockfalls considering the motion compensation of an airborne platform. The measurement system includes a hardware system and a data processing system. The hardware system consists of a slope optical target and a multi-rotor UAV array. The multi-rotor UAV array is composed of multiple UAVs that capture the slope rockfall area from the front and side. The slope optical target is composed of checkerboard targets distributed on both sides of the rockfall area on the slope. The data processing system includes an airborne platform motion attitude solution module based on the PnP algorithm and a three-dimensional trajectory reconstruction module of slope rockfalls based on deep learning. The present invention has the advantages of non-contact, remote monitoring, and high flexibility, and can be used for the rolling analysis of slope rockfalls with a large field of view.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection of falling rocks, and particularly relates to a non-contact measurement system and method for three-dimensional trajectories of slope falling rocks considering motion compensation of an airborne platform. Background Art

[0002] Collapse and falling rocks are one of the serious geological disasters with strong suddenness and great randomness. Extracting the trajectory of falling rocks and analyzing the motion characteristics of falling rocks are of great significance for disaster prevention and control. Most of the previous studies on the trajectory of collapse and falling rocks were based on two-dimensional and three-dimensional simulation software, with many parameters in the calculation, great simulation difficulty and hard-to-verify accuracy. There were also on-site falling rock rolling tests, in which acceleration sensors were placed in precast test blocks to obtain the motion information of falling rocks, or fixed-position high-speed cameras and other devices were used to monitor slope falling rocks to extract the trajectory of falling rocks. However, the cost of placing sensors in the test blocks was extremely high and they were extremely easy to damage. Devices such as high-speed cameras had problems such as high requirements for the site for erection, difficult layout of wire cables, small camera field of view and difficult adjustment. In terms of the monitoring method, the unmanned aerial vehicle (UAV) technology has gradually become a trend due to its characteristics of high efficiency, high flexibility, strong environmental adaptability and low cost. However, due to the reasons of air flow and its own rotor vibration, it will inevitably generate vibrations, resulting in disturbances in the video images captured by the airborne camera. At the same time, due to the large displacement characteristics of slope falling rocks rolling, there are also situations such as dust and damage, which makes it difficult for traditional algorithms to directly extract the motion trajectory of falling rocks from the image sequence obtained by the monitoring device. Therefore, there is an urgent need for a non-contact slope falling rock tracking trajectory extraction technology based on UAVs. Summary of the Invention

[0003] To solve the problems existing in the prior art, the present invention provides a non-contact measurement system and method for three-dimensional trajectories of slope falling rocks considering motion compensation of an airborne platform. The present invention has the advantages of non-contact and high flexibility in remote monitoring, and can be directly used for the extraction and analysis of the three-dimensional trajectory of slope falling rocks rolling, solving the problems mentioned in the above background art.

[0004] To achieve the above object, the present invention provides the following technical solution: A non-contact measurement system for three-dimensional trajectories of slope falling rocks considering motion compensation of an airborne platform, the system comprising a hardware system and a data processing system; the hardware system consists of a slope optical target and a multi-rotor UAV array, the multi-rotor UAV array is composed of multiple UAVs that shoot the slope rolling rock area from the front and side, and the slope optical target is composed of checkerboard targets distributed on the slopes on both sides of the falling rock area; the data processing system includes an airborne platform motion attitude solution module based on the PnP algorithm and a three-dimensional trajectory reconstruction module of slope falling rocks based on deep learning.

[0005] Preferably, in the slope optical target, the installation position of the optical target is determined on the slope according to the movement range of the slope rockfall. The type and size of the optical target are considered according to the camera range, and the required target is printed and pasted on a steel support of the corresponding size, and fixedly installed at a preset position on the slope;

[0006] In the multi-rotor UAV array, the position of the UAV array is arranged according to the required field of view for shooting the rockfall and the target, the viewing angle parameters of the on-board camera are adjusted, and the multi-view rolling image sequence of the hillside rockfall is captured by the UAV.

[0007] Preferably, the on-board platform motion attitude solution module based on the PnP algorithm includes: detecting the two-dimensional coordinates of the target in the initial frame image using the Shi-Tomasi 2D corner detection algorithm, updating the two-dimensional target point pair coordinates in the next frame using the template matching algorithm, matching with the three-dimensional world coordinates of the target to form a 2D-3D point pair, and using the PnP algorithm to solve and update the camera pose information of a single frame image.

[0008] Preferably, in the on-board platform motion attitude solution module based on the PnP algorithm, optical targets are arranged on the slope according to the movement range of the slope rockfall, the position of the UAV array is arranged according to the required field of view, the internal parameters of the UAV on-board camera are obtained by the checkerboard calibration method, the multi-view rolling image sequence of the hillside rockfall is captured by the UAV, the two-dimensional coordinates of the target in the initial frame image are detected using the Shi-Tomasi 2D corner detection algorithm, and the response function in (x, y) is defined as:

[0009] r(x, y) = min(λ1, λ2)

[0010] where λ1 and λ2 are the eigenvalues of the image gradient covariance matrix respectively. When the response function r is greater than the set threshold N, it is set as a corner point;

[0011] After obtaining the two-dimensional coordinates corresponding to the slope target, matching with the three-dimensional world coordinates of the target to form a 2D-3D point pair, using the PnP algorithm to solve the camera pose information of a single frame image. When the three-dimensional world coordinates of n, n≥4 spatial points, their corresponding two-dimensional image point coordinates and the internal parameters K of the camera are known, the rotation matrix R and the translation vector t can be obtained using the linear camera model; using the template matching algorithm to update the two-dimensional target point pair coordinates in the next frame, and using the PnP algorithm to update the rotation matrix R and the translation vector t matrix of the next frame, and performing this operation on the image sequence in a loop, so as to obtain the long-term pose information of the UAV.

[0012] Preferably, the slope rockfall three-dimensional trajectory reconstruction module based on deep learning includes: inputting a multi-view image sequence of slope rockfall rolling captured by a drone, using a pre-trained yolov8 deep learning neural network to track and identify the entire process of the slope rockfall movement trajectory to obtain the two-dimensional trajectory of the rockfall, and performing triangulation on the multi-view two-dimensional trajectory to obtain the three-dimensional rolling trajectory of the slope rockfall.

[0013] Preferably, in the slope rockfall three-dimensional trajectory reconstruction module based on deep learning, a slope rockfall movement image database is built, the neural network is trained, and the loss function consists of multiple parts, including the classification loss VFL Loss and the regression loss CIOU Loss;

[0014] Input the image sequence captured by the drone into the trained neural network, and the neural network outputs the detection bounding box information of the moving rockfall in each frame of the image; then use the upper left pixel coordinates (x i , y i ) of the detection bounding box and the pixel lengths in the horizontal and vertical directions of the detection bounding box to calculate the center pixel coordinates (x * , y * ), that is:

[0015]

[0016] where: l is the pixel length of the detection box in the horizontal direction of the image, and b is the pixel length of the detection box in the vertical direction of the image; thus, the two-dimensional pixel trajectory of the rockfall in the video captured by the drone is obtained;

[0017] By obtaining the two-dimensional pixel trajectories of the rockfall in the videos captured by two or more drones at different angles in the drone array data, and at the same time using the onboard platform motion attitude solution module based on the PnP algorithm to obtain the 6-degree-of-freedom pose information of each frame of the drone's onboard camera;

[0018] Given the two-dimensional coordinates (u l , v l ) and (u r , v r ) of the same rockfall in the left and right onboard cameras in the video frames at the same moment, as well as the corresponding drone poses, use the SVD decomposition to solve the multi-view 2D-3D point pair equation to obtain the true three-dimensional coordinates of the rockfall in this frame, and loop this step for each video frame to obtain the three-dimensional rolling trajectory of the rockfall.

[0019] Preferably, the classification VFL Loss formula is expressed as:

[0020]

[0021] where q is the intersection over union of the predicted box and the true box, p is the probability, and α and γ are modulation factors;

[0022] The regression loss CIoU Loss formula is expressed as:

[0023]

[0024] where IoU is the intersection over union, D is the distance between the centers of the predicted box and the ground truth box, C is the diagonal distance of the minimum bounding rectangle, and β is the correction factor.

[0025] Preferably, the multi-view 2D-3D point pair equation is specifically as follows:

[0026]

[0027] where Z left and Z Right are the z-direction coordinates of the left and right camera coordinate systems respectively; K l , R l , t l represent the internal parameter matrix, rotation vector, and displacement matrix of the left camera respectively; K r , R r , t r represent the internal parameter matrix, rotation vector, and displacement matrix of the right camera respectively; (X, Y, Z) are the true three-dimensional coordinates of the rockfall.

[0028] On the other hand, to achieve the above object, the present invention also provides the following technical solution: A non-contact measurement method for the three-dimensional trajectory of slope rockfall considering the motion compensation of the airborne platform, including the following steps:

[0029] 1) UAV airborne camera calibration: Use the airborne camera to capture the corresponding high-precision checkerboard images, and calculate and obtain the internal parameters of the airborne camera;

[0030] 2) Obtain the UAV pose: Arrange slope optical targets on the slope according to the motion range of the slope rockfall, arrange the positions of the multi-rotor UAV array according to the required field of view range, use the airborne platform motion attitude solution module based on the PnP algorithm, use the UAV to capture the multi-view rolling image sequence of the slope rockfall, detect the two-dimensional coordinates of the target with known three-dimensional coordinates in the first frame, combine the obtained camera internal parameters to solve the UAV pose information, use the tracking algorithm to update the target coordinates in the next frame, solve and update the UAV pose information, and obtain the pose information of each frame when the UAV shoots the video array;

[0031] 3) Real-time tracking of the three-dimensional trajectory of falling rocks: Based on the three-dimensional trajectory reconstruction module of falling rocks on the slope using deep learning, a training database for the falling rock slope rolling tracking algorithm is established, and a deep learning neural network is built. The image sequence captured by the airborne camera is input into the network to track the falling rocks rolling on the slope, and the two-dimensional trajectory coordinates of the falling rocks are obtained. The tracking results of multiple airborne cameras are extracted, combined with the internal parameters of the cameras and the corresponding UAV poses, and the three-dimensional coordinate trajectory of the falling rocks on the slope is calculated using SVD. By looping the above steps for each video frame, the three-dimensional trajectory of the falling rocks can be obtained.

[0032] The beneficial effects of the present invention are:

[0033] 1) The method of the present invention uses the PnP algorithm and the corner tracking algorithm to automatically calculate the pose information of the UAV array from the image sequence, improving the accuracy of remote monitoring based on the airborne camera of the UAV.

[0034] 2) By establishing a training database for the falling rock slope rolling tracking algorithm, the present invention realizes the two-dimensional trajectory tracking of falling rocks on the slope based on deep learning; based on the multi-view two-dimensional trajectory tracking results of the UAV array, the high-precision reconstruction of the three-dimensional trajectory of the falling rocks rolling on the slope is realized.

[0035] 3) The present invention has the advantages of non-contact, remote monitoring and high flexibility, reducing the difficulty of monitoring and analyzing the rolling of falling rocks on the slope under complex mountain conditions, and can be used for the analysis of the rolling of falling rocks on the large field of view slope. Description of the Drawings

[0036] Figure 1 It is the layout and schematic diagram of the non-contact measurement system for the three-dimensional trajectory of falling rocks on the slope considering the motion compensation of the airborne platform in the embodiment of the present invention;

[0037] Figure 2 It is the flowchart of the data processing system module in the embodiment of the present invention;

[0038] Figure 3 It is the schematic diagram of the corner detection result of the falling rock target on the slope in the embodiment of the present invention;

[0039] Figure 4 It is the schematic diagram of the pose information of each frame of the UAV calculated in the embodiment of the present invention;

[0040] Figure 5 It is the schematic diagram of the training database of the slope rolling tracking algorithm in the embodiment of the present invention;

[0041] Figure 6 It is the architecture diagram of the deep learning network in the embodiment of the present invention;

[0042] Figure 7 It is the schematic diagram of the tracking of the falling rocks rolling on the slope in the embodiment of the present invention;

[0043] Figure 8Schematic diagram of three-dimensional coordinate trajectory calculation of slope rockfall in the embodiments of the present invention. Detailed implementation manners

[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] Please refer to Figures 1 - 2 , the present invention provides a technical solution: a non-contact measurement system for three-dimensional trajectory of slope rockfall considering motion compensation of an airborne platform, as Figure 1 shown, which is composed of a hardware system and a data processing system. The hardware system includes:

[0046] 1) Slope optical targets: The slope optical targets are composed of checkerboard targets distributed on both sides of the rockfall area on the slope. Determine the installation positions of the optical targets on the slope according to the motion range of the slope rockfall, consider the type (circular or checkerboard) and size of the optical targets according to the camera range, print the required targets and paste them on steel brackets of corresponding sizes, and fixedly install them at preset positions on the slope.

[0047] 2) Multi-rotor UAV array: The multi-rotor UAV array is composed of multiple UAVs that shoot the rockfall area on the slope from the front and side, and is used for shooting videos of the motion of slope rockfall. Arrange the positions of the UAV array according to the required field of view for shooting the rockfall and the targets, adjust the viewing angle parameters of the airborne cameras, and use the UAVs to shoot multi-view image sequences of the rockfall rolling on the hillside.

[0048] As Figure 2 shown, the data processing system includes:

[0049] 1) Airborne platform motion attitude calculation module based on the PnP algorithm: Use the Shi-Tomasi 2D corner detection algorithm to detect the two-dimensional coordinates of the targets in the initial frame image, use the template matching algorithm to update the coordinates of the two-dimensional target point pairs in the next frame, match them with the three-dimensional world coordinates of the targets to form 2D-3D point pairs, and use the PnP algorithm to calculate and update the camera pose information of a single frame image.

[0050] 2) Three-dimensional trajectory reconstruction module of slope rockfall based on deep learning: Input the multi-view image sequences of the rockfall rolling on the hillside captured by the UAVs, use the pre-trained yolov8 deep learning network to track and identify the whole process of the motion trajectory of the slope rockfall to obtain the two-dimensional trajectory of the rockfall, and perform triangulation on the multi-view two-dimensional trajectories to obtain the three-dimensional trajectory of the slope rockfall rolling.

[0051] In the motion attitude (multi-degree-of-freedom attitude of UAV) solution module of the airborne platform based on the PnP algorithm, optical targets are arranged on the slope according to the moving range of the slope rockfall, the UAV array positions are arranged according to the required field of view range, the internal parameters of the UAV airborne camera are obtained by the checkerboard calibration method, the UAV is used to capture a sequence of rolling images of the slope rockfall from multiple perspectives, the Shi-Tomasi 2D corner detection algorithm is used to detect the two-dimensional coordinates of the targets in the initial frame image, and the response function is defined in (x, y):

[0052] r(x, y) = min(λ1, λ2)

[0053] where λ1 and λ2 are the eigenvalues of the image gradient covariance matrix respectively. When the response function r is greater than the set threshold N, it can be set as a corner point.

[0054] Furthermore, after obtaining the corresponding two-dimensional coordinates, they are matched with the three-dimensional world coordinates of the targets to form 2D-3D point pairs, and the PnP algorithm is used to solve the camera pose information of a single frame image. When the three-dimensional world coordinates of n (n≥4) spatial points, their corresponding two-dimensional image point coordinates, and the internal parameters K of the camera are known, the rotation matrix R and translation vector t in the formula can be obtained using the linear camera model.

[0055] Furthermore, the template matching algorithm is used to update the two-dimensional target point pair coordinates in the next frame, and the PnP algorithm is used to update the rotation matrix R and the t translation vector matrix of the next frame. This operation is cycled for the image sequence to obtain the long-term pose information of the UAV.

[0056] In the three-dimensional trajectory reconstruction module of the slope rockfall based on deep learning, first, the number and shooting position angles of the required UAVs are determined according to the ROI. Then, the UAV airborne camera is used to capture the process of the rockfall rolling on the slope. Secondly, an image database of the slope rockfall motion is built, and a lightweight deep learning neural network is trained. The loss function consists of multiple parts, including the classification loss VFL Loss and the regression loss CIOU Loss.

[0057] VFL Loss is expressed as:

[0058]

[0059] where: q is the intersection over union of the predicted box and the ground truth box, p is the probability, and α and γ are modulation factors.

[0060] CIoU Loss is expressed as:

[0061]

[0062] where IoU is the intersection over union, D is the distance between the centers of the predicted box and the ground truth box, C is the diagonal distance of the minimum bounding rectangle, and β is a correction factor.

[0063] Input the image sequence captured by the drone into the trained neural network, and the neural network outputs the detection bounding box information of the moving falling rocks in each frame of the image. Then, use the pixel coordinates (x i , y i ) at the upper left corner of the detection bounding box and the pixel lengths in the horizontal and vertical directions of the detection bounding box to calculate the central pixel coordinates (x * , y * ), that is:

[0064]

[0065] where: l is the pixel length of the detection box in the horizontal direction of the image, and b is the pixel length of the detection box in the vertical direction of the image; furthermore, obtain the two-dimensional pixel trajectory of the falling rocks in the video captured by the drone.

[0066] By obtaining the two-dimensional pixel trajectories of the falling rocks in the videos captured by two or more drones from different angles, and at the same time using the onboard platform motion attitude solution module based on the PnP algorithm to obtain the 6-degree-of-freedom pose information of each frame of the drone's onboard camera.

[0067] Furthermore, given the two-dimensional coordinates (u l , v l ) and (u r , v r ) of the same falling rock in the left and right onboard cameras at the same video frame and the corresponding drone poses, obtain the multi-view 2D-3D point pair equation (two-dimensional - three-dimensional coordinate system conversion equation):

[0068]

[0069] where, Z left and Z Right are the z-direction coordinates of the left and right camera coordinate systems respectively; K l , R l , t l represent the internal parameter matrix, rotation vector, and displacement matrix of the left camera respectively; K r , R r , t r represent the internal parameter matrix, rotation vector, and displacement matrix of the right camera respectively; (X, Y, Z) is the true three-dimensional coordinate of the falling rock

[0070] Solve the above equation through SVD decomposition, and the true three-dimensional coordinates of the falling rock in this frame can be obtained. Loop the above steps for each video frame to obtain the three-dimensional rolling trajectory of the falling rock.

[0071] The present invention uses a lightweight deep learning network to identify multi-view falling-rock slope motion images captured, realizes the extraction of two-dimensional coordinates of multi-view falling-rock rolling, and through inputting the pose information of a drone platform updated dynamically from multiple views, reconstructs the three-dimensional trajectory of falling-rock rolling on the slope with high precision. The present invention has the advantages of non-contact, remote monitoring and high flexibility, and can be used for the analysis of falling-rock rolling on large-field slopes.

[0072] Combined with Figures 3 - 7 , through a field falling-rock slope rolling impact test to illustrate the specific implementation process of the technical solution of this embodiment, a non-contact measurement method for the three-dimensional trajectory of falling rocks on a slope considering the motion compensation of an airborne platform, the specific steps are as follows:

[0073] 1) Calibration of the airborne camera of the drone. Use the airborne camera to capture corresponding high-precision checkerboard images, and calculate and obtain the internal parameters of the airborne camera.

[0074] 2) Obtain the pose of the drone. Arrange slope optical targets on the slope according to the motion range of falling rocks on the slope, arrange the positions of the multi-rotor drone array according to the required field of view range, and use the motion attitude solution module of the airborne platform based on the PnP algorithm to use the drone to capture a sequence of multi-view images of falling rocks rolling on the hillside; detect the two-dimensional coordinates of the targets with known three-dimensional coordinates in the first frame, as Figure 3 shown, combined with the obtained internal parameters of the camera to solve the pose information of the drone, and use the tracking algorithm to update the target coordinates in the next frame, solve and update the pose information of the drone, and obtain the pose information of each frame when the drone is shooting the video array, as Figure 4 shown.

[0075] 3) Real-time tracking of the three-dimensional trajectory of falling-rock motion. Based on the three-dimensional trajectory reconstruction module of falling rocks on the slope using deep learning, establish a training database for the tracking algorithm of falling-rock rolling on the slope, as Figure 5 shown, build a deep learning neural network, as Figure 6 shown, input the image sequence captured by the airborne camera into the network, track the falling rocks rolling on the slope, as Figure 7 shown, and obtain the two-dimensional trajectory coordinates of the falling-rock rolling. Extract the tracking results of 2 airborne cameras, combine the internal parameters of the camera and the corresponding pose of the drone, and solve the three-dimensional coordinate trajectory of the falling rocks on the slope, as Figure 8 shown, and loop the above steps for each video frame to obtain the three-dimensional trajectory of the falling-rock rolling.

[0076] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A non-contact measurement system for three-dimensional trajectory of rockfall on slope considering motion compensation of airborne platform, characterized in that: The system includes a hardware system and a data processing system; the hardware system is composed of a slope optical target and a multi-rotor drone array, the multi-rotor drone array is composed of multiple drones that shoot the rockfall area on the slope from the front and side, and the slope optical target is composed of checkerboard targets distributed on the slopes on both sides of the rockfall area; the data processing system includes an airborne platform motion posture solution module based on the PnP algorithm and a slope rockfall three-dimensional trajectory reconstruction module based on deep learning; The airborne platform motion posture solving module based on the PnP algorithm includes: using the Shi-Tomasi2D corner point detection algorithm to detect the two-dimensional coordinates of the target in the initial frame image, using the template matching algorithm to update the two-dimensional target point pair coordinates in the next frame, matching with the three-dimensional coordinates of the target world to form a 2D-3D point pair, and using the PnP algorithm to solve and update the camera posture information of the single frame image; In the slope rockfall 3D trajectory reconstruction module based on deep learning, a slope rockfall motion image database is built and a neural network is trained. The loss function consists of multiple parts, including classification loss VFL Loss and regression loss CIOU Loss. The image sequence captured by the drone is input into the trained neural network, and the neural network outputs the detection bounding box information of the moving rockfall in each frame of the image; then the pixel coordinates of the upper left corner of the detection bounding box (x i ,y i ) and the pixel lengths of the detection bounding box in the horizontal and vertical directions to calculate the center pixel coordinates (x * ,y * ),Right now: Where: l is the pixel length of the detection frame in the horizontal direction of the image, and b is the pixel length of the detection frame in the vertical direction of the image; thus, the two-dimensional pixel trajectory of the falling rock in the video shot by the drone is obtained; The two-dimensional pixel trajectory of falling rocks in the videos shot by two or more drones at different angles is obtained from the drone array data, and the 6-DOF pose information of each frame of the drone airborne camera is obtained by using the airborne platform motion posture solution module based on the PnP algorithm; It is known that the two-dimensional coordinates (u l ,v l ) and (u r ,v r ) and the corresponding UAV posture, SVD decomposition is used to solve the multi-eye 2D-3D point pair equation to obtain the real 3D coordinates of the falling rock in that frame. This step is repeated for each video frame to obtain the 3D trajectory of the falling rock.

2. The non-contact measurement system for three-dimensional trajectory of rockfall on slope considering airborne platform motion compensation according to claim 1 is characterized in that: In the slope optical target, the installation position of the optical target is determined on the slope according to the range of rockfall on the slope, the type and size of the optical target are considered according to the camera range, the required target is printed and pasted on the steel bracket of the corresponding size, and fixedly installed at the preset position on the slope; In a multi-rotor UAV array, the positions of the UAV array are arranged according to the field of view required for photographing falling rocks and targets, the viewing angle parameters of the onboard cameras are adjusted, and UAVs are used to shoot multi-perspective image sequences of rolling falling rocks on hillsides.

3. The non-contact measurement system for three-dimensional trajectory of rockfall on slope considering airborne platform motion compensation according to claim 1 is characterized in that: In the airborne platform motion posture solution module based on the PnP algorithm, optical targets are arranged on the slope according to the range of rockfall movement, and the UAV array position is arranged according to the required field of view. The internal parameters of the UAV airborne camera are obtained by the checkerboard calibration method. The UAV is used to shoot a multi-view hillside rockfall rolling image sequence, and the Shi-Tomasi2D corner detection algorithm is used to detect the two-dimensional coordinates of the target in the initial frame image, which is defined in the (x, y) response function: r(x,y)=min(λ1,λ2) Among them, λ1 and λ2 are the eigenvalues ​​of the image gradient covariance matrix respectively. When the response function r is greater than the set threshold N, it is set as a corner point; After obtaining the two-dimensional coordinates corresponding to the slope target, they are matched with the three-dimensional coordinates of the target world to form a 2D-3D point pair. The PnP algorithm is used to solve the camera pose information of a single frame image. When the three-dimensional world coordinates of n, n≥4 spatial points and their corresponding two-dimensional image point coordinates and the camera internal parameters K are known, the linear camera model can be used to obtain the rotation matrix R and translation vector t. The template matching algorithm is used to update the two-dimensional target point pair coordinates in the next frame, and the PnP algorithm is used to update the rotation matrix R and translation vector t matrix of the next frame. This operation is repeated for the image sequence to obtain the long-term pose information of the drone.

4. The non-contact measurement system for three-dimensional trajectory of rockfall on slope considering airborne platform motion compensation according to claim 1 is characterized in that: The deep learning-based slope rockfall 3D trajectory reconstruction module includes: inputting a multi-perspective hillside rockfall rolling image sequence shot by a drone, using a pre-trained yolov8 deep learning neural network to track and identify the motion trajectory of the slope rockfall throughout the entire process to obtain the two-dimensional trajectory of the rockfall, and performing triangulation measurement on the multi-view two-dimensional trajectory to obtain the three-dimensional trajectory of the slope rockfall.

5. The non-contact measurement system for three-dimensional trajectory of rockfall on slope considering airborne platform motion compensation according to claim 1 is characterized in that: The classification VFL Loss formula is expressed as: Among them, q is the intersection-over-union ratio of the predicted box and the true box, p is the probability, and α and γ are modulation factors; The regression loss CIoU Loss formula is expressed as: Among them, IoU is the intersection over union ratio, D is the distance between the center point of the predicted box and the real box, C is the diagonal distance of the minimum enclosing rectangle, and β is the correction factor.

6. The non-contact measurement system for three-dimensional trajectory of rockfall on slope considering airborne platform motion compensation according to claim 1 is characterized in that: The multi-eye 2D-3D point pair equation is as follows: Among them, Z left and Z Right are the z-direction coordinates of the camera coordinate systems of the left and right cameras respectively; K l , R l ,t l Respectively represent the intrinsic parameter matrix, rotation vector, and displacement matrix of the left camera; K r , R r ,t r They represent the intrinsic parameter matrix, rotation vector, and displacement matrix of the right camera respectively; (X, Y, Z) are the real three-dimensional coordinates of the falling rock.

7. A measurement method of a non-contact measurement system for three-dimensional trajectory of falling rocks on a slope taking into account motion compensation of an airborne platform according to any one of claims 1 to 6, characterized in that: The steps include: 1) UAV airborne camera calibration: Use the airborne camera to shoot the corresponding high-precision chessboard image and calculate the internal parameters of the airborne camera; 2) Obtaining the UAV posture: Arrange the slope optical targets on the slope according to the range of the rockfall movement, arrange the multi-rotor UAV array position according to the required field of view, and use the airborne platform motion posture solution module based on the PnP algorithm to shoot a multi-view hillside rockfall rolling image sequence with a UAV. In the first frame, detect the two-dimensional coordinates of the target with known three-dimensional coordinates, and solve the UAV posture information in combination with the obtained camera internal parameters. In the next frame, use the tracking algorithm to update the target coordinates, solve and update the UAV posture information, and obtain the posture information of each frame of the UAV when shooting the video array; 3) Real-time tracking of the three-dimensional trajectory of falling rocks: A deep learning-based three-dimensional trajectory reconstruction module for falling rocks on slopes is used to establish a training database for the rockfall slope rolling tracking algorithm, build a deep learning neural network, input the image sequence taken by the airborne camera into the network, track the rolling rocks on the slope, and obtain the two-dimensional trajectory coordinates of the rockfall. The tracking results of multiple airborne cameras are extracted, combined with the camera intrinsic parameters and the corresponding drone posture, and the SVD is used to solve the three-dimensional coordinate trajectory of the rockfall on the slope. The above steps are repeated for each video frame to obtain the three-dimensional trajectory of the rockfall.

Images