River surface flow velocity field online measurement method based on image-control-free unmanned aerial vehicle

Abstract

The invention discloses an image-control-free unmanned aerial vehicle-based river surface flow velocity field online measurement method, which combines video flow measurement and image-control-free technologies with an unmanned aerial vehicle platform, can complete online measurement of river surface flow velocity without arranging image control points, and is suitable for river surface flow velocity field measurement in a long river reach or an emergency scene. The unmanned aerial vehicle flow measurement system shows good adaptability and emergency performance, can accurately measure section flow velocity field distribution, and is low in cost compared with other types of unmanned aerial vehicle flow measurement systems.

Description

Technical Field

[0001] The present invention relates to the technical field of water conservancy measurement, and in particular to an online measurement method for a river surface velocity field based on an image-free controlled unmanned aerial vehicle. Background Art

[0002] As one of the most basic data in hydrological research, river surface velocity has always been the focus of research. The exchange processes between water and air, such as material transfer and heat diffusion, are all affected by the river surface velocity. The movement direction and speed of waste and pollutants in the river, and the erosion of river banks by sediment are also related to it. The accurate measurement of the river surface velocity field has always been one of the difficult problems in hydrological and water conservancy research.

[0003] In terms of hydrological emergency monitoring, in the past 20 years, due to the limitations of technical level and the lack of attention paid to the water conservancy industry in the past, many domestic hydrological stations mostly use contact flow meters for flow measurement. The main reason is that contact flow meters can obtain water flow data with high measurement accuracy, but the equipment entering the water also brings certain hidden dangers to the staff, and is not suitable for water flow measurement during high floods. In recent years, with the development of technology, a variety of river flow emergency measurement technologies have emerged. 1) Navigation Doppler ADCP unmanned boat. In order to reduce the safety risks during manual operations, hydrological staff began to use unmanned boats equipped with ADCP for flow measurement. The principle of ADCP flow measurement is the same as the basic principle of traditional manual ship measurement, bridge measurement, cableway measurement and wading measurement: multiple vertical lines are laid on the flow measurement section, the water depth is measured at each vertical line, and the flow velocity at multiple points is measured to obtain the average flow velocity of the vertical line. ADCP can measure many vertical lines and many measuring points on each vertical line. However, the use of ADCP also has certain disadvantages. First, it is time-consuming and labor-intensive and relies on manual work. Second, it is risky and limited during high flood seasons. Third, if the sediment content in the river is particularly high, ADCP cannot be used for emergency flow measurement. 2) Radar method. The main representatives of the radar method are side-scan radar and handheld radar gun. The radar method is a type of hydrological instrument that uses radar Bragg scattering and Doppler effect to quickly measure the radial flow and vector flow field on the river surface, and automatically calculates the water flow rate by combining the water section information. Compared with rotor flowmeters and cruise ADCPs, radar flow measurement is a non-contact automatic flow measurement device that does not require on-site operation and has high safety. Compared with bottom-mounted automatic flow measurement systems, horizontal automatic flow measurement systems, and buoy-type automatic flow measurement systems, the side-scan radar automatic flow measurement system is more convenient to install and maintain hardware. However, for handheld radar guns, their measurement range is limited, and manual measurement during flood season is dangerous. Side-scan radars are susceptible to electromagnetic disturbances and will cause co-frequency interference.

[0004] In recent years, the flow measurement technology of drones has been maturing and developing. The use of drones equipped with Doppler radars or visual cameras for flow measurement is an emerging emergency monitoring technology in recent years. This measurement method overcomes the problem of fixed-point measurement and gives full play to the advantages of the drone technology platform, but the results of drone flow measurement technology still have certain limitations. 1) UAV photogrammetry and flow field calibration rely on image control points arranged on both sides of the river, so it is impossible to directly measure the flowing water surface that lacks scale reference objects in the camera field of view. For a wide section of more than 200m, in order to make the field of view cover the river banks on both sides, it is necessary to greatly increase the flight altitude of the drone, resulting in insufficient spatial resolution of the current mainstream civil drone cameras, making it difficult to effectively observe weak tracers on the water surface at the centimeter level. 2) Video data is not processed online, and the effective data rate is difficult to guarantee. Whether it is shore-based measurement or drone measurement, if the video acquisition point, time or camera pitch angle is not set properly, it is very likely that the missing objects on the surface cannot be correctly observed or submerged in the background noise. The traditional drone offline flow velocity measurement method can easily cause the problem of low effective measurement times, which greatly increases labor and economic costs. If online flow measurement by drones is realized, effective measurements can be screened out on site. When the measurements do not meet the measurement requirements, they can be discovered and supplemented in time on site, and the algorithm parameters and the measured attitude angle, focal length, etc. can be dynamically adjusted to improve the accuracy of the final result. Summary of the invention

[0005] Purpose of the invention: The present invention provides an online measurement method for the river surface flow velocity field based on an image-free UAV. By combining video flow measurement and image-free technology with a UAV platform, the online measurement of the river surface flow velocity can be completed without the deployment of image control points. The method is suitable for measuring the river surface flow field in long river sections or emergency scenarios, exhibits good adaptability and emergency response, and can more accurately measure the cross-sectional flow velocity field distribution. Compared with other types of UAV flow measurement systems, the cost is relatively low.

[0006] Technical solution: The method for online measurement of river surface velocity field based on an image-free drone described in the present invention comprises the following steps:

[0007] Step 1: Use the data transmission antenna and TCP link to establish a three-way communication connection between the drone, ground station and PC.

[0008] Step 2: Realize image-free UAV positioning with centimeter-level precision based on RTK high-precision differential positioning system;

[0009] Step 3: Prepare the hardware debugging before the drone takes off, adjust the gimbal and the onboard camera angle, set the take-off point, flight altitude, flight route, and the number and coordinates of hovering points, and write the drone track planning file;

[0010] Step 4: The drone takes off and goes to the preset hovering measurement point. Based on the RTSP video protocol and the drone MAVLink data packet protocol, it captures the drone's onboard camera video in real time and collects the drone's onboard RTK positioning data and radar water level meter data.

[0011] Step 5: Use the first frame alignment method to achieve frame synchronization between the video and the positioning data;

[0012] Step 6: Use the positioning data to correct the left and right and up and down deviations generated during the flight of the UAV, complete image de-shaking, synthesize the spatiotemporal image, and complete the measurement of the flow velocity field at the current measuring point i;

[0013] Step 7: After completing all preset hovering measurement points, the drone returns to the home position, performs averaging or interpolation processing on the overlapping or uncovered cross-sections of the field of view, and finally synthesizes and outputs the overall velocity field of the measured river section.

[0014] Furthermore, in step 1, a data transmission antenna is used to establish a communication link between the sky end (drone) and the ground end (RTK base station and drone ground station), and the communication connection is achieved through frequency matching. TCP is used to establish a communication link between the ground end and the PC end, and the communication connection is achieved through the IP address and port number, finally forming a three-end bidirectional transmission communication link.

[0015] Furthermore, in step 1, a data transmission antenna is used to establish a communication link between the sky end (drone) and the ground end (RTK base station and drone ground station), and the communication connection is achieved through frequency matching. TCP is used to establish a communication link between the ground end and the PC end, and the communication connection is achieved through the IP address and port number, finally forming a three-end bidirectional transmission communication link.

[0016] Furthermore, in step 2, the actual distance Z between the drone and the river surface is obtained based on the radar water level meter. C Based on the real-time dynamic carrier phase difference (RTK) technology, the centimeter-level precision positioning (lat, lon) of the drone is obtained by calculating the carrier phase difference between the drone's airborne RTK and the ground base station, and no longer relying on the ground pixel control points. The positioning coordinates of the base station can be obtained by the base station self-convergence method. When the coordinate accuracy of the base station self-positioning reaches the preset accuracy value, the current coordinates can be considered to be the base station coordinates. If the exact coordinates of the base station location are known in advance, the known coordinates can be directly input for carrier phase difference calculation.

[0017] Furthermore, in step 3, the constraint relationship between the river width L, the single measuring point cross-section coverage length x, and the number of measuring points N is established, and the constraint relationship between the single point hovering time t0, the hovering point switching time t1, the take-off and landing time t2 and the maximum flight time T of the UAV is established, and the coverage rate c of the cross-section flow velocity field is guaranteed. The relevant constraint relationship is:

[0018]

[0019] N·t0+(N-1)·t1+t2<α·T;

[0020] By calculating and designing the flight height h of the UAV and the hovering measurement point interval Δx, the flight track of the UAV is planned and written into the UAV. The corresponding relationship in the present invention is:

[0021]

[0022]

[0023] Among them, f is the focal length of the camera (unit: m), v is the camera frame length (unit: m), and res is the camera spatial resolution (unit: pixel / m). The flight altitude and hovering point can be calculated and planned in advance, and the track can be saved together as a track file (waypoints). The track file can be directly loaded before takeoff to simplify the operation process and improve operation efficiency.

[0024] Furthermore, in step 4, the video image and flight data are obtained dynamically and in real time on the PC side through the communication link established in step 1. The video image of the airborne camera is transmitted and captured in real time using the Real-time Streaming Protocol (RTSP), and the flight data of the drone is transmitted and captured in real time using the Small Unmanned Vehicle Communication Protocol (MAVLink). By judging the ID of the transmitted heartbeat packet MSG, determining the data content of the current heartbeat packet transmission and parsing it, the information of each sensor of the drone can be obtained, including positioning data, flight time, gyroscope, accelerometer, etc., for subsequent flow field measurement.

[0025] Furthermore, in step 5, after the data capture is completed, determine whether the captured data is complete based on the capture time and sensor frequency. If data packet loss occurs, first search for the actual data in sequence according to the interval from the data packet timestamp and interpolate and fill it, and then record the data packet timestamp corresponding to the first frame of the captured video to achieve synchronization and frame alignment of the positioning data and video data.

[0026] Furthermore, in step 6, the corresponding relationship under different coordinate systems is obtained according to the coordinate system transformation relationship, and the corresponding relationship formula is:

[0027]

[0028] Among them, the world coordinate system (O W -X W Y W Z W ), the unit is m; camera coordinate system (O C -X C Y C Z C ), in m; image physical coordinate system (O-uv), in mm; image plane coordinate system (O-xy), in pixel. Based on the decoded video frame, the area with less distortion in the middle of the video is selected as the region of interest (ROI), and the video frame is synthesized into a spatiotemporal image. The texture angle of the spatiotemporal image is detected by frequency spectrum transform (FFT), and then the river surface velocity of the measuring point is calculated. The calculation relationship is:

[0029]

[0030] Among them, V represents the actual speed of the river, θ represents the texture angle, Δs (m / pixel) represents the scaling relationship on the speed measurement line, and Δt is the time required to synthesize the entire space-time image.

[0031] Furthermore, in step 7, after completing the measurement of the local surface velocity field of each hovering point, it is synthesized to obtain the complete velocity field of the section to be measured. The specific method is: for the measurement blind area not covered by the drone's shooting field of view, the blind area flow velocity is interpolated according to the measured flow velocity value; for the area where the shooting field of view overlaps, the measured flow velocities at the same starting point distance are fused by averaging.

[0032] Beneficial effect: Compared with the prior art, the present invention has the following significant advantages: The present invention constructs an online measurement system and method for river surface velocity field based on image-free UAV, realizes multi-sensor integration, synchronous data acquisition, wireless real-time transmission, image de-jittering and space-time image velocity measurement, it has the advantages of real-time online and image-free, and can realize the measurement of river surface velocity field by UAV without additional deployment of image control points. It is suitable for online measurement of rivers with a width of more than 200 meters, and can be used for rapid emergency monitoring of wide-section rivers under extreme conditions such as floods, and provides safe and efficient on-site emergency monitoring means for emergency monitoring issues such as flood warning and breach sealing, provides basic support for building a smart water conservancy perception system and emergency rescue decision-making, and enhances the emergency monitoring capabilities of large rivers and flood disasters. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0034] Figure 2 Schematic diagram of the system topology of the present invention.

[0035] Figure 3 It is a schematic diagram of the track planning and measurement mode of the present invention.

[0036] Figure 4 It is a schematic diagram of the pan / tilt structure of the present invention.

[0037] Figure 5 It is a schematic diagram of the integration method of the camera and RTK module of the present invention.

[0038] Figure 6 This is a block diagram of the system modules of the present invention.

[0039] Figure 7 This is the MAVLink heartbeat packet structure diagram and part of the flight data MsgID in the present invention.

[0040] Figure 8 This is the MAVLink data parsing flow chart of the present invention.

[0041] Fig. 9 This is a flow chart of RTSP video capture according to the present invention.

[0042] Fig.10 This is a schematic diagram of the positioning data frame synchronization method of the present invention.

[0043] Fig.11 Schematic diagram of coordinate system transformation of the present invention.

[0044] Fig.12 Schematic diagram of the image offset correction process of the present invention.

[0045] Fig.13 It is a schematic diagram of the correction result of the present invention.

[0046] Fig.14 It is a schematic diagram of single-point measurement results of the present invention.

[0047] Fig.15 This is a diagram showing the synthesis results of the cross-sectional flow velocity field of the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 As shown, a method for online measurement of river surface velocity field based on an image-free UAV includes the following steps:

[0049] S1: Establish a communication connection between the drone and the ground end and confirm the connection status. To use an image-free drone to measure flow using the image method, it is first necessary to connect the communication link between the sky end (drone) and the ground end (RTK base station and ground station) using a digital transmission antenna. The present invention uses a flight control model of Pixhawk4, a ground station model of Yunzhuo H16 ground station, and a digital transmission radio model of H16 digital transmission. The sky end and the ground end are connected by frequency matching through the digital transmission radio. The ground station on the ground end uses TCP to establish a communication link between the ground end and the PC end. This is achieved by configuring the IP address, port number, and baud rate. The RTK ground station on the ground end is connected to the host PC through a USB interface, eventually forming a three-terminal bidirectional transmission communication link, and testing the communication status. The topology diagram of the three-terminal system of RTK base station, ground station, and PC end is shown in the figure. Figure 2 shown.

[0050] S2: Achieve RTK base station positioning on the ground side and confirm the positioning accuracy, and complete the high-precision RTK positioning of the drone on the sky side. After confirming that the drone's communication status is good, turn on the safety switch on the drone's sky-side RTK. Based on the real-time dynamic carrier phase difference (RTK) technology, the carrier phase difference between the drone's onboard RTK and the ground base station is calculated to obtain the centimeter-level precision positioning of the drone, no longer relying on the pixel control points on the ground. The positioning coordinates of the base station can be obtained by base station self-convergence. When the coordinate accuracy of the base station's self-positioning reaches the preset accuracy value, the current coordinates can be considered to be the base station coordinates. If the exact coordinates of the base station's location are known in advance, the known coordinates can be directly entered for carrier phase difference calculation, without the need for self-convergence positioning to determine the coordinates. After the base station positioning is completed, the drone starts high-precision positioning. At this time, the drone has 4 possible states: Need GPS, 3D Fix, RTK Float, and RTK Fix. The Need GPS state means that the drone RTK is not positioned. You should check whether the drone RTK indicator is normal, whether the connection with the flight control is good, and whether the number of search satellites is too low. In this state, the drone is prohibited from taking off; 3D Fix means that the drone has completed the drone GPS positioning, but has not reached RTK positioning. The drone can take off, but because the positioning accuracy is only at the meter level, the video flow measurement result has a large error and cannot represent the actual flow rate; RTK Float means that the drone has reached RTK positioning. At this time, the measurement requirements of centimeter-level positioning have been met. The drone can take off and can meet the accuracy requirements of video flow measurement; RTK Fix means that the drone has reached RTK positioning and has high positioning accuracy. The requirements of this state are relatively stringent, generally requiring the drone to have a relatively open flight environment, good weather conditions, a high number of RTK search satellites, and less environmental electromagnetic wave interference. For this article, RTKFloat can meet the drone's image-free control needs.

[0051] S3: Complete the flight track planning and pre-takeoff hardware preparation of the drone, and write the track file to the drone. After the drone RTK positioning is completed, the flight track planning and pre-takeoff hardware preparation of the drone are required. The track planning and measurement mode of the present invention is shown as follows Figure 3 As shown, QGroudControl can be used for track planning in the UAV ground station, and Missionplanner can be used for track planning on the PC side. Track planning includes the setting of the take-off point, flight direction, flight altitude, measurement point interval and number of measurement points of the UAV. For wide sections, the endurance of the UAV will limit the hovering time of a single measurement point. Therefore, the river width L, the cross-section coverage length x of a single measurement point, and the number of measurement points N are mutually constrained. The single-point hovering time t0, the hovering point switching time t1, the take-off and landing time t2 and the maximum flight time T of the UAV are mutually constrained. At the same time, the coverage rate c of the cross-section flow velocity field must be guaranteed. At present, the flight time of most consumer-grade UAVs is between 10 and 30 minutes. For a cross-section of about 200 meters, the hovering time of a single hovering point of the UAV should be maintained at least 30 seconds, and the cross-section coverage length of a single measurement point is 20 to 40 meters. The relevant constraint relationship is:

[0052]

[0053] N·t0+(N-1)·t1+t2<α·T;

[0054] In the formula, c represents the cross-section coverage rate. To ensure the effective coverage rate of the velocity field data, the value is 0.8. The constraint of α·T is to ensure that the UAV still has some spare power to cope with possible emergencies after completing all measurement tasks. The value of the coefficient α is generally 0.8.

[0055] The flight direction of the UAV is generally set to fly along the river section. The flight altitude h of the UAV and the interval Δx between hovering measurement points are obtained by calculation, and their corresponding relationship is:

[0056]

[0057]

[0058] Among them, f is the focal length of the camera (unit: m), v is the length of the camera frame (unit: m), and res is the spatial resolution (unit: pixel / m). In order to reduce the interference of the propeller wind of the drone on the water surface, the flight altitude of the drone is generally not less than 10 meters. The flight altitude and hovering point can be calculated and planned in advance, and the track can be saved together as a track file (waypoints). The track file can be directly loaded before takeoff to simplify the operation process and improve the operation efficiency.

[0059] In the hardware preparation phase before takeoff, the camera should be installed on a three-axis self-stabilizing gimbal to control the angle of the onboard camera in real time. A rubber shock-absorbing rubber ring should be installed at the connection between the three-axis self-stabilizing gimbal and the drone to cope with the high-frequency vibration generated by the drone during flight. The gimbal structure diagram is shown in the figure below. Figure 4 To reduce the system error of the drone, the installation position of the drone camera should be on the same plumb line as the drone's airborne RTK and as close to the RTK as possible. The pitch angle of the camera should be in the vertical direction, and the horizontal direction of the captured image should be parallel to the direction of the nose. The schematic diagram of the camera and RTK module integration is shown in the figure below. Figure 5 Finally, check whether the UAV power cable tie is secure, whether the RTK is working properly, whether the rotor and propeller are firmly fixed, and ensure that the communication link, ground station, and data transmission between the flight control system are normal to ensure the flight safety of the UAV. The UAV system module connection diagram is as follows: Figure 6 shown.

[0060] S4: The drone takes off and goes to the predetermined measuring point to capture the video stream and positioning data. There are two types of drone takeoff modes - fixed-point mode and self-stabilizing mode. The fixed-point takeoff mode does not require GPS or RTK, and the takeoff can be controlled manually by turning the stick. The present invention adopts the self-stabilizing takeoff mode. Before taking off, it is necessary to ensure that the drone has an RTK positioning signal, and then go to the hovering measuring point. After arriving at the predetermined hovering measuring point, the video image and flight data can be obtained dynamically and in real time on the PC through the communication link. The drone flight data is transmitted and captured in real time through the Small Unmanned Vehicle Communication Protocol (MAVLink). The structure diagram of the MAVLink heartbeat packet is shown as follows: Figure 7 As shown, the message format is STX (data packet start flag), LEN (payload length), SEQ (data packet sequence number), SYS (system ID), COMP (component ID), MSG (message ID), PAYLOAD (message payload), CKA (low 8-bit check bit), CKB (high 8-bit check bit). By detecting the message ID of the heartbeat packet, the information of each sensor of the drone can be obtained, including positioning data, flight time, gyroscope, accelerometer, etc. The present invention captures positioning data and radar water level gauge data and records them in txt format. The MAVLink data analysis flow chart is shown in Figure 8 The video images of the onboard camera are transmitted and captured in real time using the Real-time Streaming Protocol (RTSP). The RTSP video capture flow chart is shown in Fig. 9 shown.

[0061] S5: Complete the synchronization of positioning data and perform frame alignment with the collected video frames. The synchronization and frame alignment of positioning data and video data are achieved by recording the data packet timestamp corresponding to the first frame of the captured video. The data packet timestamp represents the time difference between the current moment and the moment when the UAV flight control is powered on. The longitude and latitude coordinates of the UAV and its timestamp are recorded in real time. When the UAV arrives at the measuring point and starts to capture and save the video image of the onboard camera at the current moment, the corresponding timestamp is recorded synchronously once to achieve synchronization between the UAV onboard camera video and the positioning data. If data packet loss occurs during the data capture process, the missing data should be interpolated and supplemented before data synchronization. Using this method, the synchronization between sensor data such as gyroscope data and accelerometer data and the collected video can also be achieved. The schematic diagram of the positioning data frame synchronization method is shown in the figure. Fig.10 shown.

[0062] S6: Complete image offset correction, use the corrected video frames to synthesize spatiotemporal images, and perform spatiotemporal image velocity measurement. First, the corresponding relationship under different coordinate systems is obtained according to the coordinate system transformation relationship. The corresponding relationship formula is:

[0063]

[0064] Among them, the world coordinate system (O W -X W Y W Z W ), the unit is m; camera coordinate system (O C -X C Y C Z C ), in m; image physical coordinate system (O-uv), in mm; image plane coordinate system (O-xy), in pixel. C is the actual distance between the drone camera and the river surface, measured by the radar water level meter. The coordinate system transformation diagram is shown in Fig.11 shown.

[0065] The schematic diagram of image offset correction process is as follows: Fig.12 As shown in the figure, the synchronized video data and high-precision positioning data are used to decode the video and obtain the video frame. The area with less distortion in the middle of the video is selected as the region of interest (ROI). The positioning data is used to de-jitter the ROI image of each frame. The positioning data includes the longitude (Log), latitude (Lat) and flight altitude (Alt) from the take-off point of the drone. The ROI image is corrected according to the object-image conversion relationship so that it is always consistent with the image positioning information in the first frame of the video. The correction result diagram is shown in the figure. Fig.13After the de-shaking correction of all ROI images is completed, the ROI images are synthesized into a spatiotemporal image (STI), and the texture angle of the spatiotemporal image is detected by frequency spectrum transform (FFT), and then the river surface velocity at the measuring point is calculated. The calculation relationship is:

[0066]

[0067] Where V represents the actual velocity of the river, θ represents the texture angle, Δs (m / pixel) represents the scaling relationship on the velocity measurement line, and Δt is the time required to synthesize the entire spatiotemporal image. If the velocity field measurement result of the current measuring point is distorted or cannot represent the river surface velocity, it should be remeasured in time. The schematic diagram of the correction result is shown in the figure. Fig.13 shown.

[0068] S7: After completing all the measurement points and returning, interpolate the unmeasured cross-sections, and finally synthesize and output the overall velocity field of the measured river cross-section. The output result of the velocity field of a single point measurement point is shown in the figure below. Fig.14 As shown in the figure, since the river surface measured by the drone is generally a wide river section and a large cross-section, after the point measurement, it is necessary to synthesize the cross-sectional flow velocity fields of different measuring points in turn, and then obtain the measured cross-sectional flow velocity field. Taking the take-off point of the drone as the starting point, calculate the starting point distance, and output the complete cross-sectional flow velocity. Since it is difficult to avoid the overlap or missing of the flow velocity field between different measuring points during the flight of the drone, the flow field interpolation processing is performed on the cross-sectional positions not covered by the drone video. The specific method is: according to the distribution law that the flow velocity in the middle of the river surface is large and the flow velocity on both sides is small, the flow velocity on both sides of the blind spot of the field of view is averaged, and interpolation is performed based on the averaged value. For the positions where the flow velocity field overlaps, it is necessary to obtain the cross-sectional flow velocity by taking the measured average value, and then obtain the complete cross-sectional flow velocity field. The cross-sectional flow velocity field synthesis result is shown in the figure below. Fig.15 shown.

[0069] In view of the hardware conditions of the image-free UAV and the demand for flow field measurement, the present invention establishes a photogrammetry parameter constraint model for the image-free UAV in the track planning. According to the constraints such as river width, flow field coverage, camera spatial resolution, minimum flight altitude, and endurance time, the flight parameters such as single-point hovering time, flight altitude, and number of hovering points of the UAV are optimized. Compared with the existing UAV video flow measurement technology, the present invention solves the problem that the effective coverage of the cross section is low when the UAV is flying at a low altitude, and the number of visible tracers on the river surface is small when the flying altitude is high.

[0070] The present invention aims at the high-frequency jitter problem caused by the high-speed rotation of the propeller of the drone during flight, and uses a three-axis self-stabilizing pan-tilt camera to filter out the impact of the high-frequency vibration generated by the drone during flight on the video image. In view of the low-frequency drift of the video caused by the influence of strong winds on the drone, the drone video and flight positioning data are transmitted online in real time, and the synchronous alignment of the video frame and the sensor data is achieved by frame synchronization. The camera is calibrated with internal parameters, and the three-dimensional image offset (the change relative to the first frame) is achieved based on the drone orthophotometry model to estimate and compensate for the offset error caused by the low-frequency jitter of the drone. Compared with the existing drone video flow measurement technology, it realizes the online measurement of the drone in the flight state and compensates and corrects the jitter error of the drone, thereby improving the timeliness and accuracy of the measurement.

[0071] In order to solve the problem of low precision of elevation measurement of existing UAVs relying on image control points or RTK UAVs, the present invention uses RTK to calibrate the two-dimensional plane world coordinates and solve the starting point distance, uses a radar water level meter to measure the precise elevation of the UAV camera from the water surface in real time, and based on the local flow velocity field of multiple hovering points measured by the spatiotemporal image velocimetry method, averages the flow velocity in the overlapping area of ​​the field of view under actual flight conditions, interpolates the flow velocity in the blind area of ​​the field of view, and finally synthesizes the surface flow velocity field of the complete section. Compared with the existing UAV video flow measurement technology, the dense flow velocity distribution of the complete section can be obtained without relying on image control points, providing support for accurate river flow calculation.

Claims

1. A method for online measurement of river surface velocity field based on image-free drone, characterized in that: The steps include: Step 1: Use the data transmission antenna and TCP link to establish a three-way communication connection between the drone, ground station and PC. Step 2: Realize image-free UAV positioning with centimeter-level precision based on RTK high-precision differential positioning system; Step 3: Prepare the hardware debugging before the drone takes off, adjust the gimbal and the onboard camera angle, set the take-off point, flight altitude, flight route, and the number and coordinates of hovering points, and write the drone track planning file; Step 4: The drone takes off and goes to the preset hovering measurement point. Based on the RTSP video protocol and the drone MAVLink data packet protocol, it captures the drone's onboard camera video in real time and collects the drone's onboard RTK positioning data and radar water level meter data. Step 5: Use the first frame alignment method to achieve frame synchronization between the video and the positioning data; Step 6: Use the positioning data to correct the left and right and up and down deviations generated during the flight of the UAV, complete image de-shaking, synthesize the spatiotemporal image, and complete the measurement of the flow velocity field at the current measuring point i; Step 7: After completing all preset hovering measurement points, the drone returns to the home position, performs averaging or interpolation processing on the overlapping or uncovered cross-sections of the field of view, and finally synthesizes and outputs the overall velocity field of the measured river section.

2. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 1, a communication link is established between the drone and the ground station using a data transmission antenna, and communication connection is achieved through frequency matching. TCP is used to establish a communication link between the ground terminal and the PC terminal, and communication connection is achieved through IP addresses and port numbers, ultimately forming a three-terminal bidirectional transmission communication link.

3. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 2, the actual distance Z between the drone and the river surface is obtained based on the radar water level meter. C Based on the real-time dynamic carrier phase differential RTK technology, the centimeter-level precision positioning (lat, lon) of the drone is obtained by calculating the carrier phase difference between the drone-mounted RTK and the ground base station, and no longer relying on the pixel control points on the ground; The positioning coordinates of the base station are obtained through the self-convergence of the base station. When the coordinate accuracy of the base station self-positioning reaches the preset accuracy value, the current coordinates are considered to be the base station coordinates. If the exact coordinates of the base station location are known in advance, the known coordinates are directly input for carrier phase differential calculation.

4. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 3, the constraint relationship between the river width L, the single measuring point cross-section coverage length x, and the number of measuring points N is established, and the constraint relationship between the single-point hovering time t0, the hovering point switching time t1, the take-off and landing time t2 and the maximum flight time T of the UAV is established, and the coverage rate c of the cross-section flow velocity field is guaranteed. The relevant constraint relationship is: N·t0+(N-1)·t1+t2<α·T.

5. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 3, the flight altitude h of the designed UAV and the hovering measurement point interval Δx are calculated, the flight track of the UAV is planned and written into the UAV, and the corresponding relationship is: Where f is the focal length of the camera, unit: m, v is the camera frame length, unit: m, and res is the camera spatial resolution, unit: pixel / m. Calculate and plan the flight altitude and hovering point in advance, and save the track together as a track file waypoints. Load the track file directly before takeoff to simplify the operation process and improve operation efficiency.

6. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 4, through the communication link established in step 1, the video image and flight data are obtained dynamically and in real time on the PC side. The video image of the onboard camera is transmitted and captured in real time using the real-time streaming protocol RTSP. The UAV flight data is transmitted and captured in real time through the small unmanned vehicle communication protocol MAVLink. By judging the ID of the transmitted heartbeat packet MSG, the data content of the current heartbeat packet transmission is determined and parsed to obtain the information of each sensor of the UAV.

7. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 6, characterized in that: The information from various sensors of the drone includes positioning data, flight time, gyroscope, and accelerometer, which are used for subsequent flow field measurements.

8. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 5, after the data capture is completed, determine whether the captured data is complete based on the capture time and sensor frequency. If data packet loss occurs, first search for the actual data in sequence according to the interval from the data packet timestamp and interpolate and fill it, then record the data packet timestamp corresponding to the first frame of the captured video to achieve synchronization and frame alignment of the positioning data and video data.

9. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 6, the corresponding relationship under different coordinate systems is obtained according to the coordinate system transformation relationship, and the corresponding relationship formula is: Among them, the world coordinate system (O W -X W Y W Z W ), the unit is m; camera coordinate system (O C -X C Y C Z C ), in m; image physical coordinate system (O-uv), in mm; image plane coordinate system (O-xy), in pixel. Based on the decoded video frame, the area with less distortion in the middle of the video is selected as the region of interest ROI, and the video frame is synthesized into a spatiotemporal image. The texture angle of the spatiotemporal image is detected by spectrum transform FFT, and then the river surface velocity of the measuring point is calculated. The calculation relationship is: Among them, V represents the actual speed of the river, θ represents the texture angle, Δs (m / pixel) represents the scaling relationship on the speed measurement line, and Δt is the time required to synthesize the entire space-time image.

10. The method for online measurement of river surface velocity field based on image-free drone as claimed in claim 1, characterized in that: In step 7, after completing the measurement of the local surface velocity field of each hovering point, it is synthesized to obtain the complete velocity field of the section to be measured. The specific method is: for the measurement blind area not covered by the drone's shooting field of view, the blind area flow velocity is interpolated according to the measured flow velocity value; for the area where the shooting field of view overlaps, the measured flow velocity at the same starting point distance is merged by averaging.

Images