Unmanned aerial vehicle bank protection patrol method and computer readable storage medium

Abstract

The invention discloses an unmanned aerial vehicle bank protection patrol method and a computer readable storage medium, and the method comprises the steps: determining an initial flight position of an unmanned aerial vehicle, and controlling the unmanned aerial vehicle to fly forwards from the initial flight position; according to a preset period, performing water surface recognition on the real-time shot image of the unmanned aerial vehicle to obtain a water surface area, and performing contour extraction on the water surface area to obtain a contour point set of the water surface area; acquiring a near-shore point set and an opposite-shore point set according to the contour point set; according to the contour points in the near-shore set, calculating corresponding flight path points, so that the unmanned aerial vehicle flies along the near-shore line; and according to the contour points in the opposite bank point set, the flight path points are adjusted, so that the unmanned aerial vehicle flies above the water surface area. According to the invention, autonomous shore patrol of the unmanned aerial vehicle can be realized.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to an unmanned aerial vehicle shore protection inspection method and a computer-readable storage medium. Background Art

[0002] River bank protection is an important dividing line between the water environment and the human activity area, and is the focus of water shoreline space control and ecological governance. Arranging personnel to inspect the health of the bank protection is time-consuming and laborious, and the coverage of the bank protection by video monitoring stations is extremely limited. Using drones to fly along the river surface can clearly observe the front of the bank protection, and can cover the entire river section, so as to achieve true no-dead-angle monitoring, which is the best choice for this inspection task. Drones conduct bank protection inspections along the river, and the targets of attention include whether there is bank protection damage, weed growth on the slope, and the health of the bank protection, such as the accumulation of debris, and can also include the monitoring content of water surface pollution and personnel violations. Thanks to the rapid development of computer image processing and artificial intelligence technology, these monitoring tasks can be completed by computer algorithms instead of human eyes, and the identified event information such as location coordinates, severity, and photo evidence can be reported to river managers in real time to achieve efficient supervision and governance. The environment of small rivers, especially urban rivers, is complex. Drones must not only ensure that they move safely and steadily in the predetermined direction, but also ensure that the images of the bank protection taken are clear and stable. There are certain angle and distance requirements, which makes them not easy to operate. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a UAV bank protection inspection method and a computer-readable storage medium, which can realize autonomous bank inspection of the UAV without pre-setting the route, saving labor costs and ensuring the flight safety of the UAV.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for patrolling bank protection by using an unmanned aerial vehicle, comprising:

[0005] Determine the initial flight position of the UAV according to the preset flight altitude, pitch angle and initial orientation, and control the UAV to fly forward from the initial flight position according to the preset flight speed;

[0006] According to a preset period, the real-time shooting image of the UAV is subjected to water surface recognition to obtain a water surface area, and the contour of the water surface area is extracted to obtain a contour point set of the water surface area;

[0007] According to the contour point set, a nearshore point set and an opposite shore point set are obtained;

[0008] According to the contour points in the nearshore set, corresponding flight path points are calculated so that the UAV flies along the nearshore line;

[0009] The flight path points are adjusted according to the contour points in the opposite bank point set so that the UAV flies above the water surface area.

[0010] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0011] The beneficial effects of the present invention are as follows: by determining subsequent flight path points in real time according to the water surface recognition results of the captured images during the flight, the UAV can automatically fly along the coastline of the water area without the need to manually set the route. Compared with the designated fixed cruise route, it can be more convenient in new environments or temporary patrol scenarios; by adjusting the flight path points according to the contour points on the opposite bank, the UAV can fly within the river area, thereby ensuring the flight safety of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flow chart of a method for inspecting shore protection using an unmanned aerial vehicle of the present invention;

[0013] Figure 2 Schematic diagram of the inverse mapping transformation according to the first embodiment of the present invention;

[0014] Figure 3 This is a flow chart of a method according to Embodiment 1 of the present invention;

[0015] Figure 4 This is a schematic diagram of offshore distance correction according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0017] See also Figure 1 , a method for inspecting bank protection using a drone, comprising:

[0018] Determine the initial flight position of the UAV according to the preset flight altitude, pitch angle and initial orientation, and control the UAV to fly forward from the initial flight position according to the preset flight speed;

[0019] According to a preset period, the real-time shooting image of the UAV is subjected to water surface recognition to obtain a water surface area, and the contour of the water surface area is extracted to obtain a contour point set of the water surface area;

[0020] According to the contour point set, a nearshore point set and an opposite shore point set are obtained;

[0021] According to the contour points in the nearshore set, corresponding flight path points are calculated so that the UAV flies along the nearshore line;

[0022] The flight path points are adjusted according to the contour points in the opposite bank point set so that the UAV flies above the water surface area.

[0023] From the above description, it can be seen that the beneficial effects of the present invention are: autonomous shore patrol of the UAV can be realized without pre-setting the route, saving labor costs, and ensuring the flight safety of the UAV.

[0024] Furthermore, it also includes:

[0025] According to the real-time positioning position of the UAV, the map coordinates of each flight path point on the map are calculated respectively, and the flight path is updated according to the preset number of flight path points closest to the UAV.

[0026] From the above description, it can be seen that the flight path is updated according to the closest flight path points among the latest calculated flight path points to ensure high adaptability of the flight path to the river channel.

[0027] Furthermore, at a moment in the flight of the drone, a world coordinate system corresponding to the moment is constructed with the vertical projection position of the drone on the plane where the water surface is located as the origin, the plane where the water surface is located as the XOY plane, the direction of the drone camera as the Y-axis direction, and the vertical direction as the Z-axis direction;

[0028] The method of performing water surface recognition on the real-time image captured by the drone according to a preset period to obtain a water surface area, and performing contour extraction on the water surface area to obtain a contour point set of the water surface area includes:

[0029] According to a preset cycle, the latest captured image is obtained as the current image, and the world coordinate system corresponding to the shooting time of the current image is used as the current world coordinate system;

[0030] The current image is segmented to obtain a water surface area, and the contour of the water surface area is extracted to obtain a contour point set of the water surface area. The contour points in the contour point set take the contour point close to the upper left corner of the image as the starting point and are arranged in a counterclockwise direction.

[0031] From the above description, it can be seen that the world coordinate system corresponding to different moments will change as the position of the drone on the horizontal plane and the direction of the camera change.

[0032] Furthermore, before obtaining the nearshore point set and the opposite shore point set according to the contour point set, the method further includes:

[0033] The contour points at the boundary of the current image are removed from the contour point set.

[0034] From the above description, it can be seen that since the contour points at the image boundary are generally not points on the shoreline, the contour points at the image boundary are eliminated to reduce the number of contour points and improve the efficiency of subsequent division of the contour points on both sides.

[0035] Furthermore, before obtaining the nearshore point set and the opposite shore point set according to the contour point set, the method further includes:

[0036] Traversing the contour point set, sequentially acquiring a contour point as a current contour point, and using the first contour point acquired as a reference point;

[0037] Perform inverse perspective transformation on the current contour point to obtain the world coordinates of the current contour point in the current world coordinate system;

[0038] Calculate the distance between the current contour point and the latest reference point according to their world coordinates in the current world coordinate system;

[0039] If the distance between the current contour point and the latest reference point is greater than the preset distance threshold, the current contour point is used as the new reference point;

[0040] After traversing the contour point set, contour points that are not reference points are removed from the contour point set.

[0041] From the above description, it can be seen that by screening the contour points and retaining the contour points with moderate adjacent distances, the number of contour points can be reduced, the efficiency of subsequent path point calculation can be improved, and the uniformity of the flight path points can be ensured.

[0042] Furthermore, if the initial orientation is the right direction after the drone takes off towards the river center, the nearshore point set is the right bank point set, and the opposite bank point set is the left bank point set; if the initial orientation is the left direction after the drone takes off towards the river center, the nearshore point set is the left bank point set, and the opposite bank point set is the right bank point set;

[0043] The step of obtaining a nearshore point set and an opposite shore point set according to the contour point set is specifically as follows:

[0044] If a contour point meets the preset right bank point condition, the contour point is recorded in the right bank point set, and the right bank point condition is Y i-1 <Y i <Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |;

[0045] If a contour point meets the preset left bank point condition, the contour point is recorded in the left bank point set, and the left bank point condition is Yi-1 >Y i >Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |;

[0046] Among them, (X i , Y i ) represents the world coordinates of the contour point in the current world coordinate system, (X i-1 , Y i-1 ) represents the world coordinates of the previous contour point of the contour point in the contour point set in the current world coordinate system, (X i+1 , Y i+1 ) represents the world coordinates of the next contour point of the contour point in the contour point set in the current world coordinate system.

[0047] Furthermore, the world coordinate T of the flight path point corresponding to a contour point in the nearshore point set in the current world coordinate system is i =(X Ti , Y Ti , h0),

[0048]

[0049] Among them, (X Pi , Y Pi ) represents the world coordinates of a contour point in the nearshore point set in the current world coordinate system, (X Pi-1 , Y Pi-1 ) and (X Pi+1 , Y Pi+1 ) respectively represent the world coordinates of the previous contour point and the next contour point of the contour point in the nearshore point set in the current world coordinate system; d represents the distance from the shore, and h0 represents the preset flight altitude.

[0050] From the above description, it can be seen that by calculating the one-to-one corresponding flight path points according to the contour points located near the shore, the drone can move along the river channel and maintain a certain distance from the river bank.

[0051] Further, adjusting the flight path points according to the contour points in the opposite bank point set includes:

[0052] respectively obtaining two flight path points closest to each contour point in the opposite bank point set;

[0053] Calculate the distance between each contour point and the flight path according to the world coordinates of each contour point and the two flight path points closest to it in the current world coordinate system;

[0054] Obtain the minimum value of the distance between each contour point and the flight path as the correction distance;

[0055] If the corrected distance is less than zero, the offshore distance is corrected according to the corrected distance to obtain a new offshore distance, and the step of calculating the corresponding flight path points according to the contour points in the nearshore set is re-executed according to the new offshore distance.

[0056] From the above description, it can be seen that by adjusting the distance from the path point to the right bank as a whole according to the projected distance from the opposite bank point to the path point, the drone can fly within the river area to ensure the flight safety of the drone.

[0057] Further, according to a first distance calculation formula, the distance between each contour point in the opposite bank point set and the flight path is calculated respectively, and the first distance calculation formula is:

[0058]

[0059] Among them, (X p对 , Y p对 ) represents the world coordinates of a contour point in the opposite bank point set in the current world coordinate system, (X p1 , Y p1 ) and (X p2 , Y p2 ) represents the world coordinates of the two flight path points closest to the contour point in the current world coordinate system, Y p1 <Y p2 .

[0060] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0061] Embodiment 1

[0062] Please refer to Figure 2-4 , Embodiment 1 of the present invention is: a UAV shore protection inspection method, which can be applied to UAV shore inspection tasks.

[0063] In this embodiment, a camera is installed on the front of the drone body, and the user performs image analysis on the images captured by the camera.

[0064] The drone is equipped with a gimbal and a high-definition camera. The high-definition camera is controlled by the gimbal to maintain a certain pitch and yaw angle with the fuselage, so that the revetment surface on one side can be clearly photographed while the fuselage keeps moving forward, which is convenient for subsequent algorithms to monitor the revetment situation.

[0065] The sonar device is mounted on the bottom of the fuselage to measure the height h of the drone from the water surface in real time. Considering that the water surface height of the inland river will change, this height value is more accurate than the relative height value obtained by the drone's own sensor, and can protect the drone at a safe distance h above the water surface. 安全 activities within the scope of.

[0066] like Figure 2 As shown, in this embodiment, at a certain moment in the flight of the drone, the vertical projection position point of the drone on the plane where the water surface is located is taken as the origin O, the plane where the water surface is located is taken as the XOY plane, the direction of the drone camera (i.e., the yaw angle direction) is taken as the Y-axis direction, and the vertical upward direction is taken as the positive direction of the Z-axis to construct the world coordinate system of the drone corresponding to that moment. That is to say, in this embodiment, the world coordinate system will change with the change of the position of the drone (the change of position in the horizontal plane) and the change of the camera direction.

[0067] In addition, for the images taken by the drone, an image coordinate system is constructed with the upper left corner of the image as the origin, the right direction as the positive direction of the X-axis, and the downward direction as the positive direction of the Y-axis.

[0068] Through the inverse perspective mapping formula, a point (x, y) on an image taken at a certain moment can be mapped to the XOY plane of the world coordinate system corresponding to that moment. The specific formula is as follows. Although the binocular camera is a fixed forward-facing camera, there is an inclination during flight, so the sensor installed on the fuselage needs to provide the pitch angle θ value during calculation.

[0069]

[0070] Among them, (x, y) represents the image coordinates of a pixel point in the captured image in the image coordinate system, (X, Y) represents the world coordinates corresponding to the point (that is, the coordinates in the world coordinate system corresponding to the time when the image was captured), h represents the height of the drone camera relative to the water surface (that is, the flight height of the drone), and θ represents the pitch angle of the drone camera; width image and height image It represents the resolution of the captured image, that is, the width and height of the captured image; α and β are the internal parameters of the drone camera, representing the horizontal and vertical viewing angles, which are determined by the focal length of the camera.

[0071] like Figure 3 As shown, the method of this embodiment includes the following steps:

[0072] S1: Determine the initial flight position of the UAV according to a preset flight altitude, pitch angle and initial orientation, and control the UAV to fly forward from the initial flight position according to a preset flight speed.

[0073] Specifically, the drone takes off from the river bank, flies a certain distance (e.g., about 10 meters) toward the center of the river to reach above the water surface, and then rotates 90° to the right so that the front of the drone is roughly facing the river inspection direction. In this embodiment, the right side of the take-off direction is used as the inspection direction. In other embodiments, the left side of the take-off direction can also be used as the inspection direction. In this embodiment, the drone rotates 90° to the left after reaching above the water surface.

[0074] In this embodiment, the distance between the drone and the shoreline (i.e., the initial distance from the shore) is set to d = 10m, the flight speed is v = 5m / s, the initial flight altitude is h0 = 5-10m, and the initial pitch angle is θ0 = 45°. Furthermore, the yaw angle at this moment is recorded.

[0075] S2: According to a preset period, water surface recognition is performed on the real-time captured image of the UAV to obtain a water surface area, and contour extraction is performed on the water surface area to obtain a contour point set of the water surface area.

[0076] Specifically, according to a preset period, the latest captured image is obtained as the current image, and the world coordinate system corresponding to the capturing time of the current image is used as the current world coordinate system. In this embodiment, the preset period is 3s.

[0077] Then, the current image is segmented to obtain the water surface area, and the contour of the water surface area is extracted to obtain the contour point set of the water surface area. That is, the current image is segmented to obtain at least one water surface area, wherein the deeplabv3 network based on the convolutional neural network can be used for image segmentation; then, the contour of each water surface area is extracted to obtain the contour point set of each water surface area. In this embodiment, for each water surface area, the contour point of its upper left corner is taken as the starting point, and the contour points are obtained counterclockwise and stored in the corresponding contour point set in sequence.

[0078] Furthermore, the segmented water surface area can be screened and filtered to retain only the water surface area belonging to the river channel to exclude interference factors such as small puddles.

[0079] S3: screening the contour points in the contour point set.

[0080] Specifically, the contour points at the boundary of the current image are removed from the contour point set, and these points do not belong to the points on the water shoreline. That is, if the image coordinates of a contour point in the image coordinate system of the current image satisfy x=0, x=width image , y=0 or y=height image , then the contour point is considered to be at the boundary of the current image and is removed from the contour point set.

[0081] In this embodiment, contour points are also screened according to the distance between contour points. Specifically, each contour point in the contour point set is subjected to an inverse perspective transformation to obtain the world coordinates in the current world coordinate system. Then, each contour point in the contour point set is traversed in sequence, and the first contour point in the contour point set is directly used as a reference point. For the contour points traversed thereafter, the distance between the contour point and the latest reference point is calculated based on the world coordinates of the contour point and the latest reference point in the current world coordinate system. If the distance is greater than a preset distance threshold (such as 5 meters), the contour point is used as a new reference point. After the contour point set is traversed, the contour points that are not reference points are removed from the contour point set, and only the reference points are retained.

[0082] S4: According to the contour point set, obtain a nearshore point set and an opposite shore point set.

[0083] Specifically, each contour point in the contour point set is traversed in order (the first and last two contour points can be ignored). If a contour point meets the preset right bank point condition, the contour point is recorded in the right bank point set; if the contour point meets the preset left bank point condition, the contour point is recorded in the left bank point set. i-1 <Y i <Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |; The left bank point condition is Y i-1 >Y i >Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |;(X i , Y i ) represents the world coordinates of the contour point in the current world coordinate system, (X i-1 , Y i-1 ) and (X i+1 , Y i+1 ) respectively represent the world coordinates of the previous contour point and the next contour point of the contour point in the contour point set in the current world coordinate system.

[0084] If there are multiple contour point sets, the above operation is performed on each contour point respectively, and the finally obtained nearshore point set and opposite shore point set respectively include the nearshore points and opposite shore points of multiple water surface areas.

[0085] In this embodiment, since the right side of the take-off direction is used as the patrol direction, the near-shore point set is the right-shore point set, and the opposite-shore point set is the left-shore point set.

[0086] Further, the contour points in the nearshore point set and the opposite shore point set are arranged in ascending order according to the Y values ​​of the world coordinates in the current world coordinate system.

[0087] S5: Calculate corresponding flight path points according to the contour points in the nearshore set, so that the UAV flies along the nearshore line.

[0088] Specifically, the riverbank orientation at a contour point in the nearshore point set can be approximated as a vector (X Pi+1 -X Pi-1 , Y Pi+1 -Y Pi-1 ), and the direction toward the center of the river can be approximated by the vector (-(Y Pi+1 -Y Pi-1 ), X Pi+1 -X Pi-1 ), then the world coordinate T of the flight path point corresponding to the contour point in the current world coordinate system i =(X Ti , Y Ti , h0),

[0089]

[0090] Among them, (X Pi , Y Pi ) represents the world coordinates of the contour point in the current world coordinate system, (X Pi-1 , Y Pi-1 ) and (X Pi+1 , Y Pi+1 ) respectively represent the world coordinates of the previous contour point and the next contour point of the contour point in the nearshore point set in the current world coordinate system; d represents the distance from the shore, and h0 represents the preset flight altitude.

[0091] That is to say, in the nearshore point set, except for the first and last two contour points, other contour points can calculate a corresponding flight path point, so that the drone can move along the river channel and maintain a distance of d meters from the river bank and a height of h0 from the water surface.

[0092] Furthermore, if the calculated Y Ti If it is less than 0, it is considered to be the point behind the drone and the flight path point is discarded.

[0093] S6: According to the contour points in the opposite bank point set, the flight path points are adjusted to enable the UAV to fly above the water surface area.

[0094] In actual scenarios, if the drone is flying at a low altitude and the river is narrow, it is safer to fly the drone in the river area. Therefore, in this embodiment, according to the distance from the opposite bank point to the path point projected on the horizontal plane, the distance from the path point to the right bank is adjusted as a whole, that is, the offshore distance is adjusted to ensure the flight safety of the drone.

[0095] Specifically, the opposite bank point set is traversed, and a contour point in the opposite bank point set is sequentially acquired as the current contour point.

[0096] The two flight path points closest to the current contour point are obtained, and the distance between the current contour point and the flight path is calculated according to the world coordinates of the current contour point and the two flight path points in the current world coordinate system.

[0097] For example, suppose the current contour point is P 对 , the world coordinates in the current world coordinate system are (X p对 , Y p对 ), the two flight path points closest to the contour point are P1 and P2, and their world coordinates in the current world coordinate system are (X p1 , Y p1 ) and (X p2 , Y p2 ), where Y p1 <Y p2 , then the distance between the contour point and the flight path is:

[0098]

[0099] After traversing the opposite bank point set, the distance between each contour point in the opposite bank point set and the flight path can be obtained, and the minimum value can be found as the corrected distance d 修正 ,like Figure 4 shown.

[0100] If d 修正 <0, it means that using the current offshore distance d to calculate the flight path point will cause the path point to be outside the water area, so the offshore distance d is corrected, that is, d = d + d 修正 Then, the flight path points are recalculated according to the corrected offshore distance, that is, step S5 is re-executed.

[0101] If d 修正 ≥0, it means that the flight path points are all within the river area and no correction is required.

[0102] S7: According to the real-time positioning position of the UAV, the map coordinates of each flight path point on the map are calculated respectively, and the flight path is updated according to a preset number of flight path points closest to the UAV.

[0103] Specifically, obtain the GPS positioning information (longitude) of the drone at the current time (i.e., the time when the current image is taken in step S2). O , latitude O ), first correct the yaw angle of the current world coordinate system to make it face due north, then convert it into the offset of longitude and latitude based on the offset distance of each flight path point equivalent to the origin of the current world coordinate system. Assuming that the world coordinates of a flight path point in the current world coordinate system are (X, Y), the corresponding yaw angle is The calculation formula for its longitude and latitude on the map is:

[0104]

[0105] Y′=Ycos(φ)–X sin(φ)

[0106]

[0107] latitude=latitude O +Y′*0.00000898

[0108] Then the historical path points in the drone control system (i.e., the flight path points calculated in the previous cycle) are cleared, and the drone path points are updated to the flight path points calculated in the current cycle. In this embodiment, only the five flight path points closest to the drone are retained. In the current cycle, the drone is controlled to fly forward according to these five flight path points.

[0109] Furthermore, each time a flight path point is completed, including the position at the moment of starting to execute a new path, its map coordinates on the map are recorded, which can be used for subsequent recognition modeling.

[0110] The drone can repeat the above process when returning, so that it can patrol the other side during the return journey.

[0111] The drone should calculate the obstacle avoidance path throughout the process of moving forward to avoid protruding branches, cables, etc. In this embodiment, the obstacle avoidance function of the drone platform can be used directly. In other embodiments, a binocular camera is installed on the front of the drone fuselage. Usually, the binocular camera can provide a more detailed obstacle perception capability within a distance of 30 meters. A typical obstacle avoidance algorithm is 3DVFH. This obstacle avoidance method is a local planning, which is suitable for processing forward binocular depth images and can solve the problem of drones avoiding obstacles and reaching target path points in 3D space.

[0112] This embodiment can realize autonomous bank patrol by drones and achieve full process automation of river bank protection and surrounding environment inspection.

[0113] Embodiment 2

[0114] This embodiment is a computer-readable storage medium corresponding to the above embodiment, on which a computer program is stored. When the program is executed by the processor, the various steps of a drone shore patrol method in the above embodiment are implemented, and the same technical effect can be achieved, which will not be repeated here.

[0115] In summary, the present invention provides a UAV bank protection patrol method and computer-readable storage medium, which determines subsequent flight path points based on the water surface recognition results of the captured images in real time during the flight, so that the UAV can automatically fly along the water shoreline without manually setting the route. Compared with the designated fixed cruise route, it can be more convenient in new environments or temporary patrol scenarios; by adjusting the flight path points according to the contour points located on the opposite bank, the UAV can fly within the river area, thereby ensuring the flight safety of the UAV.

[0116] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for inspecting bank protection using an unmanned aerial vehicle, characterized in that: include: Determine the initial flight position of the UAV according to the preset flight altitude, pitch angle and initial orientation, and control the UAV to fly forward from the initial flight position according to the preset flight speed; According to a preset period, the real-time shooting image of the UAV is subjected to water surface recognition to obtain a water surface area, and the contour of the water surface area is extracted to obtain a contour point set of the water surface area; According to the contour point set, a nearshore point set and an opposite shore point set are obtained; According to the contour points in the nearshore set, corresponding flight path points are calculated so that the UAV flies along the nearshore line; The flight path points are adjusted according to the contour points in the opposite bank point set so that the UAV flies above the water surface area.

2. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 1, characterized in that: Also includes: According to the real-time positioning position of the UAV, the map coordinates of each flight path point on the map are calculated respectively, and the flight path is updated according to the preset number of flight path points closest to the UAV.

3. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 1, characterized in that: At a moment in the flight of the drone, the vertical projection position of the drone on the plane where the water surface is located is taken as the origin, the plane where the water surface is located is taken as the XOY plane, the direction of the drone camera is taken as the Y-axis direction, and the vertical direction is taken as the Z-axis direction, to construct a world coordinate system corresponding to the moment; The method of performing water surface recognition on the real-time image captured by the drone according to a preset period to obtain a water surface area, and performing contour extraction on the water surface area to obtain a contour point set of the water surface area includes: According to a preset cycle, the latest captured image is obtained as the current image, and the world coordinate system corresponding to the shooting time of the current image is used as the current world coordinate system; The current image is segmented to obtain a water surface area, and the contour of the water surface area is extracted to obtain a contour point set of the water surface area. The contour points in the contour point set take the contour point close to the upper left corner of the image as the starting point and are arranged in a counterclockwise direction.

4. The method for inspecting bank protection by using an unmanned aerial vehicle according to any one of claims 1 to 3, characterized in that: Before obtaining the nearshore point set and the opposite shore point set according to the contour point set, the method further comprises: The contour points at the boundary of the current image are removed from the contour point set.

5. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 3, characterized in that: Before obtaining the nearshore point set and the opposite shore point set according to the contour point set, the method further comprises: Traversing the contour point set, sequentially acquiring a contour point as a current contour point, and using the first contour point acquired as a reference point; Perform inverse perspective transformation on the current contour point to obtain the world coordinates of the current contour point in the current world coordinate system; Calculate the distance between the current contour point and the latest reference point according to their world coordinates in the current world coordinate system; If the distance between the current contour point and the latest reference point is greater than the preset distance threshold, the current contour point is used as the new reference point; After traversing the contour point set, contour points that are not reference points are removed from the contour point set.

6. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 3, characterized in that: If the initial orientation is the right direction after the drone takes off towards the river center, the nearshore point set is the right bank point set, and the opposite bank point set is the left bank point set; if the initial orientation is the left direction after the drone takes off towards the river center, the nearshore point set is the left bank point set, and the opposite bank point set is the right bank point set; The step of obtaining a nearshore point set and an opposite shore point set according to the contour point set is specifically as follows: If a contour point meets the preset right bank point condition, the contour point is recorded in the right bank point set, and the right bank point condition is Y i-1 <Y i <Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |; If a contour point meets the preset left bank point condition, the contour point is recorded in the left bank point set, and the left bank point condition is Y i-1 >Y i >Y i+1 And |Y i+1 -Y i-1 |>|X i+1 -X i-1 |; Among them, (X i , Y i ) represents the world coordinates of the contour point in the current world coordinate system, (X i-1 , Y i-1 ) represents the world coordinates of the previous contour point of the contour point in the contour point set in the current world coordinate system, (X i+1 , Y i+1 ) represents the world coordinates of the next contour point of the contour point in the contour point set in the current world coordinate system.

7. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 1, characterized in that: The world coordinate T of the flight path point corresponding to a contour point in the nearshore point set in the current world coordinate system i =(X Ti , Y Ti , h0), Among them, (X Pi , Y Pi ) represents the world coordinates of a contour point in the nearshore point set in the current world coordinate system, (X Pi-1 , Y Pi-1 ) and (X Pi+1 , Y Pi+1 ) respectively represent the world coordinates of the previous contour point and the next contour point of the contour point in the nearshore point set in the current world coordinate system; d represents the distance from the shore, and h0 represents the preset flight altitude.

8. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 7, characterized in that: The step of adjusting the flight path points according to the contour points in the opposite bank point set comprises: respectively obtaining two flight path points closest to each contour point in the opposite bank point set; Calculate the distance between each contour point and the flight path according to the world coordinates of each contour point and the two flight path points closest to it in the current world coordinate system; Obtain the minimum value of the distance between each contour point and the flight path as the correction distance; If the corrected distance is less than zero, the offshore distance is corrected according to the corrected distance to obtain a new offshore distance, and the step of calculating the corresponding flight path points according to the contour points in the nearshore set is re-executed according to the new offshore distance.

9. The method for inspecting bank protection by using an unmanned aerial vehicle according to claim 8, characterized in that: According to the first distance calculation formula, the distance between each contour point in the opposite bank point set and the flight path is calculated respectively. The first distance calculation formula is: Among them, (X p对 , Y p对 ) represents the world coordinates of a contour point in the opposite bank point set in the current world coordinate system, (X p1 , Y p1 ) and (X p2 , Y p2 ) represents the world coordinates of the two flight path points closest to the contour point in the current world coordinate system, Y p1 <Y p2 .

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images