Unmanned aerial vehicle automatic flight control method based on water area shoreline patrol and storage medium

Abstract

The application discloses an unmanned aerial vehicle automatic flight control method based on water area coastline patrol and a storage medium, and the method comprises the following steps: determining the flight height of the unmanned aerial vehicle according to the maximum identification distance of each identification task; determining the initial flight position of the unmanned aerial vehicle according to the flight height, a preset pitch angle and a preset initial orientation, and controlling the unmanned aerial vehicle to fly forward from the initial flight position according to a preset first flight speed; identifying the water surface of the real-time shooting image of the unmanned aerial vehicle according to a preset first adjustment period, and adjusting the yaw angle according to the water surface identification result, so that the unmanned aerial vehicle flies along the water area coastline to be patrolled. The application can realize automatic flight control of the unmanned aerial vehicle, does not need to manually set a route, has greater convenience in a new environment or a temporary patrol scene, and is more suitable for a situation that the water surface coastline changes greatly.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to an automatic flight control method and storage medium for UAVs based on shoreline patrol in waterways. Background Technology

[0002] In missions involving monitoring waterways and shorelines, two common methods are used to acquire image and video data: riverside video surveillance stations and drone patrols. Image processing techniques are then employed to identify events requiring early warning. Video surveillance stations offer the advantage of continuous, long-term monitoring, but their coverage area is limited. Drone patrols, on the other hand, provide truly wide-area coverage, but their duration is very short. Therefore, these two monitoring methods complement each other.

[0003] However, when using drones for river patrols, routes usually need to be pre-set by humans. Pre-determining fixed patrol routes is not convenient for new environments or temporary patrol scenarios, and it is also not suitable for situations where the water surface and shoreline change significantly. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an automatic flight control method and storage medium for unmanned aerial vehicles (UAVs) based on shoreline patrol, which can realize autonomous shoreline patrol by UAVs without the need for pre-setting routes, save labor costs, and has high versatility.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: an automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol, comprising:

[0006] The flight altitude of the drone is determined based on the preset maximum recognition distance for each recognition task;

[0007] Based on the flight altitude, preset pitch angle, and preset initial orientation, the initial flight position of the UAV is determined, and the UAV is controlled to fly forward from the initial flight position according to the preset first flight speed.

[0008] According to the preset first adjustment cycle, the drone's real-time captured images are used to identify the water surface, and the yaw angle is adjusted according to the water surface identification results so that the drone flies along the shoreline of the water area to be inspected.

[0009] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0010] The beneficial effects of this invention are: it can automatically control the flight process of the UAV according to the identification task, and adjust the yaw angle in real time according to the water surface identification results of the image during the flight, so that the UAV can automatically fly along the shoreline of the water area without the need for manual route setting. Compared with the fixed cruise route, this invention has greater convenience in new environments or temporary patrol scenarios, and is more suitable for situations where the shoreline of the water surface changes greatly. Attached Figure Description

[0011] Figure 1 This is a flowchart of an automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to the present invention.

[0012] Figure 2 This is a schematic diagram of the inverse mapping transformation in Embodiment 1 of the present invention;

[0013] Figure 3 This is a flowchart of the method according to Embodiment 1 of the present invention;

[0014] Figure 4 This is a schematic diagram of yaw angle adjustment during flight along the shoreline of a waterway, according to Embodiment 1 of the present invention.

[0015] Figure 5 This is a schematic diagram of the yaw angle adjustment when flying to the opposite shore according to Embodiment 1 of the present invention. Detailed Implementation

[0016] To explain the technical content, objectives, and effects of the present invention in detail, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0017] Please see Figure 1 An automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol includes:

[0018] The flight altitude of the drone is determined based on the preset maximum recognition distance for each recognition task;

[0019] Based on the flight altitude, preset pitch angle, and preset initial orientation, the initial flight position of the UAV is determined, and the UAV is controlled to fly forward from the initial flight position according to the preset first flight speed.

[0020] According to the preset first adjustment cycle, the drone's real-time captured images are used to identify the water surface, and the yaw angle is adjusted according to the water surface identification results so that the drone flies along the shoreline of the water area to be inspected.

[0021] As can be seen from the above description, the beneficial effects of the present invention are: it can realize automatic flight control of drones without the need for manual route setting, which is more convenient in new environments or temporary patrol scenarios, and is also more suitable for situations where the water surface and shoreline change greatly.

[0022] Furthermore, it also includes:

[0023] At any given moment during the drone's flight, the world coordinate system corresponding to that moment is constructed with the vertical projection point of the drone onto the plane where the water surface is located as the origin, the plane where the water surface is located as the XOY plane, the orientation of the drone's camera as the Y-axis direction, and the vertical direction as the Z-axis direction.

[0024] As described above, the world coordinate system at different times will change with the position of the drone on the horizontal plane and the orientation of the camera.

[0025] Furthermore, determining the drone's flight altitude based on the preset maximum recognition distance for each recognition task specifically involves:

[0026] Obtain the minimum value among the farthest recognition distances for each recognition task;

[0027] An effective area for image recognition is preset in the captured image, and a pixel is determined at the top edge of the effective area;

[0028] The flight altitude is calculated based on the image coordinates of the pixel in the captured image, the minimum value, and a preset set of equations and pitch angle. The set of equations includes an inverse perspective mapping formula and a distance calculation formula. The inverse perspective mapping formula is used to map the pixel in the captured image to the world coordinate system corresponding to the capture time of the captured image. The distance calculation formula is constructed based on the world coordinates of the pixel in the world coordinate system corresponding to the capture time of the captured image and the minimum value.

[0029] As described above, the minimum value among the farthest recognition distances of all recognition tasks is used as a reference for calculating the drone's flight altitude, making a portion of the image (the lower half) the effective recognition area.

[0030] Furthermore, the inverse perspective mapping formula is:

[0031]

[0032]

[0033] The distance calculation formula is d min 2 =X P 2 +Y P 2 ;

[0034] Where (x,y) represents the image coordinates of a pixel in the captured image, (X... p ,Y p) represents the world coordinates of the pixel in the world coordinate system corresponding to the time the image was captured, h represents the flight altitude of the drone, and θ represents the pitch angle of the drone camera; width image and height image α and β represent the width and height of the image, respectively; α and β represent the horizontal and vertical field of view of the drone camera, respectively.

[0035] Furthermore, the step of performing water surface identification on the real-time images captured by the UAV according to a preset first adjustment cycle, and adjusting the yaw angle based on the water surface identification results, so that the UAV flies along the shoreline of the water area to be inspected, specifically involves:

[0036] According to the preset first adjustment cycle, the latest captured image is obtained as the current image, and the world coordinate system corresponding to the capture time of the current image is used as the current world coordinate system;

[0037] The current image is segmented to obtain the water surface region, and the contour of the water surface region is extracted to obtain the contour point set of the water surface region;

[0038] Traverse the set of contour points and perform inverse perspective transformation on each contour point in turn to obtain the world coordinates of each contour point in the current world coordinate system;

[0039] Calculate the distance between each contour point and the origin of the current world coordinate system, and take the contour point with a preset first distance from the origin of the current world coordinate system as the direction anchor point;

[0040] Calculate the angle between the line connecting the origin of the current world coordinate system and the direction anchor point and the Y-axis of the current world coordinate system;

[0041] Adjust the yaw angle within the current first adjustment cycle based on the included angle.

[0042] As described above, by using the water surface segmentation results in the image to control the flight direction of the drone, only the flight action needs to be controlled without requiring the drone to have a precise state, thus having a high degree of tolerance.

[0043] Further, the step of performing image segmentation on the current image to obtain a water surface region, and extracting the contour of the water surface region to obtain a set of contour points of the water surface region, specifically involves:

[0044] Perform image segmentation on the current image to obtain at least one water surface region;

[0045] The contours of each water surface region are extracted, and the area of ​​each contour is calculated.

[0046] Get the set of contour points of the contour with the largest area.

[0047] As described above, by obtaining the water surface area with the largest outline area, interference areas such as small puddles can be filtered out.

[0048] Furthermore, adjusting the yaw angle within the current first adjustment cycle based on the included angle specifically involves:

[0049] The adjustment frequency and unit adjustment angle are determined based on the included angle, the first adjustment cycle, and the preset unit time.

[0050] Adjust the yaw angle according to the adjustment frequency and unit adjustment angle so that the drone's camera is oriented toward the line connecting the origin of the current world coordinate system and the direction anchor point.

[0051] As can be seen from the above description, the stability of angle adjustment can be maintained.

[0052] Furthermore, it also includes:

[0053] When the flight time reaches the preset time threshold, the drone's real-time captured images are used for water surface identification. Based on the water surface identification results, the yaw angle is adjusted so that the drone flies to the opposite bank and flies along the shoreline of the opposite bank.

[0054] Furthermore, when the flight time reaches a preset time threshold, water surface recognition is performed on the real-time images captured by the UAV, and the yaw angle is adjusted according to the water surface recognition result so that the UAV flies to the opposite bank and flies along the shoreline of the opposite bank. Specifically:

[0055] When the flight time reaches a preset time threshold, water surface recognition is performed on the latest captured image to obtain the water surface area.

[0056] Determine the orientation of the water surface area relative to the UAV, and determine the target side to rotate based on the orientation;

[0057] Based on the target side of the rotation and the preset rotation angle, adjust the yaw angle so that the drone camera faces the opposite bank, and control the drone to fly forward according to the preset second flight speed so that the drone flies towards the opposite bank;

[0058] During the drone's flight toward the opposite bank, according to the preset recognition cycle, the latest captured image is used as the current image, and the world coordinate system corresponding to the capture time of the current image is used as the current world coordinate system.

[0059] Perform water surface recognition and contour extraction on the current image to obtain the water surface contour;

[0060] The intersection point of the water surface contour and the midline of the image along the width direction is obtained and used as the first reference point;

[0061] If the ordinate value in the image coordinates of the first reference point is less than the preset height value, then the world coordinates of each contour point of the water surface contour in the current world coordinate system are calculated by using the inverse perspective mapping formula, and a second reference point is determined from the contour points of the water surface contour. The second reference point is located on the rotation target side of the first reference point, and the distance between the first reference point and the second reference point in the current world coordinate system is a preset second distance.

[0062] Based on the world coordinates of the second reference point in the current world coordinate system, calculate the adjustment angle, and adjust the yaw angle according to the preset second adjustment cycle and the adjustment angle, until the angle between the line connecting the first reference point and the second reference point in the world coordinate system at the same time and the Y-axis of the world coordinate system at the same time is less than the preset angle.

[0063] The drone is controlled to fly forward according to a preset first flight speed. At the same time, according to a preset first adjustment cycle, the drone's real-time captured images are used to identify the water surface. Based on the water surface identification results, the yaw angle is adjusted so that the drone flies along the shoreline of the opposite waterway.

[0064] As described above, this allows the drone to fly along one side of the riverbank for a period of time, then fly to the opposite bank and return along the other side, thus making full use of the drone and enabling it to patrol both banks of a section of the river in one round trip.

[0065] Furthermore, the step of adjusting the yaw angle according to the rotating target side and a preset rotation angle to make the drone camera face the opposite bank further includes:

[0066] Obtain the yaw angle adjustment angles from the latest preset number of times in history and sum them up to obtain the cumulative adjustment angle;

[0067] The yaw angle is corrected based on the accumulated adjustment angle.

[0068] As can be seen from the above description, the adjustment of the yaw angle is relative to a section of the coast rather than to a specific point.

[0069] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0070] Example 1

[0071] Please refer to Figure 2-5 Embodiment 1 of the present invention is: an automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol, which can be applied to UAV shoreline patrol missions.

[0072] River patrol tasks are diverse. Those based on ordinary RGB images include monitoring floating objects on the lake surface, water hyacinth and blue-green algae, swimming and fishing, illegal construction along the riverbank, and garbage and construction waste accumulation. Those based on thermal imaging channels include monitoring personnel and engineering vehicles, and monitoring sand dredging operations. These image recognition tasks all aim to identify target objects in the image and can be categorized as object detection and image segmentation tasks. In this embodiment, both object detection and image segmentation methods are based on neural network models trained on image sample datasets. For example, YOLOv8 can be chosen for object detection, and Mask2Former can be chosen for image segmentation, both of which have excellent performance. The dataset construction only requires samples from conventional viewpoints. Each recognition task has a maximum recognition distance d. task and the range of recognition angles (θ) 1 task θ 2 task The recognition angle range is 0° to 90°, and the maximum recognition distance and recognition angle range can be set based on experience.

[0073] The imaging principle of a drone is no different from that of a regular surveillance camera. At any given moment, the imaging range of the drone over the water surface is determined by the flight altitude h, pitch angle θ, and camera field of view (FOV). The pitch and yaw angles of the camera on the drone are determined by the drone's flight attitude and the gimbal. During stable cruise, the target pitch angle is achieved by controlling the gimbal. In this embodiment, the pitch angle θ is the camera's pitch angle relative to the ground. With the gimbal fixed facing forward, the yaw angle is determined by the drone's orientation, which can be obtained directly from the compass direction on the sensor as the yaw angle φ. Since the roll angle caused by shaking will cause rotation of the image, this is offset by rotating the center of the image, and will not be discussed in detail here.

[0074] like Figure 2 As shown, in this embodiment, at a certain moment during the drone's flight, the origin O is established using the vertical projection point of the drone onto the plane containing the water surface, the plane containing the water surface is designated as the XOY plane, the orientation of the drone's camera (i.e., the yaw angle direction) is taken as the Y-axis, and the vertically upward direction is defined as the positive Z-axis. In other words, in this embodiment, the world coordinate system changes with the drone's position (its position in the horizontal plane) and the orientation of the camera.

[0075] In addition, for images captured by drones, an image coordinate system is constructed with the top left corner of the image as the origin, the rightward direction as the positive X-axis, and the downward direction as the positive Y-axis.

[0076] Inverse perspective transformation, using the inverse perspective mapping formula, can map a point (x, y) on an image captured at a certain moment to the XOY plane of the world coordinate system at that moment. The specific formula is as follows. The water surface is a plane, suitable for inverse perspective transformation. Although the ground on the shore is not strictly flush with the water surface, considering the limited detection range of the method in this embodiment, the ground can be approximated as having zero height.

[0077]

[0078]

[0079] Where (x,y) represents the image coordinates of a pixel in the captured image in the image coordinate system, (X,y) p ,Y p ) represents the world coordinates of that point (i.e., the coordinates in the world coordinate system at the time the image was captured), h represents the height of the drone camera relative to the water surface (i.e., the drone's flight altitude), and θ represents the pitch angle of the drone camera; width image and height image α represents the resolution of the captured image, i.e., the width and height of the captured image; α and β are the intrinsic parameters of the drone camera, representing the horizontal and vertical field of view, which are determined by the camera's focal length.

[0080] like Figure 3 As shown, this method includes the following steps:

[0081] S101: Determine the drone's flight altitude based on the preset maximum recognition distance for each recognition task.

[0082] Specifically, find the minimum value d from the farthest recognition distance of all recognition tasks. min In this embodiment, the lower half of the drone's image is taken as the effective area for image recognition. Therefore, at pixel x = 0, y = (1 / 2)height image At this point, using the inverse perspective mapping formula described above, its world coordinates (X) in the world coordinate system can be calculated. p ,Y p And the distance between the world coordinates and the UAV's projected position (i.e., the origin) satisfies d min 2 =Y P 2 +X P 2 Substituting the values ​​into the inverse perspective mapping formula above, we can solve the equations simultaneously to obtain the value of h, which is the constant flight altitude for subsequent UAV cruise. In this embodiment, the pitch angle θ is fixed at 30°.

[0083] S102: Determine the initial flight position of the UAV based on the flight altitude, preset pitch angle, and preset initial orientation (i.e., the direction of the shoreline of the water area to be inspected), and control the UAV to fly forward from the preset first flight speed, with the initial flight position as the starting point for inspection.

[0084] This step is the flight initialization. First, place the drone on the water's edge, facing the shoreline of the area to be inspected (facing the water, with two selectable directions: left and right). Then, control the drone to take off and reach the flight altitude h calculated in step S101, adjusting the pitch angle θ to 30°, which is close to the imaging viewpoint of most samples. Finally, control the drone to fly forward at a certain speed.

[0085] S103: According to the preset first adjustment cycle, perform water surface recognition on the real-time images captured by the UAV, and adjust the yaw angle according to the water surface recognition results so that the UAV flies along the shoreline of the water area to be inspected.

[0086] Specifically, it includes the following steps:

[0087] S1031: According to the preset first adjustment cycle, acquire the latest captured image as the current image, and use the world coordinate system corresponding to the capture time of the current image as the current world coordinate system.

[0088] S1032: Perform image segmentation on the current image to obtain the water surface region, and extract the contour of the water surface region to obtain the contour point set of the water surface region.

[0089] That is, the current image is segmented into water surface regions to obtain at least one water surface region; then the contours of each water surface region are extracted and the contour area is calculated. The contour C = {(x,y)} with the largest area is selected as the analysis object, where (x,y) are the image coordinates of the contour points.

[0090] S1033: Traverse the set of contour points and perform inverse perspective transformation on each contour point in turn to obtain the world coordinates of each contour point in the current world coordinate system. That is, calculate the world coordinates (X, Y, X) of each contour point in the current world coordinate system using the inverse perspective mapping formula described above. p ,Y p ).

[0091] S1034: Calculate the distance between each contour point and the origin of the current world coordinate system, and use the contour point with a preset first distance from the origin of the current world coordinate system as the direction anchor point.

[0092] like Figure 4 As shown, through the formula Calculate the distance between the world coordinates of each contour point in the current world coordinate system and the origin O of the current world coordinate system (i.e., the vertical projection position of the UAV). In this embodiment, the preset first distance is 50 meters, that is, the contour point with d = 50 meters is selected as the direction anchor point P.

[0093] S1035: Calculate the angle between the line connecting the origin of the current world coordinate system and the direction anchor point and the Y-axis of the current world coordinate system.

[0094] like Figure 4 As shown, the angle ρ between OP and the Y-axis is calculated as ρ = arctan(X). p / Y p ).

[0095] S1036: Adjust the yaw angle within the current first adjustment cycle according to the included angle.

[0096] Specifically, the adjustment frequency and unit adjustment angle are determined based on the included angle, the first adjustment cycle, and the preset unit time; the yaw angle is adjusted based on the adjustment frequency and unit adjustment angle so that the UAV's camera orientation tends to the line connecting the origin of the current world coordinate system and the direction anchor point.

[0097] For example, in this embodiment, the first adjustment cycle is 5 seconds, and the unit time is 1 second, that is, the yaw angle is corrected by Δρ = ρ / 5 every second, so as to maintain the stability of the angle adjustment and keep the drone roughly at the waterfront boundary and always facing the waterfront directly in front.

[0098] In this embodiment, clockwise is taken as the positive direction of the yaw angle. Therefore, if the yaw angle needs to be adjusted to the right of the center line of the image, the yaw angle is increased, and vice versa.

[0099] S104: When the flight time reaches the preset time threshold, the drone performs water surface recognition on the real-time captured images and adjusts the yaw angle according to the water surface recognition results so that the drone flies to the opposite bank and flies along the shoreline of the opposite bank.

[0100] That is, set a time threshold t threshold Once the drone's flight time reaches this threshold, it will begin searching for the opposite bank. If the application scenario is patrolling a large lake, a search for the opposite bank is necessary; if the application scenario is a small reservoir, patrolling along one bank direction is sufficient, and there's no need to patrol the opposite bank. For both cases, the time threshold t can be set... threshold Setting it to infinity causes the drone to patrol along a waterfront direction until its battery is depleted and it begins its return journey.

[0101] If the actual scenario is to inspect a river, the drone can fly along one side of the riverbank for a period of time, then fly to the opposite bank and return along the other side. This allows the drone to be fully utilized, enabling it to inspect both banks of a section of the river in one round trip.

[0102] Specifically, it includes the following steps:

[0103] S1041: When the flight time reaches the preset time threshold, perform water surface recognition on the latest captured image to obtain the water surface area.

[0104] S1042: Determine the orientation of the water surface area relative to the UAV, and determine the rotating target side based on the orientation.

[0105] S1043: Adjust the yaw angle according to the rotating target side and the preset rotation angle so that the drone camera faces the opposite bank, and control the drone to fly forward according to the preset second flight speed so that the drone flies towards the opposite bank.

[0106] In this embodiment, the preset rotation angle is 90°. This means that water surface recognition is performed on the currently captured image. By analyzing the water surface area, it can be determined whether the water area is to the left or right of the drone, and that side is used as the rotation target side. If the water area is to the right of the drone, the yaw angle is adjusted 90° clockwise; if the water area is to the left of the drone, the yaw angle is adjusted 90° counterclockwise, thereby adjusting the drone's orientation to face the opposite bank.

[0107] Furthermore, before performing rotational adjustments, the yaw angle adjustments from the most recent preset number of times (e.g., 10 times) can be obtained and accumulated to obtain the cumulative adjustment angle Δρ. total First perform Δρ total The yaw angle correction of / 2 means that the yaw angle is adjusted relative to a section of the coast rather than to a specific point.

[0108] S1044: During the flight of the drone towards the opposite bank, according to the preset recognition cycle, the latest captured image is used as the current image, and the world coordinate system corresponding to the capture time of the current image is used as the current world coordinate system.

[0109] S1045: Perform water surface recognition and contour extraction on the current image to obtain the water surface contour.

[0110] S1046: Obtain the intersection point of the water surface contour and the centerline of the image along the width direction, and use it as the first reference point.

[0111] In this embodiment, the recognition cycle is 2 seconds, that is, the water surface is recognized and the contour is extracted once every two seconds, and the water surface contour is used to extract the contour of the water surface and the line x = (1 / 2)width. image The intersection point is taken as the first reference point A.

[0112] S1047: When the ordinate value in the image coordinates of the first reference point is less than the preset height value, the world coordinates of each contour point of the water surface contour in the current world coordinate system are calculated by using the inverse perspective mapping formula, and a second reference point is determined from the contour points of the water surface contour. The second reference point is located on the rotation target side of the first reference point, and the distance between the first reference point and the second reference point in the current world coordinate system is a preset second distance.

[0113] In this embodiment, the preset height value is half the image height, and the preset second distance is 50 meters.

[0114] like Figure 5 As shown, when the ordinate y in the image coordinates of the first reference point A... A <height image At / 2, the world coordinates of each contour point of the water surface contour in the current world coordinate system are calculated using the inverse perspective mapping formula. This is on the rotating target side of the first reference point A (e.g., if it's the left side, then x < width). image Find the second reference point B (point A = 2 / 2). The distance between the first reference point A and the second reference point B in the current world coordinate system is 50 meters.

[0115] S1048: Calculate the adjustment angle based on the world coordinates of the second reference point in the current world coordinate system, and adjust the yaw angle according to the preset second adjustment cycle and the adjustment angle, until the angle between the line connecting the first reference point and the second reference point in the world coordinate system at the same time and the Y-axis of the world coordinate system at the same time is less than the preset angle.

[0116] The world coordinates (X, X) of the second reference point B in the current world coordinate system can be calculated using the inverse perspective mapping formula. B ,Y B While the drone flies forward, it adjusts its yaw angle using the second reference point B as an anchor point. The adjustment angle is Δρ' = arctan(X). B / Y B This correction is performed once per second until |Y B -Y A |>|X B -X A | That is, in the world coordinate system corresponding to the same moment, the angle between the line connecting points A and B and the Y-axis is less than 45°.

[0117] S1049: Control the UAV to fly forward according to the preset first flight speed, and at the same time, perform water surface recognition on the real-time captured images of the UAV according to the preset first adjustment cycle, and adjust the yaw angle according to the water surface recognition results so that the UAV flies along the shoreline of the opposite water area.

[0118] After adjusting in step S1048, the drone has been adjusted to the direction of travel on the opposite bank. At this time, control the drone to fly forward, and refer to step S103 to adjust the yaw angle in real time during the flight so that the drone flies along the shoreline of the other side of the water.

[0119] Furthermore, the video stream captured during the drone's flight can be used to perform subsequent image recognition tasks and merge and map the recognition results onto a map.

[0120] This embodiment controls the drone's flight direction by utilizing the water surface segmentation results in the image. It only controls the flight action without requiring the drone to have a precise state, thus having a high tolerance. Furthermore, it allows for a smoother forward movement without frequent turns that could degrade image quality and affect subsequent image recognition tasks.

[0121] Example 2

[0122] This embodiment is a computer-readable storage medium corresponding to the above embodiments, on which a computer program is stored. When the program is executed by a processor, it implements the various steps of the automatic flight control method for UAVs based on shoreline patrol in the above embodiments and can achieve the same technical effect, which will not be repeated here.

[0123] In summary, the present invention provides an automatic flight control method and storage medium for unmanned aerial vehicles (UAVs) based on shoreline patrol. This method can automatically control the flight process of the UAV according to the identification task and adjust the yaw angle in real time according to the water surface identification results of the image during the flight. This allows the UAV to automatically fly along the shoreline without the need for manual route setting. Compared with a fixed cruise route, the present invention has greater convenience in new environments or temporary patrol scenarios and is more suitable for situations where the shoreline changes greatly.

[0124] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol, characterized in that, include: The flight altitude of the drone is determined based on the preset maximum recognition distance for each recognition task; Based on the flight altitude, preset pitch angle, and preset initial orientation, the initial flight position of the UAV is determined, and the UAV is controlled to fly forward from the initial flight position according to the preset first flight speed. According to the preset first adjustment cycle, the drone's real-time captured images are used to identify the water surface, and the yaw angle is adjusted according to the water surface identification results so that the drone flies along the shoreline of the water area to be inspected. Also includes: At a certain moment during the drone's flight, the world coordinate system corresponding to that moment is constructed with the vertical projection position of the drone on the plane where the water surface is located as the origin, the plane where the water surface is located as the XOY plane, the orientation of the drone's camera as the Y-axis direction, and the vertical direction as the Z-axis direction. The process of determining the drone's flight altitude based on the preset maximum recognition distance for each recognition task is as follows: Obtain the minimum value among the farthest recognition distances for each recognition task; An effective area for image recognition is preset in the captured image, and a pixel is determined at the top edge of the effective area; The flight altitude is calculated based on the image coordinates of the pixel in the captured image, the minimum value, and a preset set of equations and pitch angle. The set of equations includes an inverse perspective mapping formula and a distance calculation formula. The inverse perspective mapping formula is used to map the pixel in the captured image to the world coordinate system corresponding to the capture time of the captured image. The distance calculation formula is constructed based on the world coordinates of the pixel in the world coordinate system corresponding to the capture time of the captured image and the minimum value. The inverse perspective mapping formula is: ， ； The distance calculation formula is as follows: ; Where (x,y) represents the image coordinates of a pixel in the captured image, (X... p ,Y p ) represents the world coordinates of the pixel in the world coordinate system corresponding to the time the image was captured, h represents the flight altitude of the drone, and θ represents the pitch angle of the drone camera; width image and height image α and β represent the width and height of the image, respectively; α and β represent the horizontal and vertical field of view of the drone camera, respectively.

2. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 1, characterized in that, The process involves performing water surface identification on real-time images captured by the UAV according to a preset first adjustment cycle, and adjusting the yaw angle based on the water surface identification results to ensure the UAV flies along the shoreline of the water area to be inspected. Specifically: According to the preset first adjustment cycle, the latest captured image is obtained as the current image, and the world coordinate system corresponding to the capture time of the current image is used as the current world coordinate system; The current image is segmented to obtain the water surface region, and the contour of the water surface region is extracted to obtain the contour point set of the water surface region; Traverse the set of contour points and perform inverse perspective transformation on each contour point in turn to obtain the world coordinates of each contour point in the current world coordinate system; Calculate the distance between each contour point and the origin of the current world coordinate system, and take the contour point with a preset first distance from the origin of the current world coordinate system as the direction anchor point; Calculate the angle between the line connecting the origin of the current world coordinate system and the direction anchor point and the Y-axis of the current world coordinate system; Adjust the yaw angle within the current first adjustment cycle based on the included angle.

3. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 2, characterized in that, The process of segmenting the current image to obtain the water surface region and extracting the contour of the water surface region to obtain the contour point set of the water surface region is as follows: Perform image segmentation on the current image to obtain at least one water surface region; The contours of each water surface region are extracted, and the area of ​​each contour is calculated. Get the set of contour points of the contour with the largest area.

4. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 2, characterized in that, The adjustment of the yaw angle within the current first adjustment cycle based on the included angle specifically involves: The adjustment frequency and unit adjustment angle are determined based on the included angle, the first adjustment cycle, and the preset unit time. Adjust the yaw angle according to the adjustment frequency and unit adjustment angle so that the drone's camera is oriented toward the line connecting the origin of the current world coordinate system and the direction anchor point.

5. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 1, characterized in that, Also includes: When the flight time reaches the preset time threshold, the drone's real-time captured images are used for water surface recognition. Based on the water surface recognition results, the yaw angle is adjusted so that the drone flies to the opposite bank and flies along the shoreline of the opposite bank.

6. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 5, characterized in that, When the flight time reaches a preset time threshold, the real-time images captured by the UAV are used for water surface recognition. Based on the water surface recognition results, the yaw angle is adjusted so that the UAV flies to the opposite bank and flies along the shoreline of the opposite bank. Specifically: When the flight time reaches a preset time threshold, water surface recognition is performed on the latest captured image to obtain the water surface area. Determine the orientation of the water surface area relative to the UAV, and determine the target side to rotate based on the orientation; Based on the target side of the rotation and the preset rotation angle, adjust the yaw angle so that the drone camera faces the opposite bank, and control the drone to fly forward according to the preset second flight speed so that the drone flies towards the opposite bank; During the drone's flight toward the opposite bank, according to the preset recognition cycle, the latest captured image is used as the current image, and the world coordinate system corresponding to the capture time of the current image is used as the current world coordinate system. Perform water surface recognition and contour extraction on the current image to obtain the water surface contour; The intersection point of the water surface contour and the midline of the image along the width direction is obtained and used as the first reference point; If the ordinate value in the image coordinates of the first reference point is less than the preset height value, then the world coordinates of each contour point of the water surface contour in the current world coordinate system are calculated by using the inverse perspective mapping formula, and a second reference point is determined from the contour points of the water surface contour. The second reference point is located on the rotation target side of the first reference point, and the distance between the first reference point and the second reference point in the current world coordinate system is a preset second distance. Based on the world coordinates of the second reference point in the current world coordinate system, calculate the adjustment angle, and adjust the yaw angle according to the preset second adjustment cycle and the adjustment angle, until the angle between the line connecting the first reference point and the second reference point in the world coordinate system at the same time and the Y-axis of the world coordinate system at the same time is less than the preset angle. The drone is controlled to fly forward according to a preset first flight speed. At the same time, according to a preset first adjustment cycle, the drone's real-time captured images are used to identify the water surface. Based on the water surface identification results, the yaw angle is adjusted so that the drone flies along the shoreline of the opposite waterway.

7. The automatic flight control method for unmanned aerial vehicles (UAVs) based on shoreline patrol according to claim 6, characterized in that, Before adjusting the yaw angle according to the rotating target side and the preset rotation angle to make the drone camera face the opposite bank, the following further includes: Obtain the yaw angle adjustment angles from the latest preset number of times in history and sum them up to obtain the cumulative adjustment angle; The yaw angle is corrected based on the accumulated adjustment angle.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images