A UAV flight control method for capturing black smoke from ships

Abstract

The present invention discloses a drone flight control method for capturing black smoke from ships, comprising the following steps: based on the sun position and the orientation of the ship chimney, setting the relative position of the drone accompanying the ship; according to the AIS data of the ship, setting the initial flight speed and flight direction of the drone; locking the initial position of the ship chimney in the drone screen, setting a tracking frame; identifying the center position coordinates and area of ​​the tracking frame; based on the center position coordinates and area, respectively identifying the offset of the center position and the tracking frame zoom ratio; according to the offset of the center position and the tracking frame zoom ratio, continuously adjusting and controlling the accompanying flight speed of the drone. The present invention can automatically control the drone to accompany the ship to ensure the video recording effect of capturing the black smoke from the ship, provide an effective data source for the subsequent video-based ship black smoke level determination work, and support the maritime authorities' supervision of the black smoke from ships.

Description

Technical Field

[0001] The invention relates to the technical field of unmanned aerial vehicle flight control, and in particular to a unmanned aerial vehicle flight control method for capturing black smoke from ships. Background Art

[0002] Cameras (Lingelmann blackness colorimetric cards are built into the camera lens) have been widely used to detect black smoke emissions from fixed pollution sources (steel mills, power plants, etc.), but are not suitable for mobile pollution sources such as ships. Ships will inevitably produce short-term black smoke under variable operating conditions. Continuous black smoke emission under uniform speed navigation is a violation. Capturing the black smoke from ships requires continuous recording of videos for more than a certain period of time. However, under normal circumstances, videos recorded on patrol boats and on shore cannot guarantee that the ship is relatively still in the picture, which brings a huge workload for the subsequent second-by-second or frame-by-frame analysis of blackness level determination. Following the ship to capture and obtain stable video of the chimney mouth will help simplify the subsequent video-based ship black smoke level determination work and greatly improve work efficiency. Summary of the invention

[0003] The purpose of the present invention is to provide a UAV flight control method for capturing ship black smoke, so as to solve the problems existing in the above-mentioned prior art, automatically control the UAV to fly with the ship, ensure the video recording effect of capturing the ship's black smoke, provide an effective data source for subsequent video-based ship black smoke level determination work, and support the maritime authorities' supervision of ship black smoke.

[0004] To achieve the above object, the present invention provides the following solution: The present invention provides a UAV flight control method for capturing black smoke from a ship, comprising the following steps:

[0005] Based on the position of the sun and the ship's chimney, set the relative position of the drone accompanying the ship;

[0006] According to the ship's positioning data, set the drone's initial flight speed and flight direction;

[0007] Lock the initial position of the ship chimney in the drone image and set a tracking frame;

[0008] Identify the center position coordinates and area of ​​the tracking frame;

[0009] Based on the center position coordinates and the area, respectively identifying an offset of the center position and a tracking frame zoom factor;

[0010] The accompanying flight speed of the drone is continuously adjusted according to the offset of the center position and the zoom ratio of the tracking frame.

[0011] Optionally, based on the position of the sun and the orientation of the ship's chimney, setting the relative position of the drone accompanying the ship's flight includes setting the drone's orientation, the drone's altitude, and the distance between the drone and the ship's chimney.

[0012] Optionally, based on the drone position, including:

[0013] The position of the sun is determined. When the sun is located behind the direction of the UAV facing the chimney of the ship, the UAV is set to a non-backlight position; when the sun is located in front of the ship, the UAV is set to the non-backlight position, and the angle between the UAV and the heading of the ship is greater than a first angle threshold; when the UAV is facing the chimney of the ship, the angle between the UAV and the ship's plume is greater than a second angle threshold.

[0014] Optionally, the method for determining whether the relative position of the UAV accompanying the ship is suitable is as follows: calculating the latitude difference between the UAV latitude and the ship latitude, calculating the longitude difference between the UAV longitude and the ship longitude, and determining whether the ratio of the latitude difference to the longitude difference is within a preset orientation set; if so, the relative position of the UAV accompanying the ship is suitable; if not, the relative position of the UAV accompanying the ship is not suitable.

[0015] Optionally, the orientation set is (A s ±90° range)∩(A v ±30° range) ∩(A e ±30° range), where A s is the solar azimuth, A s Ship heading angle, A e is the horizontal azimuth of the ship plume.

[0016] Optionally, setting the drone altitude includes:

[0017] The height of the UAV is set to be lower than the height of the chimney of the ship, and the elevation angle between the UAV and the chimney of the ship is less than a third angle threshold.

[0018] Optionally, setting the distance between the drone and the chimney of the ship includes:

[0019] The horizontal distance between the UAV and the chimney of the ship is set to be no less than a first distance threshold, and the proportion of the chimney of the ship in the picture is set to be within a set threshold range.

[0020] Optionally, setting the tracking frame includes: in the drone image, selecting a rectangular range covering the chimney of the ship as the tracking frame.

[0021] Optionally, continuously adjusting the accompanying flight speed of the drone according to the offset of the center position and the zoom ratio of the tracking frame includes:

[0022] Based on the zoom ratio, a first threshold is set, and if the first threshold is exceeded, the flight speed of the drone is adjusted, otherwise, the flight speed is not adjusted;

[0023] Based on the offset, a second threshold is set, and if the second threshold is exceeded, the flight direction angle of the UAV is adjusted, otherwise, no adjustment is made;

[0024] If the flight speed and direction of the UAV are not adjusted within a set time range, it is confirmed that the flight speed and direction of the UAV are adjusted to be stable, and video recording is started; during the video recording, the flight speed of the UAV is continuously adjusted according to the zoom ratio and the offset until the video recording ends.

[0025] The present invention discloses the following technical effects:

[0026] The present invention provides a UAV flight control method for capturing black smoke from a ship, which guides the UAV to select an accompanying flight position suitable for capturing black smoke from the ship, ensures that the capturing angle is perpendicular to the exhaust plume of the ship and avoids a backlighting environment; after locking the chimney of the ship as a tracking target, the UAV can be automatically controlled to fly with the ship, ensures that the UAV is stationary relative to the ship, and ensures the video recording effect of capturing black smoke from the ship, thereby providing an effective data source for subsequent video-based determination of the level of black smoke from the ship, and supporting the supervision of black smoke from ships by maritime authorities. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 This is a flow chart of a UAV flight control method for capturing black smoke from a ship according to an embodiment of the present invention;

[0029] Figure 2 4 situations in which the relative positions of the UAV accompanying the ship are set in the embodiment of the present invention: Figure (a) shows that the wind speed is close to the ship speed and the sun is in front; Figure (b) shows that the wind speed is close to the ship speed and the sun is in the back; Figure (c) shows the normal situation and the sun is in front; Figure (d) shows the normal situation and the sun is in the back;

[0030] Figure 3 A schematic diagram of the relative positions of the drones satisfying the flight altitude and distance conditions in the embodiments of the present invention;

[0031] Figure 4Schematic diagram of correct and incorrect locking of the chimney opening position in the drone image in an embodiment of the present invention, with the white frame being correct and the black frame being incorrect. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] The present invention provides a UAV flight control method for capturing black smoke from ships, referring to Figure 1 , including the following steps:

[0035] S1. Set the relative position of the UAV accompanying the ship.

[0036] When setting the relative position of the drone accompanying the ship, it is necessary to fully refer to the relevant information of the sun's position and the ship's chimney, that is, to use the sun's position and the ship's chimney as a reference. During the setting process, the azimuth requirements, height conditions and distance conditions must be met. Figure 2 As shown, specifically including:

[0037] In order to ensure the visual effect of capturing the black smoke from the ship, the relative position needs to meet three orientation conditions:

[0038] (1) When the sun is behind the direction of the drone facing the ship's chimney, adjust the drone's position so that the drone is in a non-backlit area to avoid shooting in a backlit environment;

[0039] (2) When the sun is in front of the ship, the drone must not only avoid backlight, but also avoid the hull itself to avoid collision. That is, the angle between the drone and the ship’s heading must not be less than 30°.

[0040] (3) When the UAV is facing the direction of the ship's chimney, the angle formed with the horizontal direction of the exhaust plume shall not be less than 30°; under the special conditions where the ship speed and wind speed are close, the ship's plume mainly diffuses upward, and the UAV can meet this angle requirement at any position.

[0041] After the drone's orientation is set, to determine whether the drone's orientation is appropriate, you only need to determine the direction of the drone's nose and the drone's nose azimuth angle A. u=tan((UAV latitude - ship latitude) ÷(UAV longitude - ship longitude)) whether it belongs to the orientation set. In this embodiment, the orientation set is (A s ±90° range)∩(A v ±30° range)∩(A e ±30°). The longitude and latitude of the drone come from its own GPS data, and the longitude and latitude of the ship come from the AIS data publicly broadcast by the ship.

[0042] A s is the solar azimuth, A s Ship heading angle, A e A is the horizontal azimuth of the ship’s smoke plume. s The calculation of is relatively complicated, and the specific calculation formula is shown in formula (1):

[0043] A s =Arccos((sinφ×sinH s –sinδ)÷(cosφ×cosH s )) (1)

[0044] Where φ is the latitude of the ship, H s is the solar altitude angle, and

[0045] H s =Arcsin(sinφ×sinδ+cosφ×cosδ×cost), δ is the solar declination, δ=0.006918-0.399912cos(b)+0.070257sin(b)-0.006758cos(2b)+0 .000907sin(2b)-0.002697cos(3b)+0.00148sin(3b), where b=2×PI× (N-1)÷365, N is the number of days starting from January 1st each year, PI is pi, t is the hour angle, and t=(local time-12)×15°, and the local time ranges from 0 to 24.

[0046] In order to ensure that the ship's smoke plume is not disturbed by water bodies, coastlines, etc., the relative position needs to meet two height conditions:

[0047] (1) The altitude of the drone is lower than that of the ship's chimney, which is reflected in the chimney being above the horizon in the captured image.

[0048] (2) The angle of the drone looking up at the ship's chimney is less than 30°.

[0049] Due to the lack of size parameter data for ships and chimneys, the height condition is determined subjectively by the drone pilot based on the images transmitted back. In this embodiment, a DJI Phantom 4 Pro v2 drone is used for shooting. The equivalent focal length of the drone's built-in camera is 24mm. According to calculations, the drone's upper and lower field of view width is approximately 53°. When shooting horizontally, the distant horizon is located in the center of the picture, and the maximum pitch angle of the object in the picture is 26.5°. Therefore, as long as the object appears in the picture, it can be guaranteed that "the angle of the drone looking up at the ship's chimney is less than 30°"; as long as the chimney is higher than the horizon, it proves that "the height of the drone is lower than the height of the ship's chimney", such as Figure 3 As shown in the figure, the relative position of the drone meets the flight altitude and distance conditions to ensure that the ship exhaust plume is not disturbed by water bodies, coastlines, etc. and is clearly visible.

[0050] In order to balance flight safety and ship plume coverage, two distance conditions need to be met:

[0051] (1) The horizontal distance between the drone and the ship's chimney shall not be less than 30m. This can be determined subjectively based on the images sent back by the drone, or calculated based on the latitude and longitude of the drone and the latitude and longitude of the ship.

[0052] (2) The proportion of the ship's chimney in the picture is between 1% and 10%. When the chimney is facing upward, the proportion is the ratio of the chimney diameter to the horizontal length of the picture; when the chimney is facing backward, the proportion is the ratio of the chimney diameter to the vertical length of the picture. Figure 4 As shown, the chimney is facing left, the vertical length is 14 pixels, and the vertical length of the entire screen is 1080 pixels. The chimney occupies 1.3% of the total area, which is between 1% and 10%, ensuring that the chimney is clearly visible in the screen.

[0053] Due to the lack of size parameter data for ship chimneys, drone pilots make subjective judgments based on the images transmitted back to ensure that the chimney mouth is clearly visible in the image.

[0054] After setting, when the UAV is in a relative position to fly with a ship, the UAV is made to be perpendicular to the exhaust plume of the ship and avoid being in a backlight environment.

[0055] S2. Set the initial flight speed and direction of the drone.

[0056] According to the AIS data publicly broadcast by the ship, the ship speed data V0 of the most recent time is extracted as the initial speed of the drone flight. In this embodiment, using the Ship Information Network app, click on the target ship, record the degree of the track direction in the pop-up window, which is 0 degrees to the north and 90 degrees to the east; record the speed in the pop-up window. In an ideal situation, the drone flight trajectory can be parallel to the ship's navigation trajectory, and the ship is relatively still in the drone screen; the actual situation is not absolute, but the ship moves slowly in the drone screen, which provides convenience for the subsequent locking of the chimney mouth and provides a benchmark for fine-tuning the drone flight control, and the adjustment range will be relatively small.

[0057] According to the position of the ship's chimney in the drone image, rotate the drone so that the ship's chimney is as centered as possible.

[0058] S3. Lock the chimney opening at its initial position in the drone image.

[0059] In the drone image, a rectangular range covering the chimney of the ship is manually selected as the tracking frame. The tracking frame is used to lock the initial position of the chimney in the drone image, providing a tracking target for the drone to fly with the ship. Figure 4 As shown, the rectangular range needs to meet two conditions:

[0060] (1) The range cannot be too large. On the one hand, if the range is too large, the calculation of the tracking algorithm will increase, and the tracking algorithm will run slower, which is not conducive to fast tracking; on the other hand, if the range is too large, it is easy to include interference background. Since the background is at different distances from the ship's chimney, the moving speed in the picture is inconsistent, which can easily lead to tracking confusion, or even failure due to tracking the background.

[0061] (2) The range cannot be too small. If the range is too small, it is easy to lack corners, textures and other feature points within the range, which may lead to tracking failure.

[0062] In this embodiment, the ratio of the length and width of the range to the length and width of the picture is between 1 / 10 and 1 / 3, specifically, the length ratio is 0.288 and the width ratio is 0.207. Figure 4 shown.

[0063] Identify the center position coordinates and area of ​​the tracking frame to provide a reference for the subsequent tracking algorithm to identify the offset and scaling factor of the tracking frame.

[0064] The tracking frame is a rectangle with 4 corners. Each corner has a row and column number in the image. If it is from row m to row n horizontally, the number of pixels occupied horizontally is nm, which is the length of the tracking frame. The horizontal coordinate of the center position coordinate is (m+n) / 2. If it is from column a to column b vertically, the number of pixels occupied vertically is ba, which is the width of the length of the tracking frame. The vertical coordinate of the center position coordinate is (a+b) / 2. The area of ​​the tracking frame is the length multiplied by the width, and the unit is pixel.

[0065] S4. Continue to fine-tune the flight speed of the drone.

[0066] Keep running the tracking algorithm and adjusting the flight speed of the drone, with a frequency of no less than 1s. Keep the chimney in a relatively fixed position within the frame by continuously fine-tuning the flight speed of the drone.

[0067] The tracking algorithm identifies the offset d of the center position of the tracking frame and the zoom ratio m of the tracking frame. The tracking algorithm requires the ability to zoom in and out of the image by a certain ratio for resampling, and identifies the optimal result based on the matching of different zoom ratios before and after to determine the zoom ratio of the target.

[0068] According to the zoom ratio m, a reasonable zoom ratio threshold is set. If it exceeds the threshold, the flight speed of the drone is adjusted to V=V0÷m, otherwise it is not adjusted, where m is the real-time tracking frame area when it is in the initial state, and the area unit is the number of pixels. In this embodiment, the zoom ratio threshold range is 0.9 to 1.1, and the flight speed is adjusted when it is greater than 1.1 or less than 0.9. In the picture, if the chimney is enlarged, it means that the drone is flying too fast and getting closer, and the flight speed of the drone should be reduced; if the chimney is reduced, it means that the drone is flying too slow and getting farther and farther, and the flight speed of the drone should be increased.

[0069] According to the offset d, a reasonable offset threshold is set. If it exceeds the threshold, the flight direction of the drone is adjusted by 1°, otherwise no adjustment is made. In this embodiment, the setting of the offset threshold is calculated based on the equivalent focal length f (in mm) of the drone's built-in camera. The equivalent focal length is the default CMOS length of 36 mm. Then the angle of the horizontal field of view is equal to 2×arctan(18 / f). If the number of pixels in the length of the picture is n, then 1° represents n÷(2×arctan(18 / f)) pixels. If d exceeds these pixels, then adjust it.

[0070] If the flight speed and direction of the drone are not adjusted within the continuous time t, it is confirmed that the flight speed and direction of the drone have been adjusted to be stable, and the video recording is started. During the video recording, the tracking algorithm and the flight speed of the drone continue to be run until the video recording ends.

[0071] The UAV flight control method for capturing black smoke from a ship provided by the present invention enables the UAV to maintain a vertical position to shoot the exhaust plume of the ship, thereby avoiding changes in the visual effect of the black smoke in a backlight environment, and the UAV is stationary relative to the ship to ensure that the ship is almost motionless in the video.

[0072] In the description of the present invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0073] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0074] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A UAV flight control method for capturing black smoke from a ship, characterized in that: The following steps are involved: Based on the position of the sun and the ship's chimney, set the relative position of the drone accompanying the ship; According to the ship's positioning data, set the drone's initial flight speed and flight direction; Lock the initial position of the ship chimney in the drone image and set a tracking frame; Identify the center position coordinates and area of ​​the tracking frame; Based on the center position coordinates and the area, respectively identifying an offset of the center position and a tracking frame zoom factor; The accompanying flight speed of the drone is continuously adjusted according to the offset of the center position and the zoom ratio of the tracking frame.

2. The UAV flight control method for capturing black smoke from ships according to claim 1 is characterized in that: Based on the position of the sun and the orientation of the ship's chimney, setting the relative position of the drone accompanying the ship's flight includes setting the drone's orientation, the drone's altitude, and the distance between the drone and the ship's chimney.

3. The UAV flight control method for capturing black smoke from ships according to claim 2 is characterized in that: Based on the drone's position: The position of the sun is determined. When the sun is located behind the direction of the UAV facing the chimney of the ship, the UAV is set to a non-backlight position; when the sun is located in front of the ship, the UAV is set to the non-backlight position, and the angle between the UAV and the heading of the ship is greater than a first angle threshold; when the UAV is facing the chimney of the ship, the angle between the UAV and the ship's plume is greater than a second angle threshold.

4. The UAV flight control method for capturing black smoke from ships according to claim 3 is characterized in that: The method for judging whether the relative position of the UAV accompanying the ship is suitable is as follows: calculating the latitude difference between the UAV latitude and the ship latitude, calculating the longitude difference between the UAV longitude and the ship longitude, judging whether the ratio of the latitude difference to the longitude difference is within a preset orientation set, if yes, the relative position of the UAV accompanying the ship is suitable, if no, the relative position of the UAV accompanying the ship is not suitable.

5. The UAV flight control method for capturing black smoke from ships according to claim 4 is characterized in that: The orientation set is ( A s ±90° range)∩( A v ±30° range)∩( A e ±30° range), where: A s is the solar azimuth, A e is the horizontal azimuth of the ship plume.

6. The UAV flight control method for capturing black smoke from ships according to claim 2 is characterized in that: Setting the drone's altitude includes: The height of the UAV is set to be lower than the height of the chimney of the ship, and the elevation angle between the UAV and the chimney of the ship is less than a third angle threshold.

7. The UAV flight control method for capturing black smoke from ships according to claim 2 is characterized in that: The distance between the drone and the chimney of the ship is set to include: The horizontal distance between the UAV and the ship chimney is set to be no less than a first distance threshold, and the proportion of the ship chimney in the picture is set to be within a set threshold range.

8. The UAV flight control method for capturing black smoke from ships according to claim 1 is characterized in that: Setting the tracking frame includes: selecting a rectangular range covering the chimney of the ship in the drone image as the tracking frame.

9. The UAV flight control method for capturing black smoke from ships according to claim 1 is characterized in that: Continuously adjusting the flight speed of the drone according to the offset of the center position and the zoom ratio of the tracking frame includes: Based on the zoom ratio, a first threshold is set, and if the first threshold is exceeded, the flight speed of the drone is adjusted, otherwise, no adjustment is made; Based on the offset, a second threshold is set, and if the second threshold is exceeded, the flight direction angle of the UAV is adjusted, otherwise, no adjustment is made; If the flight speed and direction of the UAV are not adjusted within a set time range, it is confirmed that the flight speed and direction of the UAV are adjusted to be stable, and video recording is started; during the video recording, the flight speed of the UAV is continuously adjusted according to the zoom ratio and the offset until the video recording ends.

Images