A method and device for adjusting the waypoint position of a drone based on illumination angle

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, and discloses a method and device for adjusting the position of a unmanned aerial vehicle waypoint based on an illumination angle, comprising the following steps: determining an inspection time, inspection camera parameters and an initial planned track; calculating the solar azimuth of a shooting position according to the inspection time, and calculating the camera field of view angle of the shooting position according to the inspection camera parameters; judging whether the solar azimuth and the camera field of view overlap, if not, executing according to the initial planned track, if overlapping, adjusting the initial planned track; judging whether the shooting of a target at this location is affected by sunlight according to whether the solar azimuth and the camera field of view overlap, if affected by sunlight, the unmanned aerial vehicle automatically adjusts the hovering position, so that the imaging range is not affected by the illumination, and clear and refined inspection photos that meet the requirements are taken, thereby ensuring the inspection quality, avoiding rework, and improving the inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle inspection, and in particular to a method and device for adjusting a waypoint position of an unmanned aerial vehicle based on an illumination angle. Background Art

[0002] With the continuous improvement and development of drone technology, more and more industries are beginning to use drones to perform remote operations to achieve intelligent, fast and efficient work results. During the automatic track inspection process, drones need to cooperate with camera equipment to collect videos or photos of the inspection targets to help personnel quickly grasp the conditions of the targets and their surroundings. Since drone inspections are mostly carried out in outdoor environments, the photos taken during the inspection process are sometimes affected by the intensity of direct sunlight, resulting in blurred or unclear images, which do not meet the inspection requirements and need to be re-shot, resulting in low efficiency of drone inspections and high rework costs. Summary of the invention

[0003] In view of the deficiencies in the prior art, the object of the present invention is to provide a method and device for adjusting the waypoint position of a UAV based on the illumination angle.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A method for adjusting a waypoint position of a UAV based on illumination angle comprises the following steps:

[0006] Determine inspection time, inspection camera parameters and initial planning track;

[0007] Calculating the solar azimuth angle of the shooting position according to the inspection time, and calculating the camera field of view angle of the shooting position according to the inspection camera parameters;

[0008] Determine whether the solar azimuth angle and the camera field of view angle overlap. If there is no overlap, execute according to the initial planned track. If there is overlap, adjust the initial planned track.

[0009] In the present invention, preferably, adjusting the initial planned track includes the following steps:

[0010] Determine whether to adjust the pitch direction of the camera or the horizontal direction of the camera according to the minimum adjustment strategy;

[0011] Calculate the pitch direction or horizontal direction adjustment amount to determine the new shooting position, and obtain the adjusted new planned track;

[0012] Perform collision detection on the newly planned trajectory and execute it after ensuring the safety of the trajectory.

[0013] In the present invention, preferably, the step of calculating the solar azimuth angle of the shooting position according to the inspection time comprises the following steps:

[0014] Calculate the solar altitude angle based on the geographical latitude, solar declination and solar hour angle of the shooting location in the track;

[0015] Then the solar azimuth is calculated based on the solar altitude angle, geographical latitude and solar declination;

[0016] The solar direction vector sunV is calculated based on the solar altitude angle and the solar azimuth angle.

[0017] In the present invention, preferably, the camera field of view angle at the shooting position is calculated according to the inspection camera parameters, and specifically includes the following steps:

[0018] According to the camera parameters and the initial planned trajectory, the camera's orientation unit vector cameraV is calculated;

[0019] The camera field of view angle is calculated based on the camera imaging principle and the inspection camera parameters, where half of the vertical FOV angle is:

[0020]

[0021] Half the horizontal fov angle is:

[0022] Where L is the image plane sensor length, H is the sensor height, and f is the focal length;

[0023] Combined with the focal length, we can get the imaging range of the camera.

[0024] In the present invention, preferably, the step of determining whether the solar azimuth angle and the camera field of view angle overlap comprises the following steps:

[0025] The horizontal angle proj between the sun direction vector and the camera direction unit vector is calculated according to the graphics vector angle formula yaw and pitch angle proj pitch’

[0026] If proj yaw ≤fov H , and proj pitch ≤fov V , whether there is overlap between the solar azimuth angle and the camera field of view angle, and the photos taken by the inspection camera will be affected by lighting and exposure.

[0027] In the present invention, preferably, due to the influence of the actual environment on the shooting effect, the actual exposure critical value α is obtained through experimental calculation.

[0028] In the present invention, preferably, the step of determining to adjust the pitch direction of the camera or the horizontal direction of the camera according to the minimum adjustment strategy specifically includes the following steps:

[0029] Since it is necessary to re-check the collision of the front and rear related shooting positions to ensure the safety of the shooting position when adjusting the shooting position, the minimum adjustment principle is adopted here.

[0030] If fov H +α-proj yaw ≥fov V +α-proj pitch Then the pitch angle adjustment angle is smaller, and only the pitch direction needs to be adjusted.

[0031] If fov H +α-proj yaw <fov V +α-proj pitch Then the horizontal angle adjustment angle is smaller, and only the horizontal direction needs to be adjusted.

[0032] In the present invention, preferably, determining a new shooting position according to the pitch direction adjustment amount comprises the following steps:

[0033] e) Calculate the rotation matrix

[0034] Use the new pitch angle fov according to the graphics rotation matrix algorithm V +α-proj pitch Get the rotation matrix Matrix pitch

[0035] f) Calculate the new target point position

[0036] targetPos new =cameraPos+Matrix pitch *(targetPos-cameraPos)

[0037] g) Calculate the shooting target point offset

[0038] targetOffset=targetPos-targetPos new

[0039] h) Calculate the position where the camera shooting point needs to be adjusted

[0040] cameraPos new =cameraPos+targetOffset.

[0041] In the present invention, preferably, determining a new shooting position according to the horizontal adjustment amount specifically includes the following steps:

[0042] e) Calculate the rotation matrix

[0043] Use the new horizontal angle fov according to the graphics rotation matrix algorithm H +α-proj yaw Get the rotation matrix Matrix yaw

[0044] f) Calculate the new target point position

[0045] targetPos new =cameraPos+Matrix yaw *(targetPos-cameraPos)

[0046] g) Calculate the shooting target point offset

[0047] targetOffset=targetPos-targetPos new

[0048] h) Calculate the position where the camera shooting point needs to be adjusted

[0049] cameraPos new =cameraPos+targetOffset.

[0050] A device for adjusting a waypoint position of a drone based on an illumination angle comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any one of the methods of claims 1 to 9.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention combines the solar azimuth calculated by the time and longitude and latitude of the location where the photo was taken, the camera orientation when the photo was taken, and the reference value of the camera's imaging parameters to finally calculate the result, and judges whether the target photographed at this time and place is affected by sunlight. If affected by sunlight, the drone automatically adjusts the hovering position so that the imaging range is not affected by the light, and clear and detailed inspection photos that meet the requirements are taken, thereby ensuring the inspection quality, avoiding rework, and improving the inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention is a flowchart of a method for adjusting the waypoint position of a UAV based on illumination angle.

[0054] Figure 2 The present invention provides a flowchart for adjusting the initial planned track of a method for adjusting the waypoint position of a UAV based on illumination angle.

[0055] Figure 3The present invention is a schematic diagram of the solar azimuth angle calculation process of a method for adjusting the waypoint position of a UAV based on illumination angle.

[0056] Figure 4 Schematic diagram of camera imaging principle.

[0057] Figure 5 It is a graphic vector diagram. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] It should be noted that the terms “vertical”, “horizontal”, “left”, “right” and similar expressions used in this document are for illustrative purposes only.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0061] Please also see Figures 1 to 3 A preferred embodiment of the present invention provides a method for adjusting the position of a UAV waypoint based on the illumination angle. The method combines the latitude and longitude of the planned shooting position of the UAV inspection track at a specified time to calculate the solar azimuth and the field of view angle of the inspection camera and the camera imaging parameters to finally calculate the reference value, so as to judge whether the inspection camera will be affected by sunlight during shooting. Under the premise of being affected by the illumination, the hovering position of the UAV and the shooting angle of the inspection camera are automatically adjusted to avoid exposure of the inspection camera when taking photos, thereby reducing the inefficiency and rework rate of the photos taken by the UAV inspection and saving the inspection cost. The method mainly includes the following steps:

[0062] Determine inspection time, inspection camera parameters and initial planning track;

[0063] Calculating the solar azimuth angle of the shooting position according to the inspection time, and calculating the camera field of view angle of the shooting position according to the inspection camera parameters;

[0064] Determine whether the solar azimuth angle and the camera field of view angle overlap. If there is no overlap, execute according to the initial planned track. If there is overlap, adjust the initial planned track.

[0065] Specifically, the method of calculating the solar azimuth angle of the shooting position according to the inspection time includes the following steps:

[0066] S01, calculate the solar altitude angle based on the geographical latitude, solar declination and solar hour angle of the location in the track. The calculation formula is as follows:

[0067] sin H s =sinφ×sinδ+cosφ×cosδ×cos t

[0068] In the formula, H s represents the solar altitude angle, φ represents the geographical latitude of the current shooting location, δ represents the solar declination, and t represents the solar hour angle. The solar declination δ is calculated according to the following formula:

[0069] δ(deg)=0.006918-0.399912cos(b)+0.070257sin(b)-0.006758cos(2b)+0.000907sin(2b)-0.002697cos(3b)+0.00148sin(3b)

[0070] In the formula, b = 2 × PI × (N-1) / 365, N is the number of days from January 1st of each year to the calculation day, that is, January 1st, N = 1, January 2nd, N = 2, and so on, PI represents pi, and deg represents the angle degree;

[0071] The solar hour angle t is calculated according to the following formula:

[0072] t=15×(ST-12)

[0073] Wherein, ST is true solar time, which is based on 24 hours. It should be noted that in China, Beijing time is often used, not local time (true solar time). China is vast, and the maximum time difference between east and west can reach 4 hours. When conducting sunshine analysis, local time should be used. Therefore, in China, the conversion formula for true solar time is:

[0074] True solar time (ST) = Beijing time + time difference

[0075] Time difference = (local longitude - 120°) / 15°

[0076] That is, t = 15 × [Beijing time + ((local longitude - 120°) / 15°) - 12];

[0077] S02, then calculate the solar azimuth angle based on the solar altitude angle, geographic latitude and solar declination,

[0078] cos A s=(sin H s ×sinφ-sinδ)÷(cos H s ×cosφ)

[0079] In the formula, A s represents the solar azimuth, H s represents the solar altitude angle, φ represents the geographical latitude, and δ represents the solar declination;

[0080] S03, and calculate the sun direction vector sunV according to the sun altitude angle and the sun azimuth angle. Specifically, the existing standard graphics method is used to rotate the rotation matrix SunMatrix according to the sun altitude angle and the sun azimuth angle. The initial unit vector is (0, 1, 0) facing due north. The calculation formula of the unit vector of the camera position looking at the sun is:

[0081] sunV=SunMatrix*(0, 1, 0).

[0082] Specifically, the camera field of view angle at the shooting position is calculated based on the inspection camera parameters, and specifically includes the following steps:

[0083] S11, based on the camera parameters and the initial planned trajectory, calculate the camera's orientation unit vector cameraV: The rotation matrix uses the existing standard graphics method and is defined as CameraMatrix, based on the known trajectory format data parameters:

[0084] aircraftLatitude: shooting longitude

[0085] aircraftLongitude: shooting latitude

[0086] baseStationAltitude: shooting height

[0087] airCraftYawValue: The shooting horizontal angle, the angle with the true north, positive is clockwise, negative is counterclockwise

[0088] gimbalPitchValue: shooting pitch angle, positive means looking up, negative means looking down

[0089] The shooting pitch angle (gimbalPitchValue) and the shooting horizontal angle (airCraftYawValue) are rotated in sequence to obtain the rotation matrix CameraMatrix.

[0090] The initial unit vector is (0, 1, 0) pointing toward true north.

[0091] The camera's orientation unit vector is calculated as:

[0092] cameraV=CameraMatrix*(0,1,0);

[0093] S12, according to the camera imaging principle and the inspection camera parameters, the camera field of view angle is calculated, such as Figure 4 As shown, half of the vertical FOV (field of view) angle is:

[0094]

[0095] Half the horizontal fov angle is:

[0096] Where L is the image plane sensor length, H is the sensor height, and f is the focal length;

[0097] S13, the camera field of view angle, combined with the focal length, obtains the camera's imaging range.

[0098] Specifically, the determination of whether the solar azimuth angle and the camera field of view angle overlap includes the following steps:

[0099] Will be like Figure 5 As shown, the sun is regarded as a beam of parallel light. The cube in the figure represents the sun. The horizontal angle proj between the sun direction vector sunV and the camera direction unit vector cameraV is calculated according to the vector angle formula in graphics. yaw and pitch angle proj pitch ,

[0100] If proj yaw ≤fov H , and proj pitch ≤fov V , the solar azimuth and the camera field of view overlap, and the photos taken by the inspection camera will be affected by the lighting. yaw ≤fov H , then the field of view of the horizontal range photo exposure is calculated by the formula sunV = SunMatrix*(0, 1, 0). pitch ≤fov V , then the field of view angle of the exposure of the pitch range photo is calculated by the formula cameraV=CameraMatrix*(0,1,0).

[0101] Furthermore, the theoretical calculation structure can ensure that the shooting effect is not affected. However, due to the complexity of the actual environment, the sun may not be within the shooting range, but the picture will be affected to varying degrees. Therefore, after multiple tests and calculations, the actual exposure critical value α is obtained. α is used as an adjustment parameter to determine the final camera field of view angle range.

[0102] In this embodiment, the initial planned track, that is, the shooting position and angle of the drone, is adjusted to ensure that the shooting results of the track are not exposed during the inspection period, which specifically includes the following steps:

[0103] S21, determining to adjust the pitch direction of the camera or adjust the horizontal direction of the camera according to the minimum adjustment strategy;

[0104] Based on known parameters:

[0105] Shooting position cameraPos

[0106] Shooting position targetPos (corresponding to the shooting position)

[0107] Half the vertical fov angle V

[0108] Half the horizontal fov angle H

[0109] The horizontal angle between the camera direction and the sun's position proj yaw

[0110] The pitch angle between the camera and the sun's direction pitch

[0111] Actual exposure threshold deviation α

[0112] Get: If proj yaw ≤fov H +α and proj pitch ≤fov V +α will affect the photo taking effect.

[0113] Since it is necessary to re-check the collision of the front and rear related shooting positions to ensure the safety of the shooting position when adjusting the shooting position, the minimum adjustment principle is adopted here.

[0114] If fov H +α-proj yaw ≥fov V +α-proj pitch Then the pitch angle adjustment angle is smaller, and only the pitch direction needs to be adjusted.

[0115] If fov H +α-proj yaw <fov V +α-proj pitch Then the horizontal angle adjustment angle is small, and only the horizontal direction needs to be adjusted;

[0116] S22, determining a new shooting position according to the pitch direction or horizontal direction adjustment amount, and obtaining a new planned track after adjustment:

[0117] The pitch angle or horizontal angle that needs to be adjusted is used to calculate the new shooting point position. Since the shooting position cannot be modified, the offset of the new shooting position relative to the fixed position is calculated. At this time, the shooting camera position plus this offset can be used to calculate the final shooting camera position to be adjusted;

[0118] 1. Adjust the pitch angle:

[0119] a) Calculate the rotation matrix

[0120] Use the new pitch angle fov according to the graphics rotation matrix algorithm V +α-proj pitch Get the rotation matrix Matrix pitch

[0121] b) Calculate the new target point position

[0122] targetPos new =cameraPos+Matrix pitch *(targetPos-cameraPos)

[0123] c) Calculate the shooting target point offset

[0124] targetOffset=targetPos-targetPos new

[0125] d) Calculate the position to which the camera shooting point needs to be adjusted

[0126] cameraPos new =cameraPos+targetOffset.

[0127] 2. Adjust the horizontal angle:

[0128] i) Calculate the rotation matrix

[0129] Use the new horizontal angle fov according to the graphics rotation matrix algorithm H +α-proj yaw Get the rotation matrix Matrix yaw

[0130] j) Calculate the new target point position

[0131] targetPos new =cameraPos+Matrix yaw *(targetPos-cameraPos)

[0132] k) Calculate the shooting target point offset

[0133] targetOffset=targetPos-targetPos new

[0134] 1) Calculate the position where the camera shooting point needs to be adjusted

[0135] cameraPos new =cameraPos+targetOffset.

[0136] S23, performing collision detection on the newly planned track to ensure the safety of the track before executing.

[0137] Another preferred embodiment of the present invention provides a device for adjusting the position of a UAV waypoint based on the illumination angle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for adjusting the position of a UAV waypoint based on the illumination angle.

[0138] It should be pointed out that, for those skilled in the art, several improvements and modifications may be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for adjusting the waypoint position of a drone based on illumination angle, characterized in that: The following steps are involved: Determine inspection time, inspection camera parameters and initial planning track; Calculating the solar azimuth angle of the shooting position according to the inspection time, and calculating the camera field of view angle of the shooting position according to the inspection camera parameters; Determine whether the sun azimuth and the camera field of view overlap. If there is no overlap, execute according to the initial planned track. When there is overlap, adjust the initial planned track: According to the minimum adjustment strategy, determine whether to adjust the pitch direction of the camera or the horizontal direction of the camera. Calculate the pitch or horizontal adjustment amount to determine the new shooting position, and obtain the adjusted new planned track. Conduct collision detection on the newly planned trajectory and execute it after ensuring the safety of the trajectory; The step of determining to adjust the pitch direction of the camera or the horizontal direction of the camera according to the minimum adjustment strategy specifically includes the following steps: Since it is necessary to re-check the collision of the front and rear related shooting positions to ensure the safety of the shooting position when adjusting the shooting position, the minimum adjustment principle is adopted here. When fov H +α-proj yaw ≥fov V +α-proj pitch When the pitch angle is adjusted, the pitch angle is small and only the pitch direction needs to be adjusted. When fov H +α-proj yaw <fov V +α-proj pitch When the horizontal angle is adjusted, the angle is small, and only the horizontal direction needs to be adjusted. The method of determining a new shooting position according to the pitch direction adjustment amount includes the following steps: a) Calculate the rotation matrix: Use the new pitch angle fov according to the graphics rotation matrix algorithm V +α-proj pitch Get the rotation matrix Matrix pitch b) Calculate the new target point position: targetPos new =cameraPos+Matrix pitch *(targetPos-cameraPos) c) Calculate the shooting target point offset: targetOffset=targetPos-targetPos new d) Calculate the position to which the camera shooting point needs to be adjusted: comeraPos new =cameraPos+targetOffset In the above formula, cameraPos is the camera shooting position, α is the actual exposure critical deviation value, and fov V is half of the vertical fov angle, fov H is half of the horizontal FOV angle, proj yaw is the horizontal angle between the sun direction vector and the camera's unit vector, proj pitch is the pitch angle between the sun direction vector and the camera's unit vector; The new shooting position is determined according to the horizontal adjustment amount, which specifically includes the following steps: a) Calculate the rotation matrix: Use the new horizontal angle fov according to the graphics rotation matrix algorithm H +α-proj yaw Get the rotation matrix Matrix yaw b) Calculate the new target point position: targetPos new =cameraPos+Matrix yaw *(targetPos-cameraPos) c) Calculate the shooting target point offset: targetOffset=targetPos-targetPos new d) Calculate the position to which the camera shooting point needs to be adjusted: cameraPos new =cameraPos+targetOffset。 2. The method for adjusting the waypoint position of a UAV based on illumination angle according to claim 1, characterized in that: Calculating the solar azimuth of the shooting position according to the inspection time includes the following steps: Calculate the solar altitude angle based on the geographical latitude, solar declination and solar hour angle of the shooting location in the track; Then the solar azimuth is calculated based on the solar altitude angle, geographical latitude and solar declination; The solar direction vector sunV is calculated based on the solar altitude angle and the solar azimuth angle.

3. The method for adjusting the waypoint position of a UAV based on illumination angle according to claim 2, characterized in that: The method of calculating the camera field of view angle at the shooting position according to the inspection camera parameters specifically includes the following steps: According to the camera parameters and the initial planned trajectory, the camera's orientation unit vector cameraV is calculated; The camera field of view angle is calculated based on the camera imaging principle and the inspection camera parameters, where half of the vertical FOV angle is: Half the horizontal fov angle is: Where L is the image plane sensor length, H is the sensor height, and f is the focal length; Combined with the focal length, we can get the imaging range of the camera.

4. The method for adjusting the waypoint position of a UAV based on illumination angle according to claim 3, characterized in that: The step of determining whether the solar azimuth angle and the camera field of view angle overlap includes the following steps: The horizontal angle proj between the sun direction vector and the camera direction unit vector is calculated according to the graphics vector angle formula yaw and pitch angle proj pitch , When project yaw ≤fov H , and proj pitch ≤fov V When the sun's azimuth angle overlaps with the camera's field of view, the photos taken by the inspection camera will be affected by the lighting and exposure.

5. The method for adjusting the waypoint position of a UAV based on illumination angle according to claim 4, characterized in that: Due to the influence of the actual environment on the shooting effect, the actual exposure critical value α is obtained through experimental calculation.

6. A device for adjusting the waypoint position of a drone based on illumination angle, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 5.

Images