Gimbal control method, controller, unmanned aerial vehicle and unmanned aerial vehicle inspection system

Abstract

The application relates to the field of unmanned aerial vehicles, and discloses a gimbal control method, a controller, an unmanned aerial vehicle and an unmanned aerial vehicle inspection system. The gimbal control method is applied to an unmanned aerial vehicle, and the inspection information of the unmanned aerial vehicle is acquired. The inspection information comprises an observation flight section, an observation interval corresponding to the observation flight section, a total flight distance corresponding to the observation interval and a current flight distance. Then, the observation progress of the unmanned aerial vehicle is determined according to the current flight distance and the total flight distance. A first position is determined according to the observation progress and the observation interval, and the first position is the position of the center point of the field of view of the nacelle of the unmanned aerial vehicle. Finally, the angle of the gimbal of the unmanned aerial vehicle is controlled according to the first position and the current position of the unmanned aerial vehicle. In the above manner, the current shooting field of view can be adapted to the observation interval. When the inspection is completed, the entire shooting field of view can completely cover the inspection object, the inspection object is adaptively observed, the inspection object is completely observed, the risk of missing inspection is effectively avoided, and the inspection efficiency is improved.

Description

Gimbal control methods, controllers, drones and drone inspection systems Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a gimbal control method, controller, UAV, and UAV inspection system. Background Technology

[0002] A drone is an unmanned aircraft whose flight attitude is controlled by radio remote control equipment and built-in programs. It is finding increasingly widespread application in various fields. Drones are increasingly being used for inspection operations, such as inspecting power transmission equipment, pipelines, and vegetation. Workers can use drones to complete inspections of the objects being inspected.

[0003] Generally, drones used for inspection operations are equipped with gimbals, which are supporting devices for mounting camera equipment. However, during drone inspection operations, the gimbal's angle is fixed, meaning it's always pointing in a specific direction towards the drone, resulting in a fixed observation direction that cannot be automatically adjusted. This poses a risk that the field of view may not fully cover the inspected object, leading to missed inspections. Furthermore, manually setting the gimbal angle during inspection is not only labor-intensive but also reduces inspection efficiency. Summary of the Invention

[0004] The embodiments of the present invention at least partially solve one of the above-mentioned technical problems. To this end, the present invention provides a gimbal control method, a controller, a drone, and a drone inspection system, which can automatically adjust the angle of the gimbal to adapt to the inspection object, so that the shooting field of view can completely cover the inspection object, effectively avoid the risk of missed inspection, and improve inspection efficiency.

[0005] In a first aspect, embodiments of the present invention provide a gimbal control method applied to a drone, comprising:

[0006] Obtain inspection information from the UAV, which includes the observation segment, the observation area corresponding to the observation segment, the total flight distance corresponding to the observation area, and the current flight distance. The observation segment is the flight segment used by the UAV to inspect the observation area.

[0007] The observation progress of the UAV is determined based on the current flight range and the total flight range;

[0008] The first position is determined based on the observation progress and the observation range. The first position is the target position of the center point of the UAV's pod field of view.

[0009] The angle of the drone's gimbal is controlled based on the first position and the drone's current position.

[0010] In some embodiments, determining the observation progress of the UAV based on the current flight range and the total flight range includes:

[0011] The flight schedule of the drone is determined based on the current flight distance and the total flight distance.

[0012] The observation progress of the drone is determined based on the flight progress.

[0013] In some embodiments, determining the flight progress of the UAV based on the current flight distance and the total flight distance includes:

[0014] The first ratio of the current flight distance to the total flight distance is determined as the flight progress of the UAV.

[0015] In some embodiments, determining the observation progress of the UAV based on the flight progress includes:

[0016] The second ratio of the flight progress to a preset coefficient is determined as the observation progress of the UAV, where the preset coefficient is less than or equal to 1.

[0017] In some embodiments, determining the first position based on the observation progress and the observation interval includes:

[0018] Obtain the starting and ending coordinates of the observation interval;

[0019] The first unit vector pointing from the start point to the end point of the observation interval is determined based on the start point coordinates and the end point coordinates.

[0020] The coordinates of the first position are calculated using the following formula:

[0021] P = s + vec × present

[0022] Where P is the coordinate of the first position, s is the coordinate of the starting point, vec is the first unit vector, and represents the observation progress.

[0023] In some embodiments, controlling the gimbal angle of the drone based on the first position and the current position of the drone includes:

[0024] Determine the second unit vector pointing from the drone's current position to the first position based on the coordinates of the first position and the current position of the drone;

[0025] The pitch and yaw angles of the gimbal are determined based on the second unit vector.

[0026] The angle of the gimbal is controlled by the pitch and yaw angles.

[0027] In some embodiments, before obtaining the inspection information of the drone, the method further includes:

[0028] Obtain the mapping relationship between the observed flight segment and the observed interval;

[0029] Based on the mapping relationship and the observed flight segment, determine the observation interval corresponding to the observed flight segment;

[0030] Based on the mapping relationship and the observation interval, determine all observation segments associated with the observation interval;

[0031] The sum of the flight distances of all observation segments is determined as the total flight distance corresponding to the observation interval.

[0032] In some embodiments, the method further includes:

[0033] Obtain the flight speed of the drone during the observation segment;

[0034] The current flight distance corresponding to the observation interval is calculated using the following formula:

[0035]

[0036] Where S is the current flight distance corresponding to the observation range, t is the time from when the UAV starts observing the observation range to the current position of the UAV, and V is the flight speed of the UAV.

[0037] In some embodiments, when the UAV observes a new observation range, the UAV's flight progress, current flight distance, and observation progress are all recalculated from zero.

[0038] In some embodiments, when the drone is performing a temporary task, the method further includes:

[0039] If the drone is performing a temporary mission along the current flight path, the process of determining the drone's flight progress based on the current flight distance and the total flight distance will continue until the temporary mission ends.

[0040] If the drone departs from its current flight path to perform a temporary mission, the drone's current flight progress will be locked until the temporary mission ends.

[0041] In a second aspect, embodiments of the present invention provide a controller, comprising:

[0042] At least one processor; and,

[0043] A memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the gimbal control method as described above.

[0045] Thirdly, embodiments of the present invention provide a drone, including: a gimbal and a controller as described in the second aspect, the controller being used to control the angle of the gimbal.

[0046] Fourthly, embodiments of the present invention provide an unmanned aerial vehicle (UAV) inspection system, comprising:

[0047] Several ground monitoring stations and drones as mentioned in the third aspect above;

[0048] The ground monitoring station communicates with the drone. The drone is used to inspect the observation area along the inspection route and transmits the image information of the inspection route and inspection information obtained during the inspection to the ground monitoring station so that the ground monitoring station can control the drone.

[0049] Compared with existing technologies, this invention has at least the following advantages: The gimbal control method of this invention is applied to a drone. It acquires the drone's inspection information, which includes the observation segment, the corresponding observation interval, the total flight distance of the observation interval, and the current flight distance. Then, the drone's observation progress is determined based on the current flight distance and the total flight distance. A first position is determined based on the observation progress and the observation interval; the first position is the location of the center point of the drone's pod's field of view. Finally, the angle of the drone's gimbal is controlled based on the first position and the drone's current position. In this embodiment, the inspection object is divided into multiple observation intervals using the above method. For each observation interval, a first position (the target position of the pod's observation center) that can fall within the observation interval is calculated in real time based on the observation progress (flight progress). Then, the angle of the drone's gimbal is controlled based on the first position to automatically adjust the gimbal angle so that the adjusted position of the pod's observation center is located at the first position, making the current shooting field of view compatible with the observation interval. Therefore, once the inspection is completed, the entire field of view can completely cover the inspected object, adapt to the inspected object, and conduct a complete observation of the inspected object, effectively avoiding the risk of missed inspections and improving inspection efficiency. Attached Figure Description

[0050] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0051] Figure 1 is an architecture diagram of an unmanned aerial vehicle inspection system using a gimbal control method provided in some embodiments of this application;

[0052] Figure 2 is a flowchart illustrating the gimbal control method in some embodiments of this application;

[0053] Figure 3 is a schematic diagram of the inspection of the UAV in some embodiments of this application;

[0054] Figure 4 is a schematic diagram of the mapping between flight segments and observation intervals in some embodiments of this application;

[0055] Figure 5 is a schematic diagram of the current flight distance and observation progress in some embodiments of this application;

[0056] Figure 6 is a schematic diagram of the pod's field of view center in the observation range in some embodiments of this application;

[0057] Figure 7 is a schematic diagram of the target position of the pod's field of view center in some embodiments of this application;

[0058] Figure 8 is a schematic diagram of a drone performing a temporary task in some embodiments of this application;

[0059] Figure 9 is a schematic diagram of a drone performing a temporary task in some embodiments of this application;

[0060] Figure 10 is a structural block diagram of the gimbal control device in some embodiments of this application;

[0061] Figure 11 is a schematic diagram of the controller structure in some embodiments of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0063] It should be noted that, unless otherwise specified, the various features in the embodiments of this invention can be combined with each other, all of which are within the protection scope of this invention. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this invention do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0064] The gimbal control method in this embodiment is applied to a drone. The drone can be any type of unmanned aerial vehicle, such as a fixed-wing drone, tilt-rotor drone, rotary-wing drone, paraglider drone, flapping-wing drone, etc. Drones can provide various functional services, such as aerial photography, inspection, monitoring, or transportation. In some embodiments, the drone can perform inspection operations on objects (e.g., power transmission equipment, pipelines, or vegetation) to assist workers in completing the inspection. During drone operation, any type of processor on the drone can execute the gimbal control method provided in this embodiment to automatically adjust the gimbal angle to adapt to the inspection object, ensuring the shooting field of view completely covers the inspection object, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0065] The following example, taking inspection operations, illustrates the application environment of this pan-tilt control method. The inspection targets can be power transmission equipment, pipelines, roads, or vegetation, etc.

[0066] Please refer to Figure 1, which is an architecture diagram of a drone inspection system using a gimbal control method provided in this embodiment of the application. As shown in Figure 1, the drone inspection system 100 includes several ground monitoring stations 10 and drones 20. The ground monitoring stations 10 and drones 20 are connected via a wireless network. The wireless network can be a Bluetooth network, a WiFi network, a wireless cellular network, or a combination thereof, and is not limited thereto.

[0067] The ground monitoring station 10 can be an electronic device with computing or display functions, such as a computer, smartphone, remote control, or tablet computer. No restrictions are placed on the ground monitoring station; it only needs to have computing or display functions.

[0068] The drone 20 can be any type of powered unmanned aerial vehicle, including but not limited to tiltrotor drones, fixed-wing drones, paraglider drones, flapping-wing drones, and helicopter models. The drone 20 can be configured with appropriate size or power to meet specific needs, providing sufficient payload capacity, flight speed, and flight range. One or more functional modules can be added to the drone 20 to enable it to perform corresponding functions. In some embodiments, the drone 20 is equipped with a gimbal, which is a support device for mounting camera equipment. Specifically, the camera equipment or other sensors are housed in a pod and mounted on the gimbal. Those skilled in the art will understand that the gimbal can rotate around the X, Y, and Z axes, and when the gimbal adjusts its angle, it can adjust the angle of the pod to change the field of view of the camera equipment.

[0069] The drone 20 includes at least one controller, which serves as the control core for drone flight and data transmission, and integrates one or more modules to execute corresponding logic control programs, such as the gimbal control method provided in the embodiments of this application.

[0070] In some embodiments, the inspection object 30 may be a high-voltage transmission line, which may include multiple towers for erecting transmission cables, each tower corresponding to an observation feature point. When the drone is observing a feature point, it can photograph and inspect one or more components of the tower (e.g., one or more components such as insulators, anti-detachment pins, hanging bolts, and vibration dampers). When the drone is between two towers, it can photograph and inspect multiple transmission cables.

[0071] In this embodiment, the observation path can include the towers along the entire transmission line and the area where the transmission cable is located between two adjacent towers. The observation interval can be the area where the transmission cable is located between two towers.

[0072] The drone inspection system 100 can perform inspection operations on the object 30. Before automatically inspecting the object 30, the observation route can be determined. When inspecting the observation area along the observation route, the drone 20 transmits the image information about the observation route and inspection information acquired during the inspection to the ground monitoring station 10. The ground monitoring station 10 can then display the image information, helping staff to understand the inspection status of the object 30 in a timely manner. In addition, staff can also control the drone's operation through the ground monitoring station 10.

[0073] It is worth noting that the above example only uses high-voltage transmission lines as the inspection target 30 and does not impose any limitations on the inspection target. In some embodiments, the inspection target may also be roads, factories, farmland, or forests, etc.

[0074] Understandably, during UAV inspection operations, it is necessary to control the gimbal and execute the gimbal control method provided in this application embodiment to ensure that the gimbal angle meets the requirements for scanning and photographing the inspection target. That is, during the UAV's flight, the gimbal angle allows the field of view to scan and photograph each observation section along the observation path. After the inspection is completed, all fields of view can completely cover the inspection target, and the image information can reflect all areas of the inspection target.

[0075] Some gimbal control methods known to the inventors of this application typically set fixed gimbal angles for different waypoints or flight segments based on the spatial relationship between the observation path and the flight route, so that the gimbal is fixedly oriented in a certain direction towards the aircraft during the UAV's flight. That is, the observation direction is fixed and cannot be automatically adjusted.

[0076] In this approach, because the gimbal angle is fixed, it may not fully cover the inspection target when the observation route undulates or has sharp bends. Therefore, this approach requires setting a suitable gimbal angle for each waypoint or segment along the observation route, especially when the route has bends or significant altitude changes, as the pre-set gimbal angle may not accurately reflect the actual situation. Furthermore, when the observation route is too long or has too many segments, manually setting the gimbal observation angle for each segment of the route is very difficult, time-consuming, and prone to errors, thus impacting inspection efficiency.

[0077] To address the aforementioned issues, this application provides a gimbal control method applied to a drone. The method acquires the drone's inspection information, including the observation segment, the corresponding observation interval, the total flight distance of the observation interval, and the current flight distance. Then, the drone's observation progress is determined based on the current and total flight distances. A first position is determined based on the observation progress and the observation interval; this first position is the location of the center point of the drone's pod's field of view. Finally, the angle of the drone's gimbal is controlled based on this first position and the drone's current position. In this embodiment, the inspection target is divided into multiple observation intervals. For each observation interval, a first position (the target position of the pod's observation center) that can fall within the observation interval is calculated in real time based on the observation progress (flight progress). Then, the angle of the drone's gimbal is controlled based on the first position to automatically adjust the gimbal angle so that the adjusted position of the pod's observation center is located at the first position, thus adapting the current field of view to the observation interval. Therefore, once the inspection is completed, the entire field of view can completely cover the inspected object, adapt to the inspected object, and conduct a complete observation of the inspected object, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0078] As can be understood from the above, the gimbal control method provided in this application embodiment can be implemented by a drone including a gimbal, for example, by the drone's control chip or processor, or by other devices with computing power. Other devices with computing power may be ground monitoring stations or smart terminals that are communicatively connected to the drone.

[0079] The gimbal control method provided in this application embodiment will be described below with reference to exemplary applications and implementations of drones provided in the embodiments of this application. Please refer to Figure 2, which is a schematic flowchart of the gimbal control method provided in the embodiments of this application. It can be understood that the executing entity of this gimbal control method can be one or more processors of the drone.

[0080] As shown in Figure 2, method S100 may specifically include the following steps:

[0081] S10: Obtain the inspection information of the UAV, which includes the observation segment, the observation interval corresponding to the observation segment, the total flight distance corresponding to the observation interval, and the current flight distance.

[0082] The observation segment is the flight segment used by the UAV for inspection and observation of the designated area. The observation area refers to the region between two adjacent observation feature points along the observation path. The observation path refers to the scanning route of the field of view required for observing the inspected object during the inspection mission flight, and can include the observation feature points and the lines connecting them, forming a three-dimensional space. Observation feature points are the nodes that constitute the observation path during the inspection mission flight.

[0083] There is a correspondence between observation segments and observation intervals. In some embodiments, one observation interval corresponds to one observation segment; in other embodiments, one observation interval corresponds to multiple observation segments.

[0084] The total flight distance corresponding to the observation interval refers to the total mileage of at least one observation segment corresponding to the observation interval. When the observation area corresponds to one observation segment 1#, the total flight distance corresponding to the observation interval is the mileage of observation segment 1#; when the observation area corresponds to observation segment 1# and observation segment 2#, the total flight distance corresponding to the observation interval is the sum of the mileages of these two observation segments 1# and 2#.

[0085] The current flight distance corresponding to the observation interval refers to the total distance flown by the UAV on at least one observation segment within that observation interval. It is understandable that as the UAV flies, the current flight distance accumulates until the total flight distance is reached, at which point the inspection of the next observation interval begins.

[0086] Please refer to Figure 3 for an illustrative example using a high-voltage transmission line as the inspection target. A high-voltage transmission line may include multiple towers for erecting transmission cables, and each tower may correspond to an observation feature point. Multiple cables are connected between the towers.

[0087] The flight path lies above the high-voltage power transmission line. This means the flight path is the route taken by the UAV during its inspection mission. The flight path consists of various observation segments. In this embodiment, the observation route includes the towers along the entire power transmission line and the area containing the power cables between adjacent towers. In this embodiment, the observation interval can be the area containing the power cables between two towers. It is understood that the observation route can be closed or open; no limitation is made here.

[0088] Please refer to Figure 3 again. During the inspection flight, the UAV needs to observe the entire observation route and promptly detect any special situations along the eastern observation line. Therefore, as the UAV flies, the field of view (field of view) of the imaging equipment inside the pod needs to sweep across the entire observation route, not just certain areas or only the observation feature points. To achieve a complete observation of the entire route, the center point of the pod's field of view needs to move along the observation route. Here, the center point of the pod's field of view refers to the center point of the pod's field of view.

[0089] In some embodiments, prior to the aforementioned step S10, the method S100 further includes:

[0090] (1) Obtain the mapping relationship between the observation segment and the observation interval.

[0091] (2) Determine the observation interval corresponding to the observation segment based on the mapping relationship and the observation segment.

[0092] (3) Based on the mapping relationship and the observation interval, determine all observation segments associated with the observation interval.

[0093] (4) The sum of the flight distances of all observation segments is determined as the total flight distance corresponding to the observation interval.

[0094] Understandably, before automating the inspection of objects, the observation route and path can be determined first. This path can be automatically generated based on the observation route or manually set by staff. The path is divided into multiple observation segments. In some embodiments, staff can pre-set the mapping relationship between observation segments and observation intervals, storing it in the UAV's memory chip for the processor to access. This mapping relationship satisfies the following: each observation segment can be associated with at most one observation interval, and one observation interval can correspond to one observation segment.

[0095] Please refer to Figure 4. Figure 4(a) is a schematic diagram of the observation interval corresponding to the observation segment, and Figure 4(b) is a schematic diagram of the observation segment corresponding to the observation interval. As shown in Figure 4(a), the observation segments corresponding to observation interval 4-5 are segment 5, segment 6, and segment 7. Therefore, the observation interval corresponding to the observation segment can be determined based on the mapping relationship and the observation segment. As shown in Figure 4(b), observation segment 1 corresponds to observation interval 1, observation segments 2 and 3 correspond to observation interval 2, observation segment 4 corresponds to observation interval 3, and observation segments 5, 6, and 7 all correspond to observation interval 4.

[0096] Since an observation interval may correspond to one or more observation segments, all observation segments associated with the observation interval can be determined based on the mapping relationship and the observation interval. For example, all observation segments associated with observation interval 4 include observation segment 5, observation segment 6, and observation segment 7.

[0097] Finally, the sum of the flight distances of all observation segments is determined as the total flight distance corresponding to the observation interval. For example, the total flight distance corresponding to observation interval 4 is the sum of the flight distances of observation segment 5, observation segment 6, and observation segment 7.

[0098] In this embodiment, by setting the above mapping relationship, after knowing the current observation segment of the UAV, the current observation interval can be quickly found. After knowing the current observation interval of the UAV, all observation segments associated with the observation interval can be quickly found, and the total flight distance corresponding to the observation interval can be calculated.

[0099] S20: Determine the observation progress of the UAV based on the current flight range and the total flight range.

[0100] As shown above, each observation interval corresponds to at least one observation segment. Since the total flight distance refers to the total mileage of at least one observation segment corresponding to the observation interval, and the current flight distance refers to the total mileage flown by the UAV on at least one observation segment corresponding to that observation interval, there is a correlation between flight distance and observation progress. Therefore, the UAV's observation progress can be determined based on the current flight distance and the total flight distance. It can be understood that observation progress refers to the degree to which the UAV has completed its inspection of the current observation interval. For example, when the current flight distance is half of the total flight distance, the area covered by the shooting field of view within that observation interval will be greater than 50%, thus the observation progress could be 60%.

[0101] In some embodiments, the aforementioned step S20 specifically includes:

[0102] S21: Determine the flight schedule of the UAV based on the current flight range and the total flight range.

[0103] Flight progress is an indicator that evaluates how much a UAV has flown in at least one flight segment corresponding to the current observation range. Based on the definitions of current flight distance and total flight distance, the UAV's flight progress can be determined according to these parameters.

[0104] In some embodiments, the aforementioned step S21 specifically includes: determining the first ratio of the current flight distance to the total flight distance as the flight progress of the UAV.

[0105] In this embodiment, the flight progress of the UAV within a certain observation range is the ratio of the current flight distance of the UAV within that observation range to the total flight distance corresponding to that observation range. The current flight distance can be calculated by integrating the flight speed over time.

[0106] In some embodiments, the method S100 further includes:

[0107] (1) Obtain the flight speed of the UAV in the observation segment;

[0108] (2) Calculate the current flight distance corresponding to the observation interval using the following formula:

[0109]

[0110] Where S is the current flight distance corresponding to the observation range, t is the time from when the UAV starts observing the observation range to the current position of the UAV, and V is the flight speed of the UAV.

[0111] For example, please refer to Figure 5. In Figure 5, the observation interval M corresponds to observation segments A, B, and C. When the UAV is at the starting point of observation segment A, it begins to integrate the flight speed over time to calculate the current flight distance S. Here, the flight speed is the real-time acquired flight speed of the UAV.

[0112] In this embodiment, by integrating the flight speed of the UAV in real time during the observation segment, the current flight distance can be accurately obtained.

[0113] Understandably, in some embodiments, when the UAV observes a new observation range, the UAV's flight progress, current flight distance, and observation progress are all recalculated from zero. That is, starting from the starting coordinates of the new observation range, the current flight distance and observation progress corresponding to the new observation range are recalculated to ensure the accuracy of the data corresponding to each observation range, facilitating subsequent accurate adjustment of the gimbal angle.

[0114] S22: Determine the observation progress of the UAV based on the flight progress.

[0115] Referring again to Figure 5, it can be understood that there is a positive correlation between flight progress and observation progress. Therefore, based on the correlation between flight progress and observation progress, the observation progress of the UAV can be determined according to the flight progress. In some embodiments, flight progress can be roughly used as observation progress.

[0116] In some embodiments, the aforementioned step S22 specifically includes: determining the second ratio of the flight progress to a preset coefficient as the observation progress of the UAV, wherein the preset coefficient is less than or equal to 1.

[0117] As can be understood, as shown in Figure 6, the field of view of the shooting device is directed downwards from the drone, and its shape is a rectangle larger than the drone. When the drone flies to the current flight path, the area swept by the shooting field of view will be larger than the current flight path. For example, when the flight progress reaches 50%, the observation progress may reach 60%.

[0118] In this embodiment, a preset coefficient is set as the ratio between flight progress and observation progress. Considering the characteristics of the relationship between flight progress and observation progress, the ratio between observation progress and flight progress is appropriately relaxed, and the preset coefficient is set to be less than or equal to 1. For example, if 100% observation progress only corresponds to 90% flight progress, the preset coefficient can be set to 0.9. When the UAV reaches 90% of its current flight distance on the observation segment corresponding to observation interval M, that is, when the flight progress reaches 90%, the calculated observation progress has reached 1. At this time, the field of view has been completely swept across the observation interval M.

[0119] In this embodiment, based on the correlation between flight progress and observation progress, the observation progress of the UAV can be accurately determined according to the flight progress. By appropriately relaxing the ratio between observation progress and flight progress, and setting the preset coefficient to be less than or equal to 1, the second ratio of flight progress to the preset coefficient is determined as the observation progress of the UAV. This not only ensures accurate observation progress but also guarantees that every place in the observation range can be observed and photographed.

[0120] S30: Determine the first position based on the observation progress and the observation range. This first position is the target position of the center point of the UAV's pod field of view.

[0121] Understandably, the field of view of the shooting device is directed downwards towards the drone, and its shape is a rectangle larger than the drone. In this step, the center point of the pod's field of view is the center of the rectangular field of view.

[0122] Please refer to Figure 6. To ensure that the field of view can completely cover the width of the observation path in real time, the center point of the pod's field of view can be moved along the line connecting the starting and ending coordinates of each observation interval. It can be understood that the width of the observation path refers to its lateral width. In the example of inspecting high-voltage transmission lines mentioned above, the width of the observation path is the distance between the two transmission cables with the greatest lateral distance.

[0123] Therefore, the target position (first position) of the UAV's pod field of view center point is determined in real time based on the observation progress and the observation interval. If there is a deviation between the current position of the pod field of view center point and the target position, the current position is adjusted to the target position in time, so that the pod field of view center point can move on the line connecting the starting coordinates and the ending coordinates of the observation interval.

[0124] In some embodiments, the aforementioned step S30 specifically includes:

[0125] S31: Obtain the starting and ending coordinates of the observation interval.

[0126] S32: Determine the first unit vector pointing from the start point to the end point of the observation interval based on the start point coordinates and the end point coordinates.

[0127] S33: Calculate the coordinates of the first position using the following formula:

[0128] P = s + vec × present

[0129] Where P is the coordinate of the first position, s is the starting point coordinate, vec is the first unit vector, and present is the observation progress.

[0130] Please refer to Figure 7. The starting coordinate of the observation interval is s = (s x ,s y ,s z The endpoint coordinates are e = (e x ,e y ,e z The current observation progress of the drone is present. In some embodiments, in order to calculate the first unit vector, the distance between the starting point coordinates and the ending point coordinates is first calculated. Then, calculate the first unit vector.

[0131] After obtaining the first unit vector, and given the starting point coordinates s, as the observation progresses, the coordinates of the first position, i.e., the target position of the gondola's field of view center point, can be accurately calculated using the above formula.

[0132] In this embodiment, the coordinates of the first position can be accurately determined by the above method, that is, the target position of the center point of the pod's field of view can be accurately determined.

[0133] S40: Control the angle of the drone's gimbal based on the first position and the drone's current position.

[0134] The "current position of the UAV" refers to the UAV's coordinates in three-dimensional space at the current moment. Knowing the UAV's current position and the target position at the center of the pod's field of view, gimbal control commands can be calculated to control the gimbal's angle.

[0135] In some embodiments, the aforementioned step S40 specifically includes:

[0136] S41: Determine the second unit vector pointing from the drone's current position to the first position based on the coordinates of the first position and the coordinates of the drone's current position.

[0137] S42: Determine the pitch and yaw angles of the gimbal based on the second unit vector.

[0138] S43: Control the angle of the gimbal based on the pitch and yaw angles.

[0139] For example, if the current coordinates of the drone are P = (u x ,u y ,u z The coordinates of the first position are G = (g x ,g y ,g z First, the vector from the drone's current position to the first position is calculated using the following formula.

[0140]

[0141] Then, the second unit vector pointing from the drone's current position to the first position is calculated.

[0142] The following formula is used to calculate the pitch angle θ of the gimbal. Gim With yaw angle ψ Gim :

[0143] θ Gin =atan2(e2,e1)

[0144] ψ Gim =asin(e3)

[0145] Finally, the angle of the gimbal is controlled according to the pitch and yaw angles so that the center point of the pod's field of view is adjusted from the current position to the target position. This ensures that the center point of the pod's field of view can move along the line connecting the starting and ending coordinates of the observation range.

[0146] In this embodiment, the pitch and yaw angles of the gimbal are determined by the second unit vector pointing from the current position of the UAV to the first position. Then, the angle of the gimbal is controlled according to the pitch and yaw angles, so that the center point of the pod's field of view can be adjusted from the current position to the target position. This ensures that the center point of the pod's field of view can move on the line connecting the starting coordinates and the ending coordinates of the observation range, so as to fully cover the width of the observation path in real time.

[0147] In some embodiments, when the drone is performing a temporary task, the method further includes:

[0148] (1) If the UAV is performing a temporary mission along the current flight path, the aforementioned steps of determining the flight progress of the UAV based on the current flight distance and the total flight distance shall continue until the temporary mission ends.

[0149] (2) If the UAV flies away from the current flight path to perform a temporary mission, the flight progress of the current UAV is locked until the temporary mission ends.

[0150] Under normal circumstances, the UAV performs routine inspection tasks while flying along the flight path. The center point of the pod's field of view moves along the observation path, enabling automatic observation of the observation path.

[0151] Understandably, during actual flight, the drone may need to perform other temporary tasks if it detects special conditions on the route, such as locking the gimbal to observe a special area on the route, or temporarily leaving the route to perform other tasks.

[0152] To accommodate these emergency operations, flight progress is handled in two ways: one is that the drone does not leave its original flight path while performing a temporary mission, and the other is that the drone needs to leave its original flight path.

[0153] Specifically, please refer to Figures 8(a) to (c). If the UAV is performing a temporary task along its current flight path, for example, the gimbal may be controlled by other commands during flight and will not continue to automatically observe the observation path. However, since the UAV is still flying on the flight path, the UAV's flight progress will not stop being calculated. That is, the aforementioned steps of determining the UAV's flight progress based on the current flight distance and the total flight distance will continue until the temporary task ends. Thus, it can be effectively ensured that the UAV can still normally observe its corresponding observation range when it returns to the flight path for observation.

[0154] Please refer to (a) to (c) in Figure 9. If the UAV flies away from the current flight path to perform a temporary task, the current flight progress of the UAV is locked. When returning to the original flight path from other tasks, the calculation continues from the locked flight progress. Thus, it can be ensured that when the UAV returns to the flight path from outside the flight path, it can continue to observe from the position it flew away from.

[0155] In this embodiment, the two situations described above are handled when the UAV performs temporary tasks, so that the UAV can still smoothly return to the original route to continue performing the observation and inspection tasks after handling special situations.

[0156] In summary, this application provides a gimbal control method applied to a drone. It acquires the drone's inspection information, including the observation segment, the corresponding observation interval, the total flight distance of the observation interval, and the current flight distance. Then, the drone's observation progress is determined based on the current flight distance and the total flight distance. A first position is determined based on the observation progress and the observation interval; this first position is the location of the center point of the drone's pod's field of view. Finally, the angle of the drone's gimbal is controlled based on this first position and the drone's current position. In this embodiment, the inspection object is divided into multiple observation intervals using the above method. For each observation interval, a first position (the target position of the pod's observation center) that can fall within the observation interval is calculated in real time based on the observation progress (flight progress). Then, the angle of the drone's gimbal is controlled based on the first position to automatically adjust the gimbal angle so that the adjusted position of the pod's observation center is located at the first position, making the current shooting field of view compatible with the observation interval. Therefore, once the inspection is completed, the entire field of view can completely cover the inspected object, adapt to the inspected object, and conduct a complete observation of the inspected object, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0157] Please refer to Figure 10, which is a schematic diagram of a gimbal control device provided in an embodiment of this application. This gimbal control device is applied to a drone; specifically, it is applied to one or more processors of the drone.

[0158] As shown in Figure 10, the gimbal control device 200 includes: an acquisition module 201, an observation progress determination module 202, a first position determination module 203, and a control module 204.

[0159] The acquisition module 201 acquires the UAV's inspection information, including the observation segment, the corresponding observation interval, the total flight distance corresponding to the observation interval, and the current flight distance. The observation progress determination module 202 determines the UAV's observation progress based on the current flight distance and the total flight distance. The first position determination module 203 determines the first position based on the observation progress and the observation interval; this first position is the target position of the center point of the UAV's pod's field of view. The control module 204 controls the angle of the UAV's gimbal based on the first position and the UAV's current position.

[0160] In the embodiments of this application, the gimbal control device can also be constructed from hardware components. For example, the gimbal control device can be constructed from one or more chips, and the chips can work in coordination to complete the gimbal control method described in the above embodiments. Furthermore, the gimbal control device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0161] The gimbal control device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0162] The gimbal control device provided in this application embodiment can realize all the processes that the above gimbal control method can achieve. To avoid repetition, it will not be described again here.

[0163] It should be noted that the above-mentioned gimbal control device can execute the gimbal control method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the gimbal control device can be found in the gimbal control method provided in the embodiments of this application.

[0164] This application also provides a controller. Please refer to Figure 11, which is a schematic diagram of the hardware structure of a controller provided in this application.

[0165] As shown in Figure 11, the controller 300 includes at least one processor 301 and a memory 302 connected in communication (Figure 11 shows a bus connection with one processor as an example).

[0166] The processor 301 provides computing and control capabilities to control the controller 300 to perform corresponding tasks. For example, the controller 300 executes the gimbal control method in any of the above method embodiments. This method includes: acquiring inspection information of the UAV, which includes an observation segment, an observation interval corresponding to the observation segment, a total flight distance corresponding to the observation interval, and the current flight distance; determining the observation progress of the UAV based on the current flight distance and the total flight distance; determining a first position based on the observation progress and the observation interval, which is the target position of the center point of the UAV's pod field of view; and controlling the angle of the UAV's gimbal based on the first position and the UAV's current position.

[0167] In this embodiment, for each observation range, the controller calculates in real time, based on the observation progress (flight progress), the first position that can fall within the observation range (the target position of the pod observation center). Then, based on the first position, the controller controls the angle of the UAV's gimbal to automatically adjust the gimbal angle so that the adjusted position of the pod observation center is located at the first position, making the current shooting field of view compatible with the observation range. Therefore, after the inspection is completed, the entire shooting field of view can completely cover the inspected object, adapting to the inspected object and allowing for complete observation, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0168] Processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0169] The memory 302, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the gimbal control method in the embodiments of this application. The processor 301 can implement the gimbal control method in any of the above-described method embodiments by running the non-transitory software programs, instructions, and modules stored in the memory 302. To avoid repetition, these will not be described again here.

[0170] Specifically, memory 302 may include volatile memory (VM), such as random access memory (RAM); memory 302 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 302 may also include combinations of the above types of memory.

[0171] In this embodiment, memory 302 may further include memory remotely configured relative to the processor, and this remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0172] It should be noted that the controller described above can execute the gimbal control method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the controller embodiments can be found in the gimbal control method provided in the embodiments of this application.

[0173] This application also provides a drone, which includes a gimbal and the controller described in the above embodiments. The controller is used to control the angle of the gimbal. The gimbal is a support device for mounting shooting equipment. Specifically, the shooting equipment or other sensors are housed in a pod and mounted on the gimbal.

[0174] The controller can calculate in real time the first position (target position of the pod observation center) that can fall within each observation range based on the observation progress (flight progress). Then, based on the first position, it controls the angle of the UAV's gimbal to automatically adjust the angle so that the position of the pod observation center is located at the first position, making the current shooting field of view adapt to the observation range. Therefore, after the inspection is completed, the entire shooting field of view can completely cover the inspected object, adapt to the inspected object, and conduct a complete observation of the inspected object, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0175] Therefore, the drone can also adjust the gimbal angle in real time during the inspection process, adapt to the inspection object, and conduct a complete observation of the inspection object, effectively avoiding the risk of missed inspections and improving inspection efficiency.

[0176] This application also provides a drone inspection system, including: a plurality of ground monitoring stations and the drone described in the above embodiments. The ground monitoring stations are communicatively connected to the drones, wherein the wireless network can be a Bluetooth network, a WiFi network, a wireless cellular network, or a combination thereof, and is not limited thereto.

[0177] The drone is used to inspect observation areas along the designated inspection route and transmits the images and inspection information acquired during the inspection to a ground monitoring station, enabling the station to control the drone. Specifically, before automating the inspection of the target object, the inspection route can be determined first. While inspecting the observation area along this route, the drone transmits the images and inspection information acquired during the inspection to the ground monitoring station. The ground monitoring station can then display the images, helping staff to understand the inspection status of the target object in a timely manner. Furthermore, staff can also control the drone's operations through the ground monitoring station.

[0178] The drone can adjust the gimbal angle in real time during the inspection process, adapting to the inspection object and conducting a complete observation of it, effectively avoiding the risk of missed inspections and improving inspection efficiency. Therefore, this drone inspection system has the same functions as a drone, which will not be repeated here.

[0179] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the gimbal control method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0180] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. The processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the method steps of the gimbal control method provided in the above embodiments.

[0181] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A gimbal control method applied to a drone, characterized in that, The method includes: acquiring inspection information of the high-voltage power transmission line inspected by the UAV, the inspection information including an observation segment, an observation interval corresponding to the observation segment, a total flight distance corresponding to the observation interval, and a current flight distance, wherein the observation segment is the flight segment of the UAV used to inspect the observation interval; the observation interval is the area between two adjacent observation feature points on the observation path, and the observation path refers to the scanning line of the field of view required to observe the high-voltage power transmission line during the inspection mission flight; determining the observation progress of the UAV based on the current flight distance and the total flight distance; and determining a first position based on the observation progress and the observation interval, the first position being the UAV's... The target position of the pod's field of view center point, which moves along the observation path; controlling the angle of the UAV's gimbal based on the first position and the current position of the UAV; determining the first position based on the observation progress and the observation interval includes: obtaining the starting coordinates and ending coordinates of the observation interval; determining the first unit vector pointing from the starting point to the ending point of the observation interval based on the starting coordinates and the ending coordinates; calculating the coordinates of the first position using the following formula: P = s + vec × present, where P is the coordinate of the first position, s is the starting coordinate, vec is the first unit vector, and present is the observation progress.

2. The method according to claim 1, characterized in that, Determining the observation progress of the UAV based on the current flight range and the total flight range includes: determining the flight progress of the UAV based on the current flight range and the total flight range; and determining the observation progress of the UAV based on the flight progress.

3. The method according to claim 2, characterized in that, Determining the flight progress of the UAV based on the current flight range and the total flight range includes: determining the first ratio of the current flight range to the total flight range as the flight progress of the UAV.

4. The method according to claim 2, characterized in that, The step of determining the observation progress of the UAV based on the flight progress includes: determining the observation progress of the UAV as a second ratio of the flight progress to a preset coefficient, wherein the preset coefficient is less than or equal to 1.

5. The method according to claim 1, characterized in that, The step of controlling the gimbal angle of the UAV based on the first position and the current position of the UAV includes: determining a second unit vector from the current position of the UAV to the first position based on the coordinates of the first position and the coordinates of the current position of the UAV; determining the pitch angle and yaw angle of the gimbal based on the second unit vector; and controlling the angle of the gimbal based on the pitch angle and the yaw angle.

6. The method according to any one of claims 1-5, characterized in that, Before obtaining the inspection information of the UAV, the method further includes: obtaining the mapping relationship between the observation segment and the observation interval; determining the observation interval corresponding to the observation segment based on the mapping relationship and the observation segment; determining all the observation segments associated with the observation interval based on the mapping relationship and the observation interval; and determining the sum of the flight distances of all the observation segments as the total flight distance corresponding to the observation interval.

7. The method according to claim 6, characterized in that, The method further includes: obtaining the flight speed of the UAV in the observation segment; and calculating the current flight distance corresponding to the observation segment using the following formula: Where S is the current flight distance corresponding to the observation interval, t is the time from when the UAV starts observing the observation interval to the current position of the UAV, and V is the flight speed of the UAV.

8. The method according to claim 7, characterized in that, When the UAV observes a new observation range, the UAV's flight progress, current flight distance, and observation progress are all recalculated from zero.

9. The method according to any one of claims 2-5, characterized in that, When the UAV performs a temporary mission, the method further includes: if the UAV performs the temporary mission along the current flight path, then continue to perform the step of determining the flight progress of the UAV based on the current flight distance and the total flight distance, until the temporary mission ends; if the UAV flies away from the current flight path to perform the temporary mission, then lock the current flight progress of the UAV until the temporary mission ends.

10. A controller, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the gimbal control method as described in any one of claims 1-9.

11. A drone, characterized in that, include: The gimbal and the controller as described in claim 10, the controller being used to control the angle of the gimbal.

12. A drone inspection system, characterized in that, include: The system comprises several ground monitoring stations and the drone as described in claim 11; the ground monitoring stations are communicatively connected to the drone, which is used to inspect the observation area along the observation route and transmit the image information of the observation route and the inspection information obtained during the inspection to the ground monitoring stations so that the ground monitoring stations can control the drone.

Images