A method and system for controlling the gimbal of an inspection drone

Abstract

This invention discloses a gimbal control method and system for an inspection drone, relating to the field of drone technology. It solves the technical problem of simultaneously inspecting target points above and below using two cameras, calculating the comprehensive deviation value by obtaining multiple key coordinate points through image tracking, and then adjusting the deviation value. The method includes the following steps: acquiring real-time 3D map information of the inspection area; installing two cameras above and below the drone gimbal; obtaining target point coordinate information, camera coordinate information, and gimbal coordinate information from the images captured by the two cameras; obtaining the above and below spatial deviation values ​​using the target spatial angle values, the gimbal spatial angle values, and the reference angle value; and adjusting the angle and direction of the comprehensive deviation value using the minimum rotation angle adjustment unit value and the comprehensive rotation angle.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) technology, specifically relating to a gimbal control method and system for an inspection UAV. Background Technology

[0002] With the continuous development of technology, drones are being used more and more widely in various fields, especially in the field of inspection. The three-axis gimbal of the drone is equipped with two high-definition cameras, one above and one below, to simultaneously locate and monitor the target point. The contour tracking function is used to traverse the contour to extract the coordinates. The deviation value is combined with multiple coordinates, and the deviation angle is adjusted by adjusting the minimum rotation angle unit value. The rotation direction is determined and adjusted by combining the comprehensive rotation direction.

[0003] While existing drone gimbal control methods have achieved drone gimbal control to a certain extent, traditional gimbals can only be mounted on the bottom or top of the drone. They cannot simultaneously inspect targets above and below with a single gimbal. They lack the ability to extract target point coordinates by traversing contours using contour tracking functions, and they lack the ability to obtain a comprehensive deviation value from the spatial angle formed by multiple coordinate relative vectors and adjust the angle using the minimum rotation angle adjustment unit value. It is also difficult to determine the overall rotation direction of the camera. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; to this end, this invention proposes a gimbal control method and system for inspection drones, used to solve the following technical problems:

[0005] While existing drone gimbal control methods have achieved drone gimbal control to a certain extent, traditional gimbals can only be mounted on the bottom or top of the drone. They cannot simultaneously inspect targets above and below with a single gimbal. They lack the ability to extract target point coordinates by traversing contours using contour tracking functions, and they lack the ability to obtain a comprehensive deviation value from the spatial angle formed by multiple coordinate relative vectors and adjust the angle using the minimum rotation angle adjustment unit value. It is also difficult to determine the overall rotation direction of the camera.

[0006] To address the above problems, a first aspect of the present invention provides a gimbal control method for an inspection drone, comprising the following steps:

[0007] S1: Real-time acquisition of 3D map information of the inspection area, and planning of the inspection path of the UAV gimbal based on the 3D map information;

[0008] S2: Install two cameras on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below.

[0009] S3: Control the drone gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two cameras above and below;

[0010] S4: Obtain the upper and lower target space angle values ​​by using the target point coordinates obtained from the upper and lower camera devices and the upper and lower camera coordinates respectively. Obtain the upper and lower gimbal space angle values ​​by using the upper and lower target point coordinates and the gimbal coordinates respectively. Obtain the upper space deviation value based on the upper target space angle value, the upper gimbal space angle value and the reference angle value. Obtain the lower space deviation value based on the lower target space angle value, the lower gimbal space angle value and the reference angle value.

[0011] S5: Obtain the comprehensive angle deviation value by using the spatial deviation values ​​above and below. Determine whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value and make adjustments. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle by using the balance center point obtained from the coordinates of the upper target point and the lower target point.

[0012] As a further aspect of the present invention: step S1 includes the following steps:

[0013] The inspection area is acquired in real time by multiple sensors, including key equipment and target points. The inspection path of the UAV gimbal is planned based on the target points in the 3D map information.

[0014] As a further aspect of the present invention: step S2 includes the following steps:

[0015] Two cameras are mounted above and below the drone gimbal using a rotatable platform and two motors. The upper camera collects image data of the target point above, and the lower camera collects image data of the target point below. The collected image data of the target points above and below are then transmitted to the flight control system.

[0016] As a further aspect of the present invention: step S3 includes the following steps:

[0017] The flight control system initiates the operation, guiding the drone's gimbal along the planned inspection path. Simultaneously, the gimbal's movement directs the camera to capture image data from the target point. Edge detection is performed on the image data, using an edge detection algorithm to find edge information. The `findContours` contour tracking function then locates contours within the image. Each contour is iterated over, and its area is calculated. Contours with an area exceeding a preset threshold are marked as target contours. The target point coordinates are found by calculating the contour's center point. Based on the Z-axis coordinates, the coordinates of the upper and lower target points are extracted. Using the camera's internal and external parameters, the extracted target point coordinates are transformed from the image coordinate system to the actual 3D coordinate system, obtaining the real-time 3D coordinates of the upper and lower target points. Finally, the upper and lower camera coordinates and the gimbal's real-time 3D coordinates provided by the flight control system are fused with the 3D coordinate data of the upper and lower target points.

[0018] As a further aspect of the present invention: step S4 includes the following steps:

[0019] The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are combined with the vectors of the upper camera coordinates and the origin coordinates, and the vectors of the lower camera coordinates and the origin coordinates, respectively, to obtain the spatial angle values ​​of the upper and lower targets.

[0020] Formulas for calculating the spatial angles of the target above and below:

[0021]

[0022] Where i is 1 or 2, θ Ti For the target space angle value, when i is 1, θ T1 Let θ be the spatial angle value of the target above. When i is 2, θ T2 The target space angle value is below; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; C1 Y C1 Z C1 (X) represents the coordinates of the camera above, denoted as point C; C2 Y C2 Z C2 Let D be the coordinate point of the camera below, and let (0, 0, 0) be the origin of the drone's takeoff point.

[0023] The relative vectors of the target point coordinates above and below to the origin coordinates, and the relative vectors of the gimbal coordinates to the origin coordinates, are combined to obtain the spatial angle values ​​of the gimbal above and below.

[0024] Formulas for calculating the space angle of the upper and lower gimbals:

[0025]

[0026] Where i is 1 or 2, θ Pi Let θ be the spatial angle value of the gimbal. When i is 1, θ P1 θ is the spatial angle value of the upper gimbal. When i is 2, θ P2 The lower gimbal space angle value; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; P Y P Z P Let E be the coordinate point of the gimbal, and let the origin be the coordinate point of the UAV takeoff point, which is (0, 0, 0).

[0027] The upper and lower spatial deviation values ​​are obtained based on the target spatial angle values ​​above and below, the gimbal spatial angle values ​​above and below, and the reference angle value, respectively.

[0028] Formula for calculating the upper spatial deviation:

[0029]

[0030] Where D1 is the upper spatial deviation value, θ T1 θ represents the spatial angle of the target above. P1 θ is the spatial angle value of the upper gimbal, α1 and α2 are the upper deviation coefficients;

[0031] The formula for calculating the spatial deviation below is as follows:

[0032]

[0033] Where D2 is the lower spatial deviation value, θ T2 Let θ be the angular value of the target space below. P2 θ represents the lower gimbal space angle value, β1 and β2 represent the lower deviation coefficients.

[0034] As a further aspect of the present invention: step S5 includes the following steps:

[0035] The combined angular deviation value is obtained by combining the upper spatial deviation value and the lower spatial deviation value.

[0036] Formula for calculating the overall angle deviation:

[0037] D = D1 + D2

[0038] Where D is the comprehensive angular deviation calculation value, D1 is the upper spatial deviation value, and D2 is the lower spatial deviation value;

[0039] The comprehensive angle deviation value is used to determine whether it is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted accordingly.

[0040] Formula for calculating the minimum adjustment rotation angle unit value:

[0041]

[0042] Where E is the minimum rotation angle adjustment unit value, θ T1 θ represents the spatial angle of the target above. T2 θ represents the target space angle value below. P1 θ is the spatial angle value of the upper gimbal. P2 This refers to the spatial angle value of the gimbal below;

[0043] If the preset deviation value < D < 15°, then add a minimum adjustment rotation angle unit value E to the deviation angle;

[0044] If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0045] If 30° ≤ < D < 60°, then add three minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0046] If the overall angle deviation is less than the preset deviation, no adjustment is needed;

[0047] Based on the coordinates of the upper and lower target points, the balance center point is obtained, and the overall rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras, respectively.

[0048]

[0049] Among them, (X) T Y T Z T Let F be the equilibrium center point, and (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above. T2 Y T2Z T2 () represents the coordinates of the target point below;

[0050] Compare the distance between the balance center point F and the upper camera point C, and the distance between the balance center point F and the lower camera point D;

[0051] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be rotated upwards.

[0052] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be downward.

[0053] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, then the overall rotation direction of the cameras will be kept in balance.

[0054] As a further aspect of the present invention: the inspection drone gimbal control system is characterized by comprising:

[0055] 3D map navigation and path planning module: acquires 3D map information of the inspection area in real time, and plans the inspection path of the UAV gimbal based on the 3D map information;

[0056] Dual-camera inspection module: Two cameras are installed on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below.

[0057] Gimbal Inspection Path Control and Image Acquisition Module: Controls the UAV gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data, and gimbal coordinate point information obtained from the images acquired by the two cameras above and below;

[0058] The PTZ spatial deviation calculation module calculates the target spatial angle values ​​by obtaining the target point coordinates from the upper and lower camera devices and the upper and lower camera coordinates, respectively. It also obtains the upper and lower PTZ spatial angle values ​​by obtaining the target point coordinates from the PTZ coordinates. Based on the upper target spatial angle value, the upper PTZ spatial angle value, and the reference angle value, the upper spatial deviation value is obtained. Similarly, based on the lower target spatial angle value, the lower PTZ spatial angle value, and the reference angle value, the lower spatial deviation value is obtained.

[0059] The comprehensive angle deviation judgment and adjustment module obtains the comprehensive angle deviation value through the spatial deviation values ​​above and below, and judges whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is determined by the distance values ​​between the balance center point obtained by the coordinates of the upper target point and the lower target point and the upper and lower cameras, respectively.

[0060] The beneficial effects of this invention are:

[0061] This invention utilizes a three-axis gimbal mounted on a drone to carry two high-definition cameras and motors, enabling simultaneous positioning and monitoring of target points above and below. A contour tracking function is used to traverse the contour and extract coordinates. Based on the relative vectors of various coordinates, corresponding spatial angle values ​​are obtained. The combined deviation values ​​of these spatial angles are summed to obtain a comprehensive deviation value. The deviation angle is adjusted to the appropriate level using the minimum rotation angle adjustment unit value. Finally, the comprehensive rotation direction of the rotation angle is determined by the balance center point, enabling real-time adjustment of the deviation angle and improving inspection efficiency and accuracy. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a schematic diagram of the system framework of the present invention;

[0064] Figure 2 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Please see Figure 1-2 As shown, the present invention is a gimbal control method and system for an inspection drone, comprising the following steps:

[0067] S1: Real-time acquisition of 3D map information of the inspection area, and planning of the inspection path of the UAV gimbal based on the 3D map information;

[0068] S2: Install two cameras on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below.

[0069] S3: Control the drone gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two cameras above and below;

[0070] S4: Obtain the upper target space angle value and the lower target space angle value by using the target point coordinates obtained from the upper and lower camera devices and the upper and lower camera coordinates respectively. Obtain the upper and lower gimbal space angle values ​​by using the upper and lower target point coordinates and the gimbal coordinates respectively. Obtain the upper space deviation value based on the upper target space angle value, the upper gimbal space angle value and the reference angle value. Obtain the lower space deviation value based on the lower target space angle value, the lower gimbal space angle value and the reference angle value.

[0071] S5: Obtain the comprehensive angle deviation value by using the spatial deviation values ​​above and below. Determine whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value and make adjustments. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle by using the balance center point obtained from the coordinates of the upper target point and the lower target point.

[0072] Specifically, multiple sensors mounted on the drone collect data in real time. Image recognition algorithms identify key equipment and target points. Based on the identified target points, path planning algorithms plan the drone's inspection path. Upper and lower cameras simultaneously acquire image data of the target points above and below, and this data is transmitted to the flight control system in real time via wireless communication. After edge detection of the image data, edge detection algorithms find edge information in the image, and the findContours contour tracking function finds contours in the image. The coordinates of the target points are extracted from each contour, converted to actual 3D coordinates using internal and external camera parameters, and the coordinates of the upper and lower cameras and the real-time 3D coordinates of the gimbal are obtained from the flight control system. The internal parameters of the cameras are the camera lens coordinates. The parameters relating the camera and the imaging chip, including focal length, principal point coordinates, and distortion coefficients, determine how the camera maps a 3D scene from the real world onto a 2D image. The camera's external parameters, including rotation matrices and translation vectors, determine how the camera acquires data from different angles and positions. Accurate 3D reconstruction can be performed by acquiring both internal and external camera parameters. The upper and lower spatial deviations are obtained based on the target space angles above and below, the gimbal space angles above and below, and the reference angle. The calculated comprehensive angle deviation is compared with a preset deviation. If the comprehensive angle deviation is greater than the preset deviation, the minimum rotation angle adjustment unit value is calculated and adjusted. The comprehensive rotation direction is determined by the equilibrium center point obtained from the coordinates of the upper and lower target points.

[0073] In one embodiment of the present invention, step S1 includes the following steps:

[0074] The inspection area is acquired in real time by multiple sensors, including key equipment and target points. The inspection path of the UAV gimbal is planned based on the target points in the 3D map information.

[0075] Specifically, data is collected in real time by multiple sensors mounted on the drone. SLAM (Simultaneous Localization and Mapping) technology is used to locate the data collected by the sensors and build a 3D map. In the constructed 3D map, key equipment and target points are identified by image recognition algorithms and marked. Based on the identified target points, path planning algorithms are used to plan the inspection path for the drone.

[0076] In one embodiment of the present invention, step S2 includes the following steps:

[0077] Two cameras are mounted above and below the drone gimbal using a rotatable platform and two motors. The upper camera collects image data of the target point above, and the lower camera collects image data of the target point below. The collected image data of the target points above and below are then transmitted to the flight control system.

[0078] Specifically, a rotatable platform that can support two cameras and two motors is used. The motors need to have a certain torque and speed range to meet the shooting requirements of the cameras at different angles and heights. The two motors are connected to the control system of the rotatable platform to achieve synchronous rotation of the motors. During the flight of the drone, the two cameras will simultaneously collect image data of the target points above and below, and transmit the collected image data to the flight control system in real time through wireless communication technology.

[0079] In one embodiment of the present invention, step S3 includes the following steps:

[0080] The flight control system initiates the operation, guiding the drone's gimbal along the planned inspection path. Simultaneously, the gimbal's movement directs the camera to capture image data from the target point. Edge detection is performed on the image data, using an edge detection algorithm to find edge information. The `findContours` contour tracking function then locates contours within the image. Each contour is iterated over, and its area is calculated. Contours with an area exceeding a preset threshold are marked as target contours. The target point coordinates are found by calculating the contour's center point. Based on the Z-axis coordinates, the coordinates of the upper and lower target points are extracted. Using the camera's internal and external parameters, the extracted target point coordinates are transformed from the image coordinate system to the actual 3D coordinate system, obtaining the real-time 3D coordinates of the upper and lower target points. Finally, the upper and lower camera coordinates and the gimbal's real-time 3D coordinates provided by the flight control system are fused with the 3D coordinate data of the upper and lower target points.

[0081] Specifically, the flight control system is opened and initialized to ensure it is in standby mode. The pre-planned inspection path is loaded into the flight control system. The gimbal's movement range and speed are set according to the inspection requirements, allowing the camera to be aimed at the target point and obtain clear image data. Commands are sent to the flight control system to start the UAV gimbal to travel along the planned path. At the same time, the gimbal is controlled to make the camera aim at the target point on the inspection path. When the camera is aimed at the target point, the image acquisition function is triggered to acquire the current image data. Edge detection is performed on the acquired image data. The edge detection algorithm is used to find the edge information in the image. The findContours contour tracking function is used to find the contours in the image. The coordinates of the target points above and below are extracted by traversing each contour. The extracted target point coordinates are transformed from the image coordinate system to the actual three-dimensional coordinate system. The real-time coordinates of the upper and lower cameras and the gimbal are obtained from the flight control system.

[0082] In one embodiment of the present invention, step S4 includes the following steps:

[0083] The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are combined with the vectors of the upper camera coordinates and the origin coordinates, and the vectors of the lower camera coordinates and the origin coordinates, respectively, to obtain the spatial angle values ​​of the upper and lower targets.

[0084] Formulas for calculating the spatial angles of the target above and below:

[0085]

[0086] Where i is 1 or 2, θ Ti For the target space angle value, when i is 1, θ T1 Let θ be the spatial angle value of the target above. When i is 2, θ T2 The target space angle value is below; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; C1 Y C1 Z C1 (X) represents the coordinates of the camera above, denoted as point C; C2 Y C2 Z C2 Let D be the coordinate point of the camera below, and let (0, 0, 0) be the origin of the drone's takeoff point.

[0087] The relative vectors of the target point coordinates above and below to the origin coordinates, and the relative vectors of the gimbal coordinates to the origin coordinates, are combined to obtain the spatial angle values ​​of the gimbal above and below.

[0088] Formulas for calculating the space angle of the upper and lower gimbals:

[0089]

[0090] Where i is 1 or 2, θ Pi Let θ be the spatial angle value of the gimbal. When i is 1, θ P1 θ is the spatial angle value of the upper gimbal. When i is 2, θ P2 The lower gimbal space angle value; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; P Y P Z P Let E be the coordinate point of the gimbal, and let the origin be the coordinate point of the UAV takeoff point, which is (0, 0, 0).

[0091] The upper and lower spatial deviation values ​​are obtained based on the target spatial angle values ​​above and below, the gimbal spatial angle values ​​above and below, and the reference angle value, respectively.

[0092] Formula for calculating the upper spatial deviation:

[0093]

[0094] Where D1 is the upper spatial deviation value, θ T1 θ represents the spatial angle of the target above. P1 θ is the spatial angle value of the upper gimbal, α1 and α2 are the upper deviation coefficients;

[0095] The formula for calculating the spatial deviation below is as follows:

[0096]

[0097] Where D2 is the lower spatial deviation value, θ T2 θ represents the target space angle value below. P2 θ is the lower gimbal spatial angle value, β1 and β2 are the lower deviation coefficients.

[0098] Specifically, the target point coordinates and origin coordinates are obtained from the upper and lower camera devices respectively. The target point coordinates are subtracted from the origin coordinates to obtain the target point vector relative to the origin. Combined with the upper and lower camera coordinates, the camera vectors are subtracted from the origin to obtain the camera vectors relative to the origin. The upper and lower target spatial angle values ​​are obtained based on the upper and lower relative vectors. The target point coordinates are subtracted from the origin coordinates to obtain the target point vector relative to the origin. Combined with the gimbal coordinates, the gimbal vectors are subtracted from the origin to obtain the gimbal vector relative to the origin. The upper and lower gimbal spatial angle values ​​are obtained based on the upper and lower target spatial angle values, the upper and lower gimbal spatial angle values, and the reference angle value. Among them, α1 and α2 are the upper deviation coefficients, both 0.5; β1 and β2 are the lower deviation coefficients, both 0.4; the reference angle value θ is 90°.

[0099] In one embodiment of the present invention, step S5 includes the following steps:

[0100] The combined angular deviation value is obtained by combining the upper spatial deviation value and the lower spatial deviation value.

[0101] Formula for calculating the overall angle deviation:

[0102] D = D1 + D2

[0103] Where D is the comprehensive angular deviation calculation value, D1 is the upper spatial deviation value, and D2 is the lower spatial deviation value;

[0104] The comprehensive angle deviation value is used to determine whether it is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted accordingly.

[0105] Formula for calculating the minimum adjustment rotation angle unit value:

[0106]

[0107] Where E is the minimum rotation angle adjustment unit value, θ T1 θ represents the spatial angle of the target above. T2 θ represents the target space angle value below. P1 θ is the spatial angle value of the upper gimbal. P2 This refers to the spatial angle value of the gimbal below;

[0108] If the preset deviation value < D < 15°, then add a minimum adjustment rotation angle unit value E to the deviation angle;

[0109] If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0110] If 30° ≤ < D < 60°, then add three minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0111] If the overall angle deviation is less than the preset deviation, no adjustment is needed;

[0112] Based on the coordinates of the upper and lower target points, the balance center point is obtained, and the overall rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras, respectively.

[0113]

[0114] Among them, (X) T Y T Z T Let F be the equilibrium center point, and (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above. T2 Y T2 Z T2 () represents the coordinates of the target point below;

[0115] Compare the distance between the balance center point F and the upper camera point C, and the distance between the balance center point F and the lower camera point D;

[0116] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be rotated upwards.

[0117] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be downward.

[0118] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, then the overall rotation direction of the cameras will be kept in balance.

[0119] Specifically, a comprehensive angle deviation value is obtained by using the upper and lower spatial deviation values. The calculated value of the comprehensive angle deviation value is compared with a preset deviation value to determine whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and the corresponding level adjustment is performed. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. The preset deviation value is 5°. The comprehensive rotation direction is determined by comparing the distance values ​​between the key center point and the upper and lower cameras.

[0120] Angular velocity of the UAV [p,q,r] T and the resulting gimbal attitude angular velocity The relationship between them is

[0121]

[0122] When the center position deviation of the target image is [dy, dx] T At that time, in order to center it in the camera's image, the angle of the pan-tilt unit was adjusted as follows:

[0123]

[0124] Where, p x p y Let be the camera resolution, and λ be the field of view. x ,λ y .

[0125] In one embodiment of the present invention, the inspection drone gimbal control system is characterized by comprising:

[0126] 3D map navigation and path planning module: acquires 3D map information of the inspection area in real time, and plans the inspection path of the UAV gimbal based on the 3D map information;

[0127] Dual-camera inspection module: Two cameras are installed on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below.

[0128] Gimbal Inspection Path Control and Image Acquisition Module: Controls the UAV gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data, and gimbal coordinate point information obtained from the images acquired by the two cameras above and below;

[0129] The PTZ spatial deviation calculation module calculates the target spatial angle values ​​by obtaining the target point coordinates from the upper and lower camera devices and the upper and lower camera coordinates, respectively. It also obtains the upper and lower PTZ spatial angle values ​​by obtaining the target point coordinates from the PTZ coordinates. Based on the upper target spatial angle value, the upper PTZ spatial angle value, and the reference angle value, the upper spatial deviation value is obtained. Similarly, based on the lower target spatial angle value, the lower PTZ spatial angle value, and the reference angle value, the lower spatial deviation value is obtained.

[0130] The comprehensive angle deviation judgment and adjustment module obtains the comprehensive angle deviation value through the deviation values ​​of the upper and lower spaces, and judges whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is determined by the balance center point obtained by the coordinates of the upper target point and the lower target point.

[0131] Specifically, the target points identified by the image recognition algorithm in the 3D map navigation and path planning module are marked. Based on the identified target points, the path planning algorithm is used to plan the inspection path for the UAV. The dual-camera inspection module transmits the image data of the target points above and below to the flight control system. The gimbal inspection path control and image acquisition module uses the findContours contour tracking function to find the contours in the image, traverses each contour to extract the coordinates of the target points above and below and convert them into 3D coordinate points, and obtains the coordinates of the upper and lower cameras and the real-time 3D coordinates of the gimbal from the flight control system. The gimbal spatial deviation calculation module obtains the upper and lower spatial deviation values ​​by using the upper and lower target spatial angle values, the upper and lower gimbal spatial angle values ​​and the reference angle value. The comprehensive angle deviation judgment and adjustment module adjusts the angle and direction of the comprehensive deviation angle based on the minimum rotation angle adjustment unit value and the comprehensive rotation angle.

[0132] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A gimbal control method for an inspection drone, characterized in that, The method includes the following steps: S1: Real-time acquisition of 3D map information of the inspection area, and planning of the inspection path of the UAV gimbal based on the 3D map information; S2: Install two cameras on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below. S3: Control the drone gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two cameras above and below; S4: Obtain the upper and lower target space angle values ​​by using the target point coordinates obtained from the upper and lower camera devices and the upper and lower camera coordinates respectively. Obtain the upper and lower gimbal space angle values ​​by using the upper and lower target point coordinates and the gimbal coordinates respectively. Obtain the upper space deviation value based on the upper target space angle value, the upper gimbal space angle value and the reference angle value. Obtain the lower space deviation value based on the lower target space angle value, the lower gimbal space angle value and the reference angle value. S5: Obtain the comprehensive angle deviation value by using the spatial deviation values ​​above and below. Determine whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value and make adjustments. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle by using the balance center point obtained from the coordinates of the upper target point and the lower target point.

2. The gimbal control method for an inspection drone according to claim 1, characterized in that, Step S1 includes: The inspection area is acquired in real time by multiple sensors, including key equipment and target points. The inspection path of the UAV gimbal is planned based on the target points in the 3D map information.

3. The gimbal control method for an inspection drone according to claim 1, characterized in that, Step S2 includes: Two cameras are mounted above and below the drone gimbal using a rotatable platform and two motors. The upper camera collects image data of the target point above, and the lower camera collects image data of the target point below. The collected image data of the target points above and below are then transmitted to the flight control system.

4. The gimbal control method for an inspection drone according to claim 1, characterized in that, Step S3 includes the following steps: The flight control system initiates the operation, guiding the drone's gimbal along the planned inspection path. Simultaneously, the gimbal's movement directs the camera to capture image data from the target point. Edge detection is performed on the image data, using an edge detection algorithm to find edge information. The `findContours` contour tracking function then locates contours in the image, iterating through each contour and calculating its area. Contours with an area greater than a preset threshold are marked as target contours. The center point of each contour is calculated as the target point coordinates. Based on the Z-axis coordinates, the coordinates of the upper and lower target points are extracted. Using the camera's internal and external parameters, the extracted target point coordinates are transformed from the image coordinate system to the actual 3D coordinate system, obtaining the real-time 3D coordinates of the upper and lower target points. These coordinates are then fused with the upper and lower camera coordinates and the gimbal's real-time 3D coordinates provided by the flight control system, along with the 3D coordinate data of the upper and lower target points.

5. The gimbal control method for an inspection drone according to claim 1, characterized in that, The process of converting the target point coordinates extracted from the image coordinate system to the actual three-dimensional coordinate system using the camera's internal and external parameters includes: The image coordinate system is the IMU coordinate system, i.e., the camera coordinate system. The attitude of the gimbal camera in the three-dimensional space of the earth is calculated in real time by the inertial sensor measurement unit (IMU) and expressed as Euler angles or direction cosine matrix. When the Euler angles of the camera are pitch α, roll β, and yaw γ, it can be converted into the direction cosine matrix C. bn This indicates a transformation from the n-camera coordinate system to the b-Earth coordinate system, where the distance from the target point to the camera in the camera coordinate system is represented by C. bn The expression for converting to Earth's three-dimensional coordinate system is as follows:

6. The gimbal control method for an inspection drone according to claim 1, characterized in that, Step S4 includes the following steps: The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are combined with the vectors of the upper camera coordinates and the origin coordinates, and the vectors of the lower camera coordinates and the origin coordinates, respectively, to obtain the spatial angle values ​​of the upper and lower targets. Formulas for calculating the spatial angles of the target above and below: Where i is 1 or 2, θ Ti For the target space angle value, when i is 1, θ T1 Let θ be the spatial angle value of the target above. When i is 2, θ T2 The target space angle value is below; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; C1 Y C1 Z C1 (X) represents the coordinates of the camera above, denoted as point C; C2 Y C2 Z C2 Let D be the coordinate point of the camera below, and let (0, 0, 0) be the origin of the drone's takeoff point. The relative vectors of the target point coordinates above and below to the origin coordinates, and the relative vectors of the gimbal coordinates to the origin coordinates, are combined to obtain the spatial angle values ​​of the gimbal above and below. Formulas for calculating the space angle of the upper and lower gimbals: Where i is 1 or 2, θ Pi Let θ be the spatial angle value of the gimbal. When i is 1, θ P1 θ is the spatial angle value of the upper gimbal. When i is 2, θ P2 The lower gimbal space angle value; (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above, denoted as point A; T2 Y T2 Z T2 (X) represents the coordinates of the target point below, denoted as point B; P Y P Z P Let E be the coordinate point of the gimbal, and let the origin be the coordinate point of the UAV takeoff point, which is (0, 0, 0). The upper and lower spatial deviation values ​​are obtained based on the target spatial angle values ​​above and below, the gimbal spatial angle values ​​above and below, and the reference angle value, respectively. Formula for calculating the upper spatial deviation: Where D1 is the upper spatial deviation value, θ T1 θ represents the spatial angle of the target above. P1 θ is the spatial angle value of the upper gimbal, α1 and α2 are the upper deviation coefficients; The formula for calculating the spatial deviation below is as follows: Where D2 is the lower spatial deviation value, θ T2 Let θ be the angular value of the target space below. P2 θ represents the lower gimbal space angle value, β1 and β2 represent the lower deviation coefficients.

7. The gimbal control method for an inspection drone according to claim 6, characterized in that, Step S5 includes the following steps: The combined angular deviation value is obtained by combining the upper spatial deviation value and the lower spatial deviation value. Formula for calculating the overall angle deviation: D = D1 + D2 Where D is the comprehensive angular deviation calculation value, D1 is the upper spatial deviation value, and D2 is the lower spatial deviation value; The comprehensive angle deviation value is used to determine whether it is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted accordingly. Formula for calculating the minimum adjustment rotation angle unit value: Where E is the minimum rotation angle adjustment unit value, θ T1 θ represents the spatial angle of the target above. T2 Let θ be the angular value of the target space below. P1 θ is the spatial angle value of the upper gimbal. P2 This refers to the spatial angle value of the gimbal below; If the preset deviation value < D < 15°, then add a minimum adjustment rotation angle unit value E to the deviation angle; If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle; If 30° ≤ < D < 60°, then add three minimum adjustment rotation angle unit values ​​E to the deviation angle; If the overall angle deviation is less than the preset deviation, no adjustment is needed; Based on the coordinates of the upper and lower target points, the balance center point is obtained, and the overall rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras, respectively. Among them, (X) T Y T Z T Let F be the equilibrium center point, and (X) T1 Y T1 Z T1 (X) represents the coordinates of the target point above. T2 Y T2 Z T2 () represents the coordinates of the target point below.

8. A gimbal control method for an inspection drone according to claim 6, characterized in that, The method includes: Compare the distance between the balance center point F and the upper camera point C, and the distance between the balance center point F and the lower camera point D; If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be rotated upwards. If the distance between the coordinates of the balance center point F and the coordinates of the upper camera C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera D, then the overall rotation direction of the camera will be downward. If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, then the overall rotation direction of the cameras will be kept in balance.

9. The gimbal control method for an inspection drone according to claim 1, characterized in that, The angular velocity of the UAV body [p,q,r] T and the resulting gimbal attitude angular velocity The relationship between them is When the center position deviation of the target image is [dy, dx] T At that time, in order to center it in the camera's image, the angle of the pan-tilt unit was adjusted as follows: Where, p x p y Let be the camera resolution, and λ be the field of view. x ,λ y .

10. A gimbal control system for an inspection drone, characterized in that, The system includes: 3D map navigation and path planning module: acquires 3D map information of the inspection area in real time, and plans the inspection path of the UAV gimbal based on the 3D map information; Dual-camera inspection module: Two cameras are installed on the drone gimbal. One camera faces upward to inspect target points above, and the other camera faces downward to inspect target points below. Gimbal Inspection Path Control and Image Acquisition Module: Controls the UAV gimbal to travel along the planned inspection path through the flight control system, based on the target point coordinate information data, camera coordinate point information data, and gimbal coordinate point information obtained from the images acquired by the two cameras above and below; The PTZ spatial deviation calculation module calculates the target spatial angle values ​​by obtaining the target point coordinates from the upper and lower camera devices and the upper and lower camera coordinates, respectively. It also obtains the upper and lower PTZ spatial angle values ​​by obtaining the target point coordinates from the PTZ coordinates. Based on the upper target spatial angle value, the upper PTZ spatial angle value, and the reference angle value, the upper spatial deviation value is obtained. Similarly, based on the lower target spatial angle value, the lower PTZ spatial angle value, and the reference angle value, the lower spatial deviation value is obtained. The comprehensive angle deviation judgment and adjustment module obtains the comprehensive angle deviation value through the deviation values ​​of the upper and lower spaces, and judges whether the comprehensive angle deviation value is greater than the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated and adjusted. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is determined by the balance center point obtained by the coordinates of the upper target point and the lower target point.

Images