Patrol unmanned aerial vehicle holder control method and system

Abstract

The invention discloses an inspection unmanned aerial vehicle holder control method and system, relates to the technical field of unmanned aerial vehicles, and solves the technical problems that two cameras are utilized to inspect upper and lower target points at the same time, a plurality of key coordinate points are obtained through image tracking, upper and lower space angles are calculated to obtain a comprehensive deviation value, and the comprehensive deviation value is adjusted. Comprising the following steps: acquiring three-dimensional map information of an inspection area in real time; two cameras are mounted above and below the holder of the unmanned aerial vehicle; obtaining target point coordinate information data, camera coordinate point information data and holder coordinate point information according to the images collected by the upper camera device and the lower camera device; obtaining an upper space deviation value and a lower space deviation value through the upper target space angle value, the lower target space angle value, the upper holder space angle value, the lower holder space angle value and the reference angle value; and the angle and direction of the comprehensive deviation value are adjusted through the minimum rotation angle adjusting unit value and the comprehensive rotation angle.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a method and system for controlling a pan / tilt platform of an inspection unmanned aerial vehicle. Background Art

[0002] With the continuous development of science and technology, drones are being used more and more widely in various fields, especially in the inspection field. The three-axis gimbal of the drone is equipped with two upper and lower high-definition cameras to simultaneously locate and monitor the target points. The contour tracking function is used to traverse the contour to extract the coordinates, and the deviation values ​​are combined with multiple coordinates. The deviation angle is adjusted by the minimum rotation angle adjustment unit value, and the rotation direction is determined and adjusted in combination with the comprehensive rotation direction.

[0003] Although the existing UAV gimbal control method has achieved UAV gimbal control to a certain extent, the traditional gimbal can only be mounted under or above the UAV, and it is impossible for a gimbal to inspect both the upper and lower targets at the same time. There is a lack of contour tracking function to traverse the contour to extract the target point coordinates, and there is a lack of comprehensive deviation values ​​obtained from the spatial angles composed of multiple coordinate relative vectors and the angle adjustment through the minimum rotation angle adjustment unit value, making it difficult to determine the comprehensive rotation direction of the camera. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a method and system for controlling a pan / tilt platform of an inspection drone, which are used to solve the following technical problems:

[0005] Although the existing UAV gimbal control method has achieved UAV gimbal control to a certain extent, the traditional gimbal can only be mounted under or above the UAV, and it is impossible for a gimbal to inspect both the upper and lower targets at the same time. There is a lack of contour tracking function to traverse the contour to extract the target point coordinates, and there is a lack of comprehensive deviation values ​​obtained from the spatial angles composed of multiple coordinate relative vectors and the angle adjustment through the minimum rotation angle adjustment unit value, making it difficult to determine the comprehensive rotation direction of the camera.

[0006] To solve the above problems, the first aspect of the present invention provides a method for controlling a pan / tilt platform of an inspection drone, comprising the following steps:

[0007] S1: Obtain the three-dimensional map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the three-dimensional map information;

[0008] S2: Two cameras are installed on the gimbal of the drone, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below;

[0009] S3: Control the gimbal of the drone to travel along the planned inspection route through the flight control system, according to the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two camera devices above and below;

[0010] S4: obtaining upper and lower target space angle values ​​through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtaining upper and lower pan-tilt space angle values ​​through the upper and lower target point coordinate points and the pan-tilt coordinate points, obtaining the upper space deviation value according to the upper target space angle value, the upper pan-tilt space angle value and the reference angle value, and obtaining the lower space deviation value according to the lower target space angle value, the lower pan-tilt space angle value and the reference angle value;

[0011] S5: Obtain a comprehensive angle deviation value through the upper and lower spatial deviation values, and determine whether the comprehensive angle deviation value is greater than a preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle through the balance center point obtained through the upper target point coordinates and the lower target point coordinates.

[0012] As a further solution of the present invention: Step S1 comprises the following steps:

[0013] Through a variety of sensors, the three-dimensional map information of the inspection area is obtained in real time, including key equipment and target points for inspection, and the inspection path of the drone gimbal is planned according to the target points in the three-dimensional map information.

[0014] As a further solution of the present invention: the step S2 comprises the following steps:

[0015] Two cameras are installed above and below the UAV gimbal using a rotatable platform and two motors. The upper camera collects image data of the upper target point, and the lower camera collects image data of the lower target point. The collected image data of the upper and lower target points are transmitted to the flight control system.

[0016] As a further solution of the present invention: Step S3 comprises the following steps:

[0017] Start the flight control system to control the UAV gimbal to drive along the planned inspection route. At the same time, control the movement of the gimbal so that the camera is aimed at the target point to collect image data. After edge detection of the image data, use the edge detection algorithm to find the edge information in the image, and then use the findContours contour tracking function to find the contour in the image. Traverse each contour and calculate its area value. If the contour area is greater than the preset area threshold, it is marked as the target contour. The target point coordinates are found by calculating the center point of the contour, and the coordinates of the upper and lower target points are extracted according to the coordinate values ​​on the Z axis. The extracted target point coordinates are converted from the image coordinate system to the actual three-dimensional coordinate system through the internal and external parameters of the camera, and the real-time three-dimensional coordinates of the upper and lower target points are obtained. The upper and lower camera coordinates and the real-time three-dimensional coordinates of the gimbal provided in the flight control system are fused with the three-dimensional coordinate data of the upper and lower target points.

[0018] As a further solution of the present invention: the step S4 comprises the following steps:

[0019] The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are respectively combined with the vectors of the upper camera coordinate point and the origin coordinates and the vectors of the lower camera coordinate point and the origin coordinates to obtain the upper and lower target space angle values ​​respectively;

[0020] The upper and lower target space angle calculation formula is:

[0021]

[0022] Where i is 1 or 2, θ Ti is the target space angle value. When i is 1, θ T1 is the spatial angle value of the upper target. When i is 2, θ T2 is the spatial angle value of the target below; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X C1 , Y C1 , Z C1 ) is the coordinate point of the upper camera, denoted as point C; (X C2 , Y C2 , Z C2 ) is the coordinate point of the camera below, recorded as point D, and the origin of the coordinates is the coordinate of the take-off point of the drone, which is (0, 0, 0);

[0023] The relative vectors of the upper and lower target point coordinates and the origin coordinates, and the relative vectors of the gimbal coordinates and the origin coordinates are combined to obtain the upper and lower gimbal space angle values ​​respectively;

[0024] The calculation formula for the upper and lower gimbal space angles is:

[0025]

[0026] Where i is 1 or 2, θ Pi is the PTZ space angle value. When i is 1, θ P1 is the spatial angle value of the upper gimbal. When i is 2, θ P2 is the space angle value of the lower gimbal; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X P , Y P , Z P ) is the gimbal coordinate point, recorded as point E, and the coordinate origin is the coordinate of the drone take-off point, which is (0, 0, 0);

[0027] The upper and lower space deviation values ​​are obtained respectively according to the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle value;

[0028] The calculation formula of the upper space deviation value is:

[0029]

[0030] Where D1 is the upper space deviation value, θ T1 is the spatial angle of the upper target, θ P1 is the upper gimbal space angle value, θ is the reference angle value, α1 and α2 are the upper deviation coefficients;

[0031] The calculation formula of the deviation value of the lower space is:

[0032]

[0033] Where D2 is the deviation value of the lower space, θ T2 is the spatial angle of the target below, θ P2 is the spatial angle value of the lower gimbal, θ is the reference angle value, and β1 and β2 are the lower deviation coefficients.

[0034] As a further solution of the present invention: the step S5 comprises the following steps:

[0035] The comprehensive angle deviation value is obtained by the upper space deviation value and the lower space deviation value;

[0036] The calculation formula of comprehensive angle deviation value is:

[0037] D=D1+D2

[0038] Where D is the calculated value of the comprehensive angle deviation, D1 is the upper space deviation value, and D2 is the lower space deviation value;

[0039] Determine whether the calculated value of the comprehensive angle deviation is greater than the preset deviation value according to the calculation value, if the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment;

[0040] The calculation formula for the minimum adjustment rotation angle unit value is:

[0041]

[0042] Among them, E is the minimum rotation angle adjustment unit value, θ T1 is the spatial angle of the upper target, θ T2 is the spatial angle of the target below, θ P1 is the spatial angle of the upper gimbal, θ P2 is the spatial angle value of the lower gimbal;

[0043] If the preset deviation value is less than D and less than 15°, a minimum adjustment rotation angle unit value E is added to the deviation angle;

[0044] If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0045] If 30°≤<D<60°, then three minimum adjustment rotation angle unit values ​​E are added to the deviation angle;

[0046] If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required;

[0047] The balance center point is obtained according to the coordinates of the upper target point and the lower target point, and the comprehensive rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras respectively;

[0048]

[0049] Among them, (X T , Y T , Z T ) is the equilibrium center point, denoted as point F, (X T1 , Y T1 , Z T1 ) is the coordinate of the upper target point, (X T2 , Y T2, Z T2 ) is the coordinate of the target point below;

[0050] Compare the distance between the balance center point F and the upper camera point C and the distance between the balance center point F and the lower camera point D;

[0051] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated upward;

[0052] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated downward;

[0053] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the overall rotation direction of the camera is kept balanced.

[0054] As a further solution of the present invention: the inspection drone gimbal control system is characterized by comprising:

[0055] 3D map navigation and path planning module: obtain 3D map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the 3D map information;

[0056] Dual-camera inspection module: Two cameras are installed on the drone gimbal, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below;

[0057] PTZ inspection path control and image acquisition module: The flight control system controls the drone PTZ to travel along the planned inspection path, and obtains target point coordinate information data, camera coordinate point information data and PTZ coordinate point information from the images collected by the two camera devices above and below;

[0058] PTZ space deviation value calculation module: obtain the upper and lower target space angle values ​​through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtain the upper and lower PTZ space angle values ​​through the upper and lower target point coordinate points and the PTZ coordinate points, obtain the upper space deviation value according to the upper target space angle value, the upper PTZ space angle value and the reference angle value, and obtain the lower space deviation value according to the lower target space angle value, the lower PTZ space angle value and the reference angle value;

[0059] Comprehensive angle deviation judgment and adjustment module: The comprehensive angle deviation is obtained through the upper and lower spatial deviation values, and it is judged whether the comprehensive angle deviation is greater than the preset deviation value. If the comprehensive angle deviation is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated for adjustment. If the comprehensive angle deviation is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is judged by the distance values ​​between the balance center point obtained by the upper target point coordinates and the lower target point coordinates and the upper and lower cameras respectively.

[0060] Beneficial effects of the present invention:

[0061] The present invention realizes simultaneous positioning and monitoring of upper and lower target points by carrying two upper and lower high-definition cameras and a motor on a three-axis gimbal of an unmanned aerial vehicle, uses a contour tracking function to traverse the contour to extract coordinates, obtains corresponding space angle values ​​according to relative vectors of multiple coordinates, adds data deviation values ​​composed of joint space angles to obtain a comprehensive deviation value, adjusts the deviation angle to a corresponding level through a minimum rotation angle adjustment unit value, and judges the comprehensive rotation direction of the rotation angle through a balance center point, thereby realizing real-time adjustment of the deviation angle and improving inspection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0064] Figure 2 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] See also Figure 1-2 As shown, the present invention is a method and system for controlling a pan / tilt platform of an inspection drone, comprising the following steps:

[0067] S1: Obtain the three-dimensional map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the three-dimensional map information;

[0068] S2: Two cameras are installed on the gimbal of the drone, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below;

[0069] S3: Control the gimbal of the drone to travel along the planned inspection route through the flight control system, according to the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two camera devices above and below;

[0070] S4: obtaining an upper target space angle value and a lower target space angle value through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtaining an upper target space angle value and a lower pan-tilt space angle value through the upper and lower target point coordinate points and the pan-tilt coordinate points, obtaining an upper space deviation value according to the upper target space angle value, the upper pan-tilt space angle value and the reference angle value, and obtaining a lower space deviation value according to the lower target space angle value, the lower pan-tilt space angle value and the reference angle value;

[0071] S5: Obtain a comprehensive angle deviation value through the upper and lower spatial deviation values, and determine whether the comprehensive angle deviation value is greater than a preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle through the balance center point obtained through the upper target point coordinates and the lower target point coordinates.

[0072] Specifically, data is collected in real time by a variety of sensors carried on the drone, key equipment and target points are identified through image recognition algorithms, and the inspection path of the drone is planned using a path planning algorithm based on the identified target points; the upper and lower cameras simultaneously collect image data of the upper and lower target points, and the collected image data is transmitted to the flight control system in real time through wireless communication technology; after edge detection of the image data, the edge detection algorithm is used to find the edge information in the image, and then the findContours contour tracking function is used to find the contour in the image, and each contour is traversed to extract the coordinates of the target point, which are converted to the coordinates in the actual three-dimensional coordinates through the internal and external parameters of the camera, and the upper and lower camera coordinates and the real-time three-dimensional coordinates of the gimbal are obtained from the flight control system, where the internal parameters of the camera are the camera mirror The parameters of the relationship between the head and the imaging chip, including focal length, principal point coordinates and distortion coefficient, determine how the camera maps the three-dimensional scene in the real world to a two-dimensional image. The external parameters of the camera are the position parameters of the camera in the world coordinate system, including the rotation matrix and translation vector, which determine how the camera obtains data from different angles and positions. By obtaining the internal and external parameters of the camera, three-dimensional reconstruction can be accurately performed; the upper and lower space deviation values ​​are obtained according to the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle value respectively; the calculated value of the comprehensive angle deviation value is compared with the preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated for adjustment, and the comprehensive rotation direction of the rotation angle is determined by the balance center point obtained by the upper target point coordinates and the lower target point coordinates.

[0073] In one embodiment of the present invention, the step S1 comprises the following steps:

[0074] Through a variety of sensors, the three-dimensional map information of the inspection area is obtained in real time, including key equipment and target points for inspection, and the inspection path of the drone gimbal is planned according to the target points in the three-dimensional map information.

[0075] Specifically, data is collected in real time by using a variety of sensors mounted on the drone, and SLAM synchronous positioning and mapping technology is used to locate the data collected by the sensors and construct a three-dimensional map. In the constructed three-dimensional map, key equipment and target points are identified and marked through an image recognition algorithm, and a path planning algorithm is used to plan the inspection path for the drone based on the identified target points.

[0076] In one embodiment of the present invention, the step S2 comprises the following steps:

[0077] Two cameras are installed above and below the UAV gimbal using a rotatable platform and two motors. The upper camera collects image data of the upper target point, and the lower camera collects image data of the lower target point. The collected image data of the upper and lower target points are transmitted to the flight control system.

[0078] Specifically, a rotatable platform that can carry two cameras and two motors is used. The motors need to have a certain torque and speed range to meet the shooting requirements of the cameras at different angles and heights. The two motors are respectively connected to the control system of the rotatable platform to achieve synchronous rotation of the motors. During the flight of the drone, the two cameras will simultaneously collect image data of the target points above and below, and transmit the collected image data to the flight control system in real time through wireless communication technology.

[0079] In one embodiment of the present invention, step S3 comprises the following steps:

[0080] Start the flight control system to control the UAV gimbal to drive along the planned inspection route. At the same time, control the movement of the gimbal so that the camera is aimed at the target point to collect image data. After edge detection of the image data, use the edge detection algorithm to find the edge information in the image, and then use the findContours contour tracking function to find the contour in the image. Traverse each contour and calculate its area value. If the contour area is greater than the preset area threshold, it is marked as the target contour. The target point coordinates are found by calculating the center point of the contour, and the coordinates of the upper and lower target points are extracted according to the coordinate values ​​on the Z axis. The extracted target point coordinates are converted from the image coordinate system to the actual three-dimensional coordinate system through the internal and external parameters of the camera, and the real-time three-dimensional coordinates of the upper and lower target points are obtained. The upper and lower camera coordinates and the real-time three-dimensional coordinates of the gimbal provided in the flight control system are fused with the three-dimensional coordinate data of the upper and lower target points.

[0081] Specifically, turn on and initialize the flight control system, ensure that the system is in standby mode, load the pre-planned inspection path into the flight control system, set the motion range and speed of the gimbal according to the inspection requirements, so that the camera can aim at the target point to obtain clear image data, send instructions to the flight control system, start the drone gimbal to drive according to the planned path, and control the gimbal to aim the camera at the target point on the inspection path. When the camera is aimed at the target point, trigger the image acquisition function to obtain the current image data, perform edge detection on the acquired image data, use the edge detection algorithm to find the edge information in the image, use the findContours contour tracking function to find the contour in the image, traverse each contour to extract the coordinates of the upper and lower target points, convert the extracted target point coordinates from the image coordinate system to the actual three-dimensional coordinate system, and obtain the real-time coordinates of the upper and lower cameras and the gimbal from the flight control system.

[0082] In one embodiment of the present invention, the step S4 comprises the following steps:

[0083] The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are respectively combined with the vectors of the upper camera coordinate point and the origin coordinates and the vectors of the lower camera coordinate point and the origin coordinates to obtain the upper and lower target space angle values ​​respectively;

[0084] The upper and lower target space angle calculation formula is:

[0085]

[0086] Where i is 1 or 2, θ Ti is the target space angle value. When i is 1, θ T1 is the spatial angle value of the upper target. When i is 2, θ T2 is the spatial angle value of the target below; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X C1 , Y C1 , Z C1 ) is the coordinate point of the upper camera, denoted as point C; (X C2 , Y C2 , Z C2 ) is the coordinate point of the camera below, recorded as point D, and the origin of the coordinates is the coordinate of the take-off point of the drone, which is (0, 0, 0);

[0087] The relative vectors of the upper and lower target point coordinates and the origin coordinates, and the relative vectors of the gimbal coordinates and the origin coordinates are combined to obtain the upper and lower gimbal space angle values ​​respectively;

[0088] The calculation formula for the upper and lower gimbal space angles is:

[0089]

[0090] Where i is 1 or 2, θ Pi is the PTZ space angle value. When i is 1, θ P1 is the spatial angle value of the upper gimbal. When i is 2, θ P2 is the space angle value of the lower gimbal; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X P , Y P , Z P ) is the gimbal coordinate point, recorded as point E, and the coordinate origin is the coordinate of the drone take-off point, which is (0, 0, 0);

[0091] The upper and lower space deviation values ​​are obtained respectively according to the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle value;

[0092] The calculation formula of the upper space deviation value is:

[0093]

[0094] Where D1 is the upper space deviation value, θ T1 is the spatial angle of the upper target, θ P1 is the upper gimbal space angle value, θ is the reference angle value, α1 and α2 are the upper deviation coefficients;

[0095] The calculation formula of the deviation value of the lower space is:

[0096]

[0097] Where D2 is the deviation value of the lower space, θ T2 is the spatial angle of the target below, θ P2 is the spatial angle value of the lower gimbal, θ is the reference angle value, and β1 and β2 are the lower deviation coefficients.

[0098] Specifically, the target point coordinates and the origin coordinates are obtained by the upper and lower camera devices respectively, and the target point coordinates are subtracted from the origin coordinates to obtain the vector of the target point relative to the origin. The upper and lower camera coordinates are combined and subtracted from the origin coordinates to obtain the vector of the camera relative to the origin. The upper and lower target space angle values ​​are obtained according to the upper and lower relative vectors respectively. The vector of the target point relative to the origin is obtained by subtracting the upper and lower target point coordinates from the origin coordinates. The vector of the gimbal relative to the origin is obtained by subtracting the gimbal coordinates from the origin coordinates. The upper and lower gimbal space angle values ​​are obtained according to the upper and lower relative vectors respectively. The upper and lower space deviation values ​​are obtained according to the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle value. Among them, α1 and α2 are upper deviation coefficients, and α1 and α2 are both 0.5; β1 and β2 are lower deviation coefficients, and β1 and β2 are both 0.4; the reference angle value θ is 90°.

[0099] In one embodiment of the present invention, the step S5 comprises the following steps:

[0100] The comprehensive angle deviation value is obtained by the upper space deviation value and the lower space deviation value;

[0101] The calculation formula of comprehensive angle deviation value is:

[0102] D=D1+D2

[0103] Where D is the calculated value of the comprehensive angle deviation, D1 is the upper space deviation value, and D2 is the lower space deviation value;

[0104] Determine whether the calculated value of the comprehensive angle deviation is greater than the preset deviation value according to the calculation value, if the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment;

[0105] The calculation formula for the minimum adjustment rotation angle unit value is:

[0106]

[0107] Among them, E is the minimum rotation angle adjustment unit value, θ T1 is the spatial angle of the upper target, θ T2 is the spatial angle of the target below, θ P1 is the spatial angle of the upper gimbal, θ P2 is the spatial angle value of the lower gimbal;

[0108] If the preset deviation value is less than D and less than 15°, a minimum adjustment rotation angle unit value E is added to the deviation angle;

[0109] If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle;

[0110] If 30°≤<D<60°, then three minimum adjustment rotation angle unit values ​​E are added to the deviation angle;

[0111] If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required;

[0112] The balance center point is obtained according to the coordinates of the upper target point and the lower target point, and the comprehensive rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras respectively;

[0113]

[0114] Among them, (X T , Y T , Z T ) is the equilibrium center point, denoted as point F, (X T1 , Y T1 , Z T1 ) is the coordinate of the upper target point, (X T2 , Y T2 , Z T2 ) is the coordinate of the target point below;

[0115] Compare the distance between the balance center point F and the upper camera point C and the distance between the balance center point F and the lower camera point D;

[0116] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated upward;

[0117] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated downward;

[0118] If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the overall rotation direction of the camera is kept balanced.

[0119] Specifically, the comprehensive angle deviation value is obtained by using the upper space deviation value and the lower space deviation value, and the calculated value of the comprehensive angle deviation value is compared with the preset deviation value to determine whether the comprehensive angle deviation value is greater than the preset deviation value; if the comprehensive angle deviation value is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated to perform corresponding level adjustment; if the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required, where the preset deviation value is 5°; and the distance value between the key center point and the upper camera and the distance value between the lower camera are compared to determine the comprehensive rotation direction.

[0120] Drone body angular velocity [p, q, r] T The angular velocity of the gimbal caused by The relationship between

[0121]

[0122] When the target image center position deviation is [dy, dx] T In order to make it in the center of the camera image, the angle of the pan / tilt adjustment is

[0123]

[0124] Among them, p x , p y is the camera resolution, and the viewing angles are x ,λ y .

[0125] In one embodiment of the present invention, the inspection drone gimbal control system is characterized by comprising:

[0126] 3D map navigation and path planning module: obtain 3D map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the 3D map information;

[0127] Dual-camera inspection module: Two cameras are installed on the drone gimbal, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below;

[0128] PTZ inspection path control and image acquisition module: The flight control system controls the drone PTZ to travel along the planned inspection path, and obtains target point coordinate information data, camera coordinate point information data and PTZ coordinate point information from the images collected by the two camera devices above and below;

[0129] PTZ space deviation value calculation module: obtain the upper and lower target space angle values ​​through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtain the upper and lower PTZ space angle values ​​through the upper and lower target point coordinate points and the PTZ coordinate points, obtain the upper space deviation value according to the upper target space angle value, the upper PTZ space angle value and the reference angle value, and obtain the lower space deviation value according to the lower target space angle value, the lower PTZ space angle value and the reference angle value;

[0130] Comprehensive angle deviation judgment and adjustment module: The comprehensive angle deviation is obtained through the upper and lower spatial deviation values, and it is judged whether the comprehensive angle deviation is greater than the preset deviation value. If the comprehensive angle deviation is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated for adjustment. If the comprehensive angle deviation is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is judged by the balance center point obtained through the coordinates of the upper target point and the lower target point.

[0131] Specifically, the target points identified by the image recognition algorithm in the three-dimensional map navigation and path planning module are marked, and the path planning algorithm is used to plan the inspection path of the UAV according to the identified target points; the image data of the upper and lower target points collected in the dual-camera inspection module is transmitted to the flight control system; the findContours contour tracking function is used in the gimbal inspection path control and image acquisition module to find the contours in the image, traverse each contour to extract the coordinates of the upper and lower target points and convert them into three-dimensional coordinate points, and obtain the upper and lower camera coordinates and the real-time three-dimensional coordinates of the gimbal from the flight control system; the gimbal space deviation value calculation module obtains the upper and lower space deviation values ​​respectively through the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle values; the comprehensive angle deviation value judgment and adjustment module adjusts the comprehensive deviation angle in angle and direction according to the minimum rotation angle adjustment unit value and the comprehensive rotation angle.

[0132] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for controlling a pan / tilt platform of an inspection drone, characterized in that: The method comprises the following steps: S1: Obtain the three-dimensional map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the three-dimensional map information; S2: Two cameras are installed on the gimbal of the drone, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below; S3: Control the gimbal of the drone to travel along the planned inspection route through the flight control system, according to the target point coordinate information data, camera coordinate point information data and gimbal coordinate point information obtained from the images collected by the two camera devices above and below; S4: obtaining upper and lower target space angle values ​​through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtaining upper and lower pan-tilt space angle values ​​through the upper and lower target point coordinate points and the pan-tilt coordinate points, obtaining the upper space deviation value according to the upper target space angle value, the upper pan-tilt space angle value and the reference angle value, and obtaining the lower space deviation value according to the lower target space angle value, the lower pan-tilt space angle value and the reference angle value; S5: Obtain a comprehensive angle deviation value through the upper and lower spatial deviation values, and determine whether the comprehensive angle deviation value is greater than a preset deviation value. If the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment. If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required. Determine the comprehensive rotation direction of the rotation angle through the balance center point obtained through the upper target point coordinates and the lower target point coordinates.

2. A method for controlling a pan / tilt platform of an inspection drone according to claim 1, characterized in that: The step S1 comprises: Through a variety of sensors, the three-dimensional map information of the inspection area is obtained in real time, including key equipment and target points for inspection, and the inspection path of the drone gimbal is planned according to the target points in the three-dimensional map information.

3. The method for controlling the pan / tilt platform of an inspection drone according to claim 1, characterized in that: The step S2 comprises: Two cameras are installed above and below the UAV gimbal using a rotatable platform and two motors. The upper camera collects image data of the upper target point, and the lower camera collects image data of the lower target point. The collected image data of the upper and lower target points are transmitted to the flight control system.

4. The method for controlling a pan / tilt platform of an inspection drone according to claim 1, characterized in that: The step S3 comprises the following steps: Start the flight control system to control the UAV gimbal to drive along the planned inspection route. At the same time, control the movement of the gimbal so that the camera is aimed at the target point to collect image data. After edge detection of the image data, use the edge detection algorithm to find the edge information in the image, and then use the findContours contour tracking function to find the contour in the image. Traverse each contour and calculate its area value. If the contour area is greater than the preset area threshold, it is marked as the target contour. The center point of the contour is calculated as the target point coordinate, and the coordinates of the upper and lower target points are extracted according to the coordinate value on the Z axis. The extracted target point coordinates are converted from the image coordinate system to the actual three-dimensional coordinate system through the internal and external parameters of the camera, and the real-time three-dimensional coordinates of the upper and lower target points are obtained. According to the upper and lower camera coordinates and the real-time three-dimensional coordinates of the gimbal provided in the flight control system, they are merged with the three-dimensional coordinate data of the upper and lower target points.

5. The method for controlling a pan / tilt platform of an inspection drone according to claim 1, characterized in that: The method of converting the extracted target point coordinates from the image coordinate system to the actual three-dimensional coordinate system by using the internal and external parameters of the camera includes: The image coordinate system is the IMU coordinate system, that is, the camera coordinate system. The posture of the gimbal camera in the three-dimensional space of the earth is solved in real time by the inertial sensor measurement unit IMU, which is expressed in Euler angles or direction cosine matrices. When the Euler angles of the camera are pitch α, roll β, and heading γ, they can be converted into direction cosine matrices C bn , which means that the distance from the target point to the camera in the camera coordinate system is converted from the n camera coordinate system to the b earth coordinate system. bn The expression converted to the earth's three-dimensional space coordinate system is:

6. The method for controlling a pan / tilt platform of an inspection drone according to claim 1, characterized in that: The step S4 comprises the following steps: The relative vectors of the target point coordinates and the origin coordinates obtained by the upper and lower camera devices are respectively combined with the vectors of the upper camera coordinate point and the origin coordinates and the vectors of the lower camera coordinate point and the origin coordinates to obtain the upper and lower target space angle values ​​respectively; The upper and lower target space angle calculation formula is: Where i is 1 or 2, θ Ti is the target space angle value. When i is 1, θ T1 is the spatial angle value of the upper target. When i is 2, θ T2 is the spatial angle value of the target below; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X C1 , Y C1 , Z C1 ) is the coordinate point of the upper camera, denoted as point C; (X C2 , Y C2 , Z C2 ) is the coordinate point of the camera below, recorded as point D, and the origin of the coordinates is the coordinate of the take-off point of the drone, which is (0, 0, 0); The relative vectors of the upper and lower target point coordinates and the origin coordinates, and the relative vectors of the gimbal coordinates and the origin coordinates are combined to obtain the upper and lower gimbal space angle values ​​respectively; The calculation formula for the upper and lower gimbal space angles is: Where i is 1 or 2, θ Pi is the PTZ space angle value. When i is 1, θ P1 is the spatial angle value of the upper gimbal. When i is 2, θ P2 is the space angle value of the lower gimbal; (X T1 , Y T1 , Z T1 ) is the coordinate point of the upper target point, recorded as point A; (X T2 , Y T2 , Z T2 ) is the coordinate point of the target point below, recorded as point B; (X P , Y P , Z P ) is the gimbal coordinate point, recorded as point E, and the coordinate origin is the coordinate of the drone take-off point, which is (0, 0, 0); The upper and lower space deviation values ​​are obtained respectively according to the upper and lower target space angle values, the upper and lower gimbal space angle values ​​and the reference angle value; The calculation formula of the upper space deviation value is: Where D1 is the upper space deviation value, θ T1 is the spatial angle of the upper target, θ P1 is the upper gimbal space angle value, θ is the reference angle value, α1 and α2 are the upper deviation coefficients; The calculation formula of the deviation value of the lower space is: Where D2 is the deviation value of the lower space, θ T2 is the spatial angle of the target below, θ P2 is the spatial angle value of the lower gimbal, θ is the reference angle value, and β1 and β2 are the lower deviation coefficients.

7. A method for controlling a pan / tilt platform of an inspection drone according to claim 6, characterized in that: The step S5 comprises the following steps: The comprehensive angle deviation value is obtained by the upper space deviation value and the lower space deviation value; The calculation formula of comprehensive angle deviation value is: D=D1+D2 Where D is the calculated value of the comprehensive angle deviation, D1 is the upper space deviation value, and D2 is the lower space deviation value; Determine whether the calculated value of the comprehensive angle deviation is greater than the preset deviation value according to the calculation value, if the comprehensive angle deviation value is greater than the preset deviation value, calculate the minimum rotation angle adjustment unit value for adjustment; The calculation formula for the minimum adjustment rotation angle unit value is: Among them, E is the minimum rotation angle adjustment unit value, θ T1 is the spatial angle of the upper target, θ T2 is the spatial angle of the target below, θ P1 is the spatial angle of the upper gimbal, θ P2 is the spatial angle value of the lower gimbal; If the preset deviation value is less than D and less than 15°, a minimum adjustment rotation angle unit value E is added to the deviation angle; If 15°≤D<30°, then add two minimum adjustment rotation angle unit values ​​E to the deviation angle; If 30°≤<D<60°, then three minimum adjustment rotation angle unit values ​​E are added to the deviation angle; If the comprehensive angle deviation value is less than the preset deviation value, no adjustment is required; The balance center point is obtained according to the coordinates of the upper target point and the lower target point, and the comprehensive rotation direction of the rotation angle is determined by the distance values ​​between the balance center point and the upper and lower cameras respectively; Among them, (X T , Y T , Z T ) is the equilibrium center point, denoted as point F, (X T1 , Y T1 , Z T1 ) is the coordinate of the upper target point, (X T2 , Y T2 , Z T2 ) are the coordinates of the target point below.

8. The method for controlling the pan / tilt platform of an inspection drone according to claim 6, characterized in that: The method comprises: Compare the distance between the balance center point F and the upper camera point C and the distance between the balance center point F and the lower camera point D; If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is less than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated upward; If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is greater than the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the comprehensive rotation direction of the camera is rotated downward; If the distance between the coordinates of the balance center point F and the coordinates of the upper camera point C is equal to the distance between the coordinates of the balance center point F and the coordinates of the lower camera point D, the overall rotation direction of the camera is kept balanced.

9. The method for controlling a pan / tilt platform of an inspection drone according to claim 1, characterized in that: The angular velocity of the drone body [p, q, r] T The angular velocity of the gimbal caused by The relationship between When the target image center position deviation is [dy, dx] T In order to make it in the center of the camera image, the angle of the pan / tilt adjustment is Among them, p x , p y is the camera resolution, and the viewing angles are x ,λ y .

10. A pan-tilt control system for an inspection drone, characterized in that: The system comprises: 3D map navigation and path planning module: obtain 3D map information of the inspection area in real time, and plan the inspection path of the drone gimbal according to the 3D map information; Dual-camera inspection module: Two cameras are installed on the drone gimbal, one camera faces upward to inspect the target point above, and the other camera faces downward to inspect the target point below; PTZ inspection path control and image acquisition module: The flight control system controls the drone PTZ to travel along the planned inspection path, based on the target point coordinate information data, camera coordinate point information data and PTZ coordinate point information obtained from the images collected by the two camera devices above and below; PTZ space deviation value calculation module: obtain the upper and lower target space angle values ​​through the target point coordinates obtained by the upper and lower camera devices and the upper and lower camera coordinate points, obtain the upper and lower PTZ space angle values ​​through the upper and lower target point coordinate points and the PTZ coordinate points, obtain the upper space deviation value according to the upper target space angle value, the upper PTZ space angle value and the reference angle value, and obtain the lower space deviation value according to the lower target space angle value, the lower PTZ space angle value and the reference angle value; Comprehensive angle deviation judgment and adjustment module: The comprehensive angle deviation is obtained through the upper and lower spatial deviation values, and it is judged whether the comprehensive angle deviation is greater than the preset deviation value. If the comprehensive angle deviation is greater than the preset deviation value, the minimum rotation angle adjustment unit value is calculated for adjustment. If the comprehensive angle deviation is less than the preset deviation value, no adjustment is required. The comprehensive rotation direction of the rotation angle is judged by the balance center point obtained through the coordinates of the upper target point and the lower target point.

Images