UAV gimbal adjustment control system and method for traffic inspection
Remote sensing images are obtained through drones and adaptive adjustment of the gimbal using PID controllers and optimization algorithms, solving the problem that traditional gimbal adjustment methods cannot adapt to real-time changes, and achieving efficient and accurate traffic infrastructure inspections.
Patent Information
- Application Number
- CN202411251868.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-09
AI Technical Summary
When UAVs conduct traffic infrastructure inspections, traditional gimbal adjustment methods cannot adapt to real-time changes during flight, resulting in tilting and incomplete angles of the subjects, which are prone to missed and missed inspections, affecting data reliability and patrol efficiency .
The remote sensing image of the transportation infrastructure is obtained by using drones, and the features are identified through image correction and Mask-RCNN model, and the target deviation is calculated. The adaptive adjustment of the gimbal is achieved by using the PID controller, Ziegler-Nichols method and particle swarm optimization algorithm to achieve unmanned Adaptive adjustment of machine posture and target tracking.
Improve image quality and clarity, ensure the accuracy and stability of target tracking, and improve the reliability and accuracy of inspection data.
Smart Images

Figure CN118759833B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of unmanned aerial vehicle control, and in particular to an unmanned aerial vehicle gimbal adjustment control system and method for traffic inspection. Background Art
[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.
[0003] At present, drones are widely used to patrol along roads, bridges, tunnels and other transportation infrastructure to efficiently obtain images along the transportation infrastructure. The high-resolution images taken by drones can obtain the status of transportation infrastructure, detect potential damage or abnormalities in a timely manner, significantly improve the efficiency of transportation infrastructure inspections, and save time and human resources.
[0004] At present, when drones are used to inspect transportation infrastructure, the gimbal carried by the drone is a supporting device for installing cameras. Due to the influence of waypoint deviation, positioning error and external environment, the images captured during the flight of the drone may deviate. The angle of the traditionally installed gimbal is mostly fixed, that is, the observation direction is fixed, and the automatic adjustment of the gimbal and the adaptive adjustment of the drone's attitude cannot be performed, resulting in angle tilt and incomplete shooting of the object being photographed. The key features of the transportation infrastructure cannot be captured, and missed detection and false detection errors are very likely to occur, thus affecting the reliability of the data and inspection efficiency.
[0005] The traditional method of adjusting the gimbal of a drone is manual adjustment. The operator manually adjusts the gimbal angle in real time through a remote control. However, this manual adjustment method requires continuous monitoring and adjustment of the gimbal angle, which affects the accuracy of drone control. Existing intelligent adjustment methods include automatic adjustment of the gimbal of a drone based on preset waypoints. The principle is that when the drone flies to a certain waypoint, the gimbal angle is automatically adjusted. However, this method lacks flexibility and cannot adapt to the real-time changing shooting needs during flight. Summary of the invention
[0006] In order to solve the above problems, the present invention proposes a UAV gimbal adjustment control system and method for traffic inspection, which obtains remote sensing images of traffic infrastructure through UAVs, and performs adaptive adjustment of the UAV camera gimbal and the UAV attitude according to the deviation parameters of the targets in the image, to ensure that the shooting completely covers the inspection objects, realize intelligent flight attitude adjustment control and autonomous operation of the UAV, and ensure that the target focus point is located in the center of the screen.
[0007] According to some embodiments, the present disclosure adopts the following technical solutions:
[0008] The UAV gimbal adjustment control method for traffic inspection includes:
[0009] Acquire low-altitude remote sensing images of transportation infrastructure from drones and perform preprocessing;
[0010] Perform image correction on the preprocessed UAV low-altitude remote sensing image to obtain a corrected target image;
[0011] Input the target image into the Mask-RCNN model to identify the features along the transportation infrastructure and obtain the bounding box coordinates of the target object;
[0012] Based on the bounding box coordinates of the target object, calculate the center position of the target image and the center position of the target object in the target image;
[0013] The deviation between the center position of the target image and the center position of the target object in the target image is calculated. If the deviation exceeds the maximum allowable deviation threshold, the deviation is input into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation control of the UAV gimbal and accurate tracking of the target.
[0014] According to some embodiments, the present disclosure adopts the following technical solutions:
[0015] The UAV gimbal adjustment and control system for traffic inspection includes:
[0016] The data acquisition module is used to acquire low-altitude remote sensing images of traffic infrastructure from drones and perform preprocessing;
[0017] An image correction module is used to perform image correction on the pre-processed UAV low-altitude remote sensing image to obtain a corrected target image;
[0018] The target recognition module is used to input the target image into the Mask-RCNN model, identify the features along the transportation infrastructure, and obtain the bounding box coordinates of the target object;
[0019] A position determination module, used for calculating the center position of the target image and the center position of the target object in the target image based on the bounding box coordinates of the target object;
[0020] The control module is used to calculate the deviation between the center position of the target image and the center position of the target object in the target image. If the deviation exceeds the maximum allowable deviation threshold, the deviation amount is input into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation control of the UAV gimbal and accurate tracking of the target.
[0021] According to some embodiments, the present disclosure adopts the following technical solutions:
[0022] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the unmanned aerial vehicle gimbal adjustment and control method for traffic inspection is implemented.
[0023] According to some embodiments, the present disclosure adopts the following technical solutions:
[0024] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the unmanned aerial vehicle gimbal adjustment and control method for traffic inspection.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] The disclosed UAV gimbal adjustment and control method for traffic inspection uses a UAV to shoot low-altitude remote sensing images along the traffic infrastructure and perform pre-processing operations such as denoising and image enhancement, and performs image correction on the images acquired by the UAV, which significantly improves the quality and clarity of the images, making subsequent tracking of traffic infrastructure targets more accurate, thereby improving the reliability and accuracy of the inspection data.
[0027] The disclosed UAV gimbal adjustment and control method for traffic inspection takes into account that the images shot by the UAV may deviate due to factors such as waypoint deviation, positioning error or wind force, and introduces the Mask-RCNN deep learning model to accurately identify targets along the traffic infrastructure. After pre-training the model based on a public low-altitude remote sensing image dataset of traffic infrastructure on the Internet, the model is applied to the UAV traffic infrastructure inspection process to identify the features along the traffic infrastructure in the image in real time, and calculates the deviation between the target center position and the image center position based on the bounding box result of the model, thereby efficiently identifying and locating the target, ensuring that the UAV can track the traffic infrastructure targets in real time and provide accurate monitoring data.
[0028] The disclosed UAV gimbal adjustment and control method for traffic inspection uses a PID control formula to adaptively adjust the camera gimbal of the UAV according to the calculated target deviation, and performs UAV flight control. The PID controller is optimized by combining the Ziegler-Nichols method and the particle swarm optimization algorithm, and the PID controller parameters are adjusted according to the real-time flight attitude of the UAV and the image target performance. The parameters of the PID controller are continuously updated according to the adjusted feedback to realize closed-loop control of the UAV gimbal, thereby dynamically adjusting the gimbal position according to the target offset, ensuring that the focus point always remains in the center of the screen, and improving the stability and reliability of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.
[0030] Figure 1 It is a flowchart of the implementation process of the UAV gimbal adjustment control method for traffic inspection according to an embodiment of the present disclosure;
[0031] Figure 2 This is a control flow chart of the PID controller according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0032] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.
[0033] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.
[0034] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0035] Explanation of terms
[0036] 1. Canny edge detection algorithm is a standard algorithm widely used in edge detection. Its goal is to find an optimal edge detection solution or find the location with the strongest change in grayscale intensity in an image. The optimal edge detection is mainly evaluated by three criteria: low error rate, high localization and minimum response.
[0037] 2. Histogram equalization is a simple and effective image enhancement technology. It changes the grayscale of each pixel in the image by changing the image histogram. It is mainly used to enhance the contrast of images with a small dynamic range. Histogram equalization is a commonly used method to enhance image contrast. It expands the brightness range by redistributing the pixel values of the image.
[0038] Example 1
[0039] In one embodiment of the present disclosure, a method for adjusting and controlling the gimbal of a UAV for traffic inspection is provided. The remote sensing images along the traffic infrastructure are taken by the UAV, and pre-processing operations such as denoising and image enhancement as well as image correction are performed to further improve the image quality. The Mask-RCNN deep learning model is introduced to accurately identify the targets along the traffic infrastructure. After pre-training the model based on the low-altitude remote sensing image data set of traffic infrastructure publicly available on the Internet, the model is applied to the UAV railway inspection process to identify the features along the traffic infrastructure in the image in real time, and calculate the deviation between the target center position and the image center position. Based on the target offset, the PID control formula is used to adaptively adjust the camera gimbal of the UAV, and the UAV flight control is performed. According to the adjusted feedback, the parameters of the PID controller are continuously updated to realize the closed-loop control of the UAV gimbal, so as to ensure that the shooting points to be paid attention to during the flight are always kept on the screen, and the real-time tracking of the traffic infrastructure is realized. The specific implementation process is as follows:
[0040] Step 1: Acquire low-altitude remote sensing images of transportation infrastructure from drones and preprocess them in real time
[0041] The present disclosure uses drones to obtain low-altitude remote sensing images along the transportation infrastructure, and performs real-time preprocessing operations such as denoising and image enhancement. The specific implementation steps include:
[0042] S1: The drone flies along a pre-planned route, accurately tracks the route through a real-time positioning and navigation system to ensure coverage of the designated area, and uses a high-resolution camera to record images along the transportation infrastructure, which are then transmitted and saved in real time.
[0043] S2: Use filtering methods to denoise the acquired low-altitude remote sensing images. Apply Gaussian filtering to remove Gaussian noise in the image, and then apply median filtering to remove salt and pepper noise and speckle noise.
[0044] Specifically, the process of S2 denoising is as follows:
[0045] SS1: Evaluate the type and nature of noise in the image. Analyze the noise intensity by calculating the mean, standard deviation, and variance of the image. Then use Fourier transform to analyze the frequency domain information of the image to identify the frequency characteristics of the noise. Select the denoising filtering method based on the image noise type obtained. The noise types of low-altitude remote sensing images of transportation infrastructure include Gaussian noise, salt and pepper noise, and speckle noise.
[0046] SS2: Gaussian noise appears as the noise of the whole image in the image. The noise value conforms to the Gaussian distribution. Gaussian filtering method is used for denoising. The core idea of Gaussian filtering is to use a Gaussian kernel to perform convolution operation on the image, so as to achieve image smoothing and noise reduction effects.
[0047] The convolution kernel is a symmetrical bell-shaped function, and its mathematical expression is:
[0048]
[0049] in is the standard deviation of the Gaussian kernel and controls the smoothness of the kernel. The larger it is, the smoother the kernel is and the more obvious the filtering effect is.
[0050] Convolution is the process of multiplying the Gaussian kernel and the image pixel by pixel and summing them up. Its mathematical expression is:
[0051]
[0052] in, is the filtered image, is the original image.
[0053] SS3: Salt and pepper noise appears as black and white dots in the image, that is, random extreme brightness values. Median filtering is used for denoising. The principle of median filtering is to sort the bundles in a window for a given window size and then use the middle value as the output.
[0054] Mathematical formula of median filtering:
[0055]
[0056] in is the original signal, is the filtered signal, is the current location, is the window size.
[0057] SS4: Speckle noise appears as abnormal contrast or brightness in certain areas of the image. Median filtering can effectively remove speckle noise, while Gaussian filtering has a better effect in the case of mild noise.
[0058] S3: After removing the noise from low-altitude remote sensing images along the transportation infrastructure, the image clarity can be further improved through image enhancement processing.
[0059] Specifically, the steps of S3 image enhancement processing are as follows:
[0060] SS1: The histogram equalization method is used to improve the contrast of the image. The grayscale values with a large number of pixels in the image are widened, and the grayscale values with a small number of pixels are merged, thereby increasing the contrast, making the image clear, and achieving the purpose of enhancement.
[0061] SS2: Identify and enhance the edge details of transportation infrastructure based on the Canny edge detection algorithm, making the outline of the track clearer.
[0062] SS3: Based on the concept of histogram equalization, it is extended to each color channel to adjust the color balance of the image to compensate for color distortion under different lighting conditions.
[0063] Step 2: Perform image correction on the preprocessed UAV low-altitude remote sensing image to obtain the corrected target image. The specific implementation steps are as follows:
[0064] S1: Corrects the image geometric distortion caused by the lens.
[0065] Further preferably, the specific implementation steps of S1 image geometric distortion correction are as follows:
[0066] SS1: Use the checkerboard calibration plate and the camera on the drone to shoot multiple checkerboard images at different angles and positions to ensure that the images cover different angles and distances to improve the accuracy of calibration. Detect the corner points of the checkerboard in each image, and obtain the correspondence between the physical space coordinates and the image space coordinates through the corner point data of multiple calibration images to form the camera intrinsic parameter matrix.
[0067] Camera intrinsic parameter matrix:
[0068]
[0069] in, and is Axis and The focal length on axis, in pixels; and are the coordinates of the principal point of the image, usually the center of the image; is the scaling factor in the homogeneous coordinate representation.
[0070] Furthermore, the distortion coefficient includes radial distortion coefficient and tangential distortion coefficient, which are used to describe the image distortion caused by the physical properties of the camera lens. Radial distortion coefficient: (Sometimes only ); Tangential distortion coefficient: .
[0071] SS2: The camera’s intrinsic parameters and distortion coefficients are solved by minimizing the reprojection error. The reprojection error is the error between the image coordinates predicted from the physical coordinates by the camera model and the actual detected image coordinates.
[0072] Minimize the reprojection error:
[0073]
[0074] in, are the coordinates of the corner points in the actual image; are the predicted coordinates calculated by the camera model; is the number of all corner points in the image.
[0075] A nonlinear optimization algorithm is used to minimize this error to obtain the camera's intrinsic parameter matrix and distortion coefficients.
[0076] S2: Corrects image geometric deformation caused by shooting angle, camera tilt, etc.
[0077] Further preferably, the specific implementation steps of the S2 image geometric deformation correction are as follows:
[0078] SS1: Identify feature points in the image, such as the endpoints of the road centerline, and perform coordinate matching. Calculate the transformation matrix using these points.
[0079] SS2: Process the translation, rotation, scaling and other transformations in the image and perform affine transformation. Use the least squares method to calculate the affine transformation matrix as follows:
[0080]
[0081] in, are the transformed coordinates, are the transformed coordinates, is the affine transformation matrix.
[0082] SS3: Process the perspective distortion in the image and perform perspective transformation. Use the RANSAC algorithm to estimate the perspective transformation matrix as follows:
[0083]
[0084] in, are the transformed coordinates, are the transformed coordinates, is the perspective transformation matrix.
[0085] Step 3: Input the target image into the Mask-RCNN model to identify the features along the transportation infrastructure and obtain the bounding box coordinates of the target object;
[0086] Specifically, S1: First, the Mask-RCNN model is trained by using the publicly available image dataset of transportation infrastructure along the Internet to train the Mask-RCNN deep learning model to accurately identify the features along the transportation infrastructure in the image.
[0087] Specifically, the specific process of step S1 is as follows:
[0088] SS1: Based on the open low-altitude remote sensing image dataset along the transportation infrastructure on the Internet, including road pavement images with center lines, road edge line images, bridge structure images, tunnel inner wall images, etc., the Mask-RCNN model is used to train the model to identify the features along the transportation infrastructure.
[0089] S2: After obtaining low-altitude remote sensing images of transportation infrastructure during the flight of the drone and preprocessing them, the trained Mask-RCNN model is used to identify the features along the transportation infrastructure in the preprocessed images. Through this recognition model, the target bounding box is obtained for targets such as road center lines, road edge lines, bridge piers, bridge decks, bridge cables, tunnel inner walls, and tunnel ceilings in the transportation infrastructure. The bounding box is defined by the upper left corner. and lower right corner Coordinate definition of .
[0090] in:
[0091] : The horizontal coordinate of the upper left corner of the bounding box, indicating the leftmost position of the tracked object in the image;
[0092] : The vertical coordinate of the upper left corner of the bounding box, indicating the top position of the tracked target in the image;
[0093] : The horizontal coordinate of the lower right corner of the bounding box, indicating the rightmost position of the tracked object in the image;
[0094] : The vertical coordinate of the lower right corner of the bounding box, indicating the bottommost position of the tracked object in the image.
[0095] Step 4: Based on the bounding box coordinates of the target object, calculate the center position of the target image and the center position of the target object in the target image;
[0096] Specifically, S1: based on the bounding box results of the target objects along the traffic infrastructure identified by the Mask-RCNN model in step 3, calculate the position of the target object in the image.
[0097] Furthermore, the specific implementation process of step S1 is as follows:
[0098] SS1: Calculate the center position of the target based on the coordinates of the target object's bounding box .
[0099] The formula for calculating the center position of the target object is:
[0100]
[0101]
[0102] SS2: According to the width of the image and height Calculate the center position of the image .
[0103] The calculation formula for the center position of the image is:
[0104]
[0105]
[0106] The center position of the target object is .
[0107] S2: Calculate the deviation between the center position of the target object and the center position of the image , and determines whether the deviation exceeds the threshold that requires gimbal adjustment.
[0108] Furthermore, the specific steps of S2 are as follows:
[0109] SS1: Calculate the deviation between the target center position and the image center position based on the calculated results of the target object center position and the image center position .
[0110] Deviation calculation formula:
[0111]
[0112]
[0113] SS2: According to the accuracy requirements of the traffic infrastructure inspection task, set the maximum allowable deviation, i.e. the threshold . Determine the calculated deviation Whether it exceeds the threshold , if it exceeds the threshold, the gimbal will be adaptively adjusted.
[0114] Step 5: Calculate the deviation between the center position of the target image and the center position of the target object in the target image. If the deviation exceeds the maximum allowable deviation threshold, input the deviation into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation and control of the UAV gimbal and accurate tracking of the target.
[0115] Specifically, S1: calculated deviation Threshold exceeded , the offset distance and Input into the PID controller and use the adaptive PID control formula for PID control. PID controllers are divided into proportional (P) control, integral (I) control, and differential (D) control. Proportional control directly adjusts the control amount according to the current value of the deviation to solve static errors; integral control adjusts according to the historical value of the deviation to eliminate long-term errors; differential control adjusts according to the rate of change of the deviation to predict the future behavior of the system and reduce oscillations.
[0116] Adaptive PID control formula:
[0117]
[0118] in, is the step length, which is greater than 0 and is an integer; is the integral optimization coefficient, if >Threshold , ;on the contrary, ; is the proportionality coefficient; is the integration coefficient; is the differential coefficient.
[0119] By integrating the optimization coefficients, Greater than threshold When the integral term is removed from the controller, the offset distance is quickly shortened by only using the PD controller. , and then add the integral term to make the curve slowly eliminate the static error, thereby improving the operating speed and control accuracy of the PID controller; similarly, we can get control process.
[0120] Furthermore, the UAV flight control is performed according to the target deviation.
[0121] S1: UAV flight control systems usually include attitude control, position control, and speed control.
[0122] Further preferably, the specific implementation steps of S1 are as follows:
[0123] SS1: Use PID controller to achieve precise attitude control.
[0124] Control output The formula is as follows:
[0125]
[0126] in, is the input of the controller; It is a deviation; is the proportional gain; is the integral gain; is the differential gain.
[0127] Roll deviation control formula:
[0128]
[0129]
[0130] in, is the roll deviation, i.e. the difference between the desired roll angle and the actual roll angle; is the proportional gain of the roll angle; is the integral gain of the roll angle; is the differential gain of the roll angle.
[0131] Pitch deviation control formula:
[0132]
[0133]
[0134] in, is the pitch deviation, i.e. the difference between the desired pitch angle and the actual pitch angle; is the proportional gain of the pitch angle; is the integral gain of the pitch angle; is the differential gain of the pitch angle.
[0135] Course deviation control formula:
[0136]
[0137]
[0138] in, is the heading deviation, i.e. the difference between the desired heading angle and the actual heading angle; is the proportional gain of the heading angle; is the integral gain of the heading angle; is the differential gain of the heading angle.
[0139] SS2: Adjust the position of the drone by controlling its movement in three-dimensional space.
[0140]
[0141]
[0142]
[0143] in, , and It is the speed control signal of the UAV in the lateral, longitudinal and height directions. is the proportional gain; is the integral gain; is the differential gain; Respectively represent the position error of the drone in the lateral, longitudinal and height directions, that is, the difference between the expected position and the current actual position; It represents the integral of the error of the UAV in the lateral, longitudinal and height directions, that is, the accumulated error from the initial moment to the current moment; It represents the rate of change of the error of the UAV in the lateral, longitudinal and height directions over time.
[0144] Furthermore, the PID controller is adaptively adjusted.
[0145] Adaptive tuning is critical in drone flight control systems, where PID controller parameters are adjusted based on the system’s real-time performance and environmental conditions during the process of adjusting camera gimbal adjustments and drone flight control.
[0146] Further preferably, the specific steps for implementing the adaptive adjustment of the PID controller are as follows:
[0147] S1: Use the Ziegler-Nichols method to set the initial PID control parameters, i.e. the proportional gain , integral gain and the differential gain , providing a reasonable initial point for the gimbal control system so that the gimbal can begin to respond to the offset of the tracking target in the image.
[0148] S2: For each control cycle, the control signal is calculated according to the following steps.
[0149] Further preferably, the specific steps of S2 are as follows:
[0150] SS1: The calculated deviation between the target center position and the image center position Get real-time deviation .
[0151]
[0152] SS2: Calculate the proportional term.
[0153]
[0154] SS3: Update the integral term.
[0155]
[0156] in, is the sampling time interval, is the integral value at the previous moment.
[0157] SS4: Calculate the differential term.
[0158]
[0159] SS5: Calculate the control output and convert the calculated control signal Send to the gimbal to adjust the gimbal angle.
[0160]
[0161] S3: After sending the control signal to the UAV gimbal, the tracking target deviation in the low-altitude remote sensing image of the traffic infrastructure taken by the UAV after adjustment is monitored in real time, and whether the adjusted tracking target returns to the ideal position is analyzed, so as to determine whether the current PID parameters are appropriate.
[0162] S4: If the target offset is detected to be adjusted slowly or oscillates, an adaptive algorithm based on the particle swarm optimization algorithm is used to adaptively adjust the PID parameters.
[0163] Further preferably, the specific steps of S4 are as follows:
[0164] SS1: Each particle represents a PID controller parameter combination, that is, a set of The particle swarm contains multiple particles, each of which has its position (PID parameter) and velocity (the amount of change in PID parameter).
[0165] SS2: Randomly generate the PID parameter value of each particle within a reasonable range, initialize the speed of each particle, usually set to a small random value, initialize the best PID parameters of each particle in the iteration process and the best PID parameters of the entire particle swarm.
[0166] SS3: Define the fitness function that minimizes the error, calculate the offset of the railway line in the image, and design the fitness function to be the negative value or inverse of the total deviation.
[0167] SS4: Particle update speed, adjust the position of the particle according to the updated speed, that is, update the PID parameters. After the update, calculate the fitness of each particle, that is, evaluate the minimum railway line deviation of the current PID parameter combination. When the set maximum number of iterations is reached or the improvement of the fitness function is less than a certain threshold, stop the iteration and select the PID parameters of the global best particle as the final controller parameters.
[0168] S5: The adaptive algorithm to adjust the UAV gimbal is a closed-loop process. The initial posture and PID parameters of the gimbal are set. During the inspection of transportation infrastructure, low-altitude remote sensing images are obtained by drone photography and preprocessed to obtain the deviation between the target position and the center position of the image. The adjustment amount is calculated according to the PID control algorithm, and the gimbal is adjusted through the control system instructions. According to the feedback after adjustment, the parameters of the PID controller are continuously updated to maintain accurate tracking of transportation infrastructure targets and stable control of the gimbal.
[0169] Example 2
[0170] In one embodiment of the present disclosure, a UAV gimbal adjustment and control system for traffic inspection is provided, comprising:
[0171] The data acquisition module is used to acquire low-altitude remote sensing images of traffic infrastructure from drones and perform preprocessing;
[0172] An image correction module is used to perform image correction on the pre-processed UAV low-altitude remote sensing image to obtain a corrected target image;
[0173] The target recognition module is used to input the target image into the Mask-RCNN model, identify the features along the transportation infrastructure, and obtain the bounding box coordinates of the target object;
[0174] A position determination module, used for calculating the center position of the target image and the center position of the target object in the target image based on the bounding box coordinates of the target object;
[0175] The control module is used to calculate the deviation between the center position of the target image and the center position of the target object in the target image. If the deviation exceeds the maximum allowable deviation threshold, the deviation amount is input into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation control of the UAV gimbal and accurate tracking of the target.
[0176] Example 3
[0177] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the unmanned aerial vehicle gimbal adjustment and control method for traffic inspection is implemented.
[0178] Example 4
[0179] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the drone gimbal adjustment and control method for traffic inspection.
[0180] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0182] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.
Claims
1. UAV gimbal adjustment and control method for traffic inspection, It is characterized in that include: Acquire low-altitude remote sensing images of transportation infrastructure from drones and perform preprocessing; Perform image correction on the preprocessed UAV low-altitude remote sensing image to obtain a corrected target image; Input the target image into the Mask-RCNN model to identify the features along the transportation infrastructure and obtain the bounding box coordinates of the target object; Based on the bounding box coordinates of the target object, calculate the center position of the target image and the center position of the target object in the target image; The deviation between the center position of the target image and the center position of the target object in the target image is calculated. If the deviation exceeds the maximum allowable deviation threshold, the deviation is input into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation control of the UAV gimbal and accurate tracking of the target.
2. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection according to claim 1, It is characterized in that Acquire low-altitude remote sensing images of transportation infrastructure from drones and perform preprocessing, including: the drone flies along a pre-planned route, accurately tracks the route through a real-time positioning and navigation system to ensure coverage of the designated area, uses a high-resolution camera to acquire images along the transportation infrastructure, and transmits and saves them in real time; uses filtering methods to denoise the acquired low-altitude remote sensing images from drones, applies Gaussian filtering to remove Gaussian noise in the image, then applies median filtering to remove salt and pepper noise and speckle noise, and then uses image enhancement to improve image clarity.
3. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection according to claim 1, It is characterized in that The preprocessed UAV low-altitude remote sensing image is corrected to obtain the corrected target image, including correcting the image geometric distortion and image geometric deformation. The image geometric distortion is corrected by using a checkerboard calibration plate, and using the camera on the UAV to shoot multiple checkerboard images at different angles and positions. The corner points of the checkerboard are detected in each image, and the correspondence between the physical space coordinates and the image space coordinates is obtained through the corner point data of multiple calibration images to form an internal parameter matrix; the camera internal parameters and distortion coefficients are solved by minimizing the reprojection error.
4. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection as claimed in claim 3, It is characterized in that Correcting image geometric deformation includes: identifying feature points in the low-altitude remote sensing image of the UAV, calculating the transformation matrix through the feature points, performing affine transformation on the image, and using the least squares method to calculate the affine transformation matrix; performing perspective transformation on the image, and using the RANSAC algorithm to estimate the perspective transformation matrix.
5. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection according to claim 1, It is characterized in that The trained Mask-RCNN model is used to identify the features along the traffic infrastructure in the rectified target image, including the road centerline, road edge line, bridge piers, bridge deck, bridge cables, tunnel inner wall and tunnel roof target objects in the traffic infrastructure, and obtain the bounding box of the target object. The bounding box is composed of the upper left corner and lower right corner Coordinate definition of ; : The horizontal coordinate of the upper left corner of the bounding box, indicating the leftmost position of the tracked object in the image; : The vertical coordinate of the upper left corner of the bounding box, indicating the top position of the tracked target in the image; : The horizontal coordinate of the lower right corner of the bounding box, indicating the rightmost position of the tracked object in the image; : The vertical coordinate of the lower right corner of the bounding box, indicating the bottommost position of the tracked object in the image.
6. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection according to claim 1, It is characterized in that Based on the bounding box coordinates of the target object, the center position of the target image and the center position of the target object in the target image are calculated, including: The center position of the target object in the target image is: The center position of the target object is , according to the width of the target image W and height H Calculate the center position of the image ,for: 。 7. The method for adjusting and controlling the pan / tilt platform of a UAV for traffic inspection according to claim 1, It is characterized in that According to the calculated center position of the target object and the center position of the image, the deviation between the center position of the target object and the center position of the image is calculated, and the deviation is input into the PID controller. The initial PID control parameters, namely the proportional gain, integral gain and differential gain, are set using the Ziegler-Nichols method so that the gimbal can begin to respond to the offset of the tracking target in the image. For each control cycle, the control signal is calculated, and the parameters of the PID controller are continuously updated according to the adjusted feedback to maintain the accurate tracking of the traffic infrastructure targets and the stable control of the gimbal.
8. UAV gimbal adjustment and control system for traffic inspection, It is characterized in that include: The data acquisition module is used to acquire low-altitude remote sensing images of traffic infrastructure from drones and perform preprocessing; An image correction module is used to perform image correction on the pre-processed UAV low-altitude remote sensing image to obtain a corrected target image; The target recognition module is used to input the target image into the Mask-RCNN model, identify the features along the transportation infrastructure, and obtain the bounding box coordinates of the target object; A position determination module, used for calculating the center position of the target image and the center position of the target object in the target image based on the bounding box coordinates of the target object; The control module is used to calculate the deviation between the center position of the target image and the center position of the target object in the target image. If the deviation exceeds the maximum allowable deviation threshold, the deviation amount is input into the PID controller. According to the real-time performance and environmental conditions of the system, the Ziegler-Nichols method and particle swarm optimization algorithm are used to optimize the PID controller parameters to perform closed-loop regulation control of the UAV gimbal and accurate tracking of the target.
9. A non-transitory computer-readable storage medium, It is characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the unmanned aerial vehicle gimbal adjustment and control method for traffic inspection as described in any one of claims 1-7 is implemented.
10. An electronic device, It is characterized in that include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the drone gimbal adjustment and control method for traffic inspection as described in any one of claims 1-7.
