Unmanned aerial vehicle power inspection infrared image analysis method based on cloud computing and AI

Abstract

The application discloses a UAV power inspection infrared image analysis method based on cloud computing and AI, relates to the technical field of UAV power inspection, and comprises the following steps: acquiring an original infrared frame sequence collected by a UAV during cruising flight, acquiring attitude angle data and angular velocity data corresponding to each original infrared frame collection moment of a UAV holder, determining a motion blur direction parameter and a motion blur amplitude parameter based on the attitude angle data and the angular velocity data, performing frame motion compensation processing on the original infrared frame sequence based on the motion blur direction parameter and the motion blur amplitude parameter, generating a compensated infrared frame sequence, and sending the compensated infrared frame sequence and the corresponding attitude angle data to a ground cloud processing unit to perform interframe spatial consistency reconstruction and global temperature field splicing, thereby generating a wide-area infrared panoramic image. The compensated infrared frame sequence restores the definition of power equipment contours and temperature field distribution while maintaining the signal-to-noise ratio of the infrared image.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) power line inspection technology, specifically to an infrared image analysis method for UAV power line inspection based on cloud computing and AI. Background Technology

[0002] Drones equipped with infrared imaging modules are used to patrol and inspect power facilities such as transmission lines and substations. This is an important means of detecting overheating defects in equipment and ensuring the safe operation of the power grid. During the patrol flight, the drone gimbal drives the infrared imaging module to move continuously. Within the exposure duration of each frame of the image, the attitude change of the gimbal causes the image points formed by the object points in the target scene on the focal plane array to move, resulting in a trailing blur in the infrared image along the direction of motion. The trailing blur reduces the clarity of the outline of the power equipment and the spatial accuracy of the temperature field distribution, affecting the subsequent identification and positioning accuracy of abnormal heat points in the equipment.

[0003] Existing UAV infrared inspection image processing methods are mainly divided into two categories. The first category relies on airborne gimbal mechanical stabilization and shortening exposure time to suppress motion blur. This involves reducing image point displacement by improving the gimbal servo control precision and using shorter exposure durations. However, mechanical stabilization is limited by the response bandwidth of the gimbal actuator, and residual angular velocity still exists under UAV maneuvering or wind disturbance conditions. Shortening the exposure time leads to a decrease in the infrared image signal-to-noise ratio and a reduction in the detection capability of targets with weak temperature differences. The second category involves post-processing the infrared image sequence acquired at the ground station. This method uses image registration to stitch multiple frames of local infrared images into a wide-area panoramic image. This type of method lacks targeted compensation for motion blur in single-frame images. Directly involving blurred frames in the stitching process can lead to inaccurate feature point extraction and temperature field jumps at the stitching seams. Furthermore, ground station processing typically uses a single-resolution registration strategy, which is insufficient to adapt to image scale differences and perspective distortion caused by changes in flight altitude and viewing angle during UAV cruise. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In existing technologies, mechanical stabilization and shortening the exposure time cannot completely eliminate motion blur and sacrifice the signal-to-noise ratio. Post-processing does not provide targeted compensation for motion blur in a single frame, resulting in a decrease in registration accuracy. To address the shortcomings of existing technologies, this invention provides an infrared image analysis method for UAV power line inspection based on cloud computing and AI.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI, specifically including the following steps:

[0008] S1. Obtain the original infrared frame sequence collected by the infrared imaging core on the UAV during cruise flight, and obtain the attitude angle data and angular velocity data of the UAV gimbal at each original infrared frame acquisition time.

[0009] S2. Based on the attitude angle data and angular velocity data, determine the motion blur direction parameter and motion blur amplitude parameter corresponding to each original infrared frame;

[0010] S3. Based on the motion blur direction parameter and motion blur amplitude parameter, perform frame-by-frame motion compensation processing on the original infrared frame sequence to generate a compensated infrared frame sequence.

[0011] S4. The compensated infrared frame sequence and the corresponding attitude angle data are sent to the ground cloud processing unit, which performs inter-frame spatial consistency reconstruction and global temperature field stitching to generate a wide-area infrared panoramic image covering the inspection area.

[0012] Preferably, in S1, the infrared imaging core mounted on the UAV continuously collects infrared radiation intensity data at the first frame frequency to form an original infrared frame sequence. The exposure timing of the infrared imaging core is synchronized with the integration time of the readout circuit inside the core, so that all pixels of the focal plane array start integration at the same time and end integration at the same time.

[0013] The angular velocity and attitude angular components of the gimbal in the pitch, yaw and roll directions are obtained by the angular velocity sensing unit and attitude calculation unit attached to the drone gimbal at a second sampling frequency. The second sampling frequency is higher than the first frame frequency.

[0014] Before data acquisition begins, the UAV flight control system simultaneously sends synchronization pulse signals to the infrared imaging core and the gimbal attitude calculation unit. The infrared imaging core and the gimbal attitude calculation unit record the local clock count value at the moment the synchronization pulse signal is received as the zero point reference of their respective time axes.

[0015] For each frame in the original infrared frame sequence, the attitude angle data group and angular velocity data group with the closest timestamp are found in the attitude data time series according to the frame timestamp. If the frame timestamp is between two attitude sampling points, the attitude angle and angular velocity are linearly interpolated according to the time difference ratio to establish the correspondence between each original infrared frame and the corresponding attitude angle data and angular velocity data.

[0016] Preferably, in step S2, the exposure duration corresponding to the current original infrared frame is obtained, and the exposure duration is the length of the time window for the infrared imaging core focal plane array pixel to integrate infrared radiation.

[0017] Extract all pitch angular velocity samples from the attitude calculation unit between the start and end of the exposure, sum them up, and divide by the number of samples to obtain the average pitch angular velocity. Extract all yaw angular velocity samples, sum them up, and divide by the number of samples to obtain the average yaw angular velocity.

[0018] The first directional blur offset is obtained by multiplying the average pitch angular velocity by the exposure duration, and the second directional blur offset is obtained by multiplying the average yaw angular velocity by the exposure duration.

[0019] Calculate the square root of the sum of the squares of the first direction blur offset and the squares of the second direction blur offset, and use it as the motion blur amplitude parameter;

[0020] The arctangent of the ratio of the second-direction blur offset to the first-direction blur offset is calculated to obtain the motion blur direction parameter, which represents the angle between the image point movement trajectory and the first direction axis of the focal plane array.

[0021] Preferably, in step S3, for each original infrared frame, the angle value represented by the motion blur direction parameter is mapped to the row and column coordinate system of the focal plane array, and the main axis direction of the arrangement of the compensation filter kernel weight coefficients is determined as the filtering direction, and the filtering direction is consistent with the image point trailing extension direction.

[0022] The pixel displacement represented by the motion blur amplitude parameter is converted into the number of pixels. When the displacement is an integer number of pixels, the filter length is an integer number of pixels. When the displacement contains a fractional part, the filter length is a minimum odd number of pixels not less than the displacement.

[0023] A compensation filter kernel with filtering direction and filtering length is constructed. The center weight of the compensation filter kernel is positive and the edge weight is negative. The center of the compensation filter kernel is aligned with the current pixel position to be processed in the original infrared frame. The pixel position and several pixels before and after it in the filtering direction are extracted and multiplied by the corresponding weight coefficient of the compensation filter kernel and then accumulated. The accumulated sum is used as the new gray value after the reverse filtering operation. The part of the image edge region that exceeds the boundary is filled by pixel value copying or mirror expansion.

[0024] After performing inverse filtering on each pixel position in the original infrared frame, a compensated infrared frame is obtained. The compensated infrared frames are then arranged in the order of the acquisition time of the original infrared frames to form a compensated infrared frame sequence.

[0025] Preferably, in step S4, the ground-to-cloud processing unit receives the compensated infrared frame sequence and corresponding attitude angle data, and maps the pixel coordinates of each infrared frame to a unified spatial coordinate system based on the UAV's spatial position, gimbal attitude angle, infrared imaging core field of view parameters, and digital elevation model at the time of each frame acquisition.

[0026] For the first and second infrared frames with adjacent acquisition times, the spatial intersection of the ground coverage polygon in the same spatial coordinate system is obtained as the overlapping area. Within the overlapping area, the amplitude of the rate of change of pixel temperature field values ​​in the horizontal and vertical directions is calculated respectively. Pixels with a rate of change amplitude greater than the feature response judgment value are selected as the first temperature field feature point set and the second temperature field feature point set after being filtered by local non-maximum.

[0027] Extract the temperature field distribution pattern of the neighborhood around each feature point in the first temperature field feature point set. In the second temperature field feature point set, match the feature points with the most similar temperature field distribution patterns to establish several feature point pairs. Based on the feature point pairs, calculate the translation parameters of the second infrared frame relative to the first infrared frame in the row and column directions, the rotation parameters around the image center, and the scaling parameters, which together constitute the spatial transformation parameter group.

[0028] Based on the spatial transformation parameter set, scale, rotation and translation operations are performed on the second infrared frame to transform the second infrared frame into the coordinate space of the first infrared frame. In the overlapping area, the first weight is determined according to the distance of the pixel position from the center of the coverage area of ​​the first infrared frame and the second weight is determined according to the distance from the pixel position to the center of the coverage area of ​​the second infrared frame. The product of the temperature field value of the first infrared frame and the first weight and the product of the temperature field value of the second infrared frame and the second weight are added to obtain the fused temperature field value.

[0029] All infrared frames in the compensated infrared frame sequence are sequentially transformed and fused into a unified coordinate system to form a wide-area infrared panoramic image.

[0030] Based on the pre-established spatial location template of power equipment, the corresponding temperature field distribution sub-regions of each power equipment are cropped from the wide-area infrared panoramic image. The difference between the maximum temperature field value and the arithmetic mean of the temperature field in the sub-region is calculated as the first temperature difference. The maximum absolute value of the temperature field difference between adjacent pixels in the sub-region is calculated as the spatial change gradient.

[0031] When the first temperature difference exceeds the first temperature difference reference value and the spatial change gradient exceeds the first gradient reference value, the corresponding power equipment will be marked as abnormally heated equipment, and an equipment alarm message containing the equipment location coordinates and the abnormal temperature difference level will be generated.

[0032] (III) Beneficial Effects

[0033] This invention provides an infrared image analysis method for power line inspection using unmanned aerial vehicles (UAVs) based on cloud computing and AI, which has the following beneficial effects:

[0034] This invention acquires the original infrared frame sequence of the infrared imaging core and the attitude angle and angular velocity data of the gimbal, aligns the frame timestamp with the attitude timestamp, establishes a precise correspondence between the frame and the attitude, and calculates the first directional blur offset and the second directional blur offset based on the exposure duration and the average gimbal angular velocity during the exposure, and synthesizes them into motion blur direction parameters and amplitude parameters, thereby realizing a quantitative description of the motion blur characteristics of each frame image.

[0035] For each original infrared frame, a compensation filter kernel with matched filtering direction and filtering length is constructed and an inverse filtering operation is performed to generate a compensated infrared frame sequence, which restores the clarity of the outline of the power equipment and the temperature field distribution while maintaining the signal-to-noise ratio of the infrared image.

[0036] By extracting the temperature field distribution sub-regions of each power device from a wide-area infrared panoramic image, calculating the first temperature difference between the maximum and average temperature values ​​within the sub-region, as well as the spatial gradient of the temperature field values, and marking the device as abnormally overheating when the first temperature difference exceeds the first temperature difference reference value and the spatial gradient exceeds the first gradient reference value, the device is judged by combining the two dimensions of temperature difference amplitude and spatial gradient, which effectively distinguishes between the overall temperature rise of the device and local defect overheating, and improves the accuracy of abnormal overheating alarm. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1 This invention provides a method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI, comprising the following steps:

[0040] S1. Obtain the original infrared frame sequence collected by the infrared imaging core on the UAV during cruise flight, and obtain the attitude angle data and angular velocity data of the UAV gimbal at each original infrared frame acquisition time.

[0041] Furthermore, in S1, the infrared imaging core mounted on the UAV continuously collects infrared radiation intensity data at the first frame frequency to form an original infrared frame sequence. The exposure timing of the infrared imaging core is synchronized with the integration time of the readout circuit inside the core, so that all pixels of the focal plane array start integration at the same time and end integration at the same time.

[0042] The angular velocity and attitude angular components of the gimbal in the pitch, yaw and roll directions are obtained by the angular velocity sensing unit and attitude calculation unit attached to the drone gimbal at a second sampling frequency. The second sampling frequency is higher than the first frame frequency.

[0043] Before data acquisition begins, the UAV flight control system simultaneously sends synchronization pulse signals to the infrared imaging core and the gimbal attitude calculation unit. The infrared imaging core and the gimbal attitude calculation unit record the local clock count value at the moment the synchronization pulse signal is received as the zero point reference of their respective time axes.

[0044] For each frame in the original infrared frame sequence, the attitude angle data group and angular velocity data group with the closest timestamp are found in the attitude data time series according to the frame timestamp. If the frame timestamp is between two attitude sampling points, the attitude angle and angular velocity are linearly interpolated according to the time difference ratio to establish the correspondence between each original infrared frame and the corresponding attitude angle data and angular velocity data.

[0045] It should be noted that an infrared imaging core is a detection device that converts the infrared radiation emitted by objects in a target scene into electrical signals and forms image data. Its core component is an infrared focal plane array, which is composed of multiple infrared sensitive pixels arranged in rows and columns. Each infrared sensitive pixel independently senses the intensity of infrared radiation projected onto its photosensitive surface and outputs a voltage signal that corresponds to the intensity of radiation.

[0046] The exposure sequence of an infrared imaging module refers to the time window during which each pixel in the focal plane array integrates infrared radiation. Within this time window, the pixel continuously accumulates infrared radiation energy. The longer the integration time, the more energy is accumulated and the stronger the output signal. The readout circuit inside the module is responsible for reading out the charge accumulated by each pixel in sequence after the exposure is completed and converting it into a digital quantity.

[0047] Synchronization of the exposure timing with the integration time of the readout circuit inside the camera mechanism means that the start and end times of exposure for each frame of the image are driven by a unified timing control unit inside the camera mechanism, so that all pixels in the focal plane array start integration at the same time and end integration at the same time.

[0048] The first frame frequency refers to the number of infrared frames acquired and output by the infrared imaging core per unit time. Its value is determined according to the drone's cruising speed and the spatial resolution requirements of the inspected target. The core periodically triggers exposure and readout operations according to the frame interval determined by the reciprocal of the first frame frequency, continuously outputting frame after frame of infrared image data, which are arranged in the order of output time to form the original infrared frame sequence.

[0049] Each frame of the original infrared frame sequence consists of multiple pixels. The value of each pixel represents the infrared radiation intensity at the corresponding spatial location. After the radiation intensity is converted by the internal radiation calibration parameters of the mechanism, it can be mapped to the temperature value at the corresponding location.

[0050] The angular velocity sensing unit and attitude calculation unit attached to the UAV gimbal are used to acquire the angular velocity components and attitude angular components of the gimbal in the pitch, yaw and roll directions at a second sampling frequency. The UAV gimbal is a mechanical suspension device that connects the UAV fuselage and the infrared imaging core. It contains a drive motor and an angle sensor to isolate the vibration of the UAV fuselage and control the pointing direction of the infrared imaging core. The angular velocity sensing unit is usually composed of a three-axis microelectromechanical gyroscope. The three-axis microelectromechanical gyroscope is sensitive to the instantaneous angular velocity of the gimbal rotating around the pitch axis, yaw axis and roll axis respectively. When the attitude of the gimbal changes during the cruise flight of the UAV, the mass block inside the gyroscope will generate displacement or stress change proportional to the rotation angular velocity due to the Coriolis effect. This change is converted into a voltage signal, and after analog-to-digital conversion, the pitch angular velocity component, yaw angular velocity component and roll angular velocity component are output.

[0051] The attitude calculation unit receives angular velocity data output from the three-axis gyroscope. By integrating the angular velocity over time and combining it with the gravity direction reference provided by the accelerometer, it calculates the pitch, yaw, and roll attitude angles of the gimbal relative to the reference coordinate system. The specific calculation logic steps are as follows:

[0052] The attitude calculation unit reads the instantaneous angular velocity values ​​of the gimbal around the pitch, yaw and roll axes from the three-axis gyroscope at a fixed calculation cycle. The three angular velocity values ​​correspond to the speed of rotation of the gimbal in the pitch, yaw and roll directions, respectively. The attitude calculation unit reads the acceleration measurement values ​​of the gimbal in three mutually perpendicular directions from the three-axis accelerometer. The accelerometer senses the magnitude of the specific force on the gimbal. When the gimbal is stationary or in uniform motion, the accelerometer measurement mainly reflects the projection components of gravitational acceleration on the three axes of the gimbal coordinate system.

[0053] Using the gimbal attitude angle determined at the end of the previous calculation cycle as the initial attitude, the pitch angular velocity value read in the current calculation cycle is multiplied by the calculation cycle duration to obtain the gimbal rotation angle increment around the pitch axis in this cycle. The yaw angular velocity value is multiplied by the calculation cycle duration to obtain the rotation angle increment around the yaw axis. The roll angular velocity value is multiplied by the calculation cycle duration to obtain the rotation angle increment around the roll axis.

[0054] The three rotation angle increments are added to the pitch, yaw, and roll attitude angles of the previous cycle, respectively, to obtain the pitch, yaw, and roll angle predictions calculated solely from the angular velocity integral. Since the gyroscope has zero bias error and integral drift, relying solely on the angular velocity integral for a long time will cause the attitude angles to gradually deviate from the true values. Therefore, it is necessary to introduce the gravity direction reference provided by the accelerometer for correction.

[0055] Based on the predicted pitch and roll angles, the theoretical projection components of the gravitational acceleration vector in the three axes of the gimbal coordinate system under the current predicted attitude are calculated. It is assumed that the direction of the gravitational acceleration vector is always perpendicular to the reference horizontal plane and points towards the center of the earth, and its magnitude is a fixed gravity value. Based on the pitch and roll tilt of the gimbal relative to the reference horizontal plane, the distribution ratio of the gravity vector on the front-back axis, left-right axis, and up-down axis of the gimbal coordinate system is determined, thereby obtaining the theoretical three-axis acceleration components.

[0056] The calculated theoretical triaxial acceleration components are compared with the triaxial acceleration components actually measured by the accelerometer. The first deviation in the front-to-back axis direction, the second deviation in the left-to-right axis direction, and the third deviation in the up-down axis direction are calculated. The first and second deviations mainly reflect the errors between the predicted pitch and roll angles and the actual attitude. The third deviation can be used to help determine whether the gimbal is in a state of violent motion.

[0057] The pitch angle prediction and roll angle prediction are corrected using the first deviation and the second deviation. The first deviation is multiplied by the first correction coefficient to obtain the pitch angle correction. The pitch angle prediction is then added to the pitch angle correction to obtain the corrected pitch attitude angle. The second deviation is multiplied by the second correction coefficient to obtain the roll angle correction. The roll angle prediction is then added to the roll angle correction to obtain the corrected roll attitude angle.

[0058] The first and second correction factors determine the strength of the accelerometer's correction effect. When the gimbal moves relatively smoothly, the accelerometer is less affected by motion interference, so a larger correction factor is used to accelerate error convergence. When the gimbal is in a large dynamic motion, the accelerometer is more affected by centrifugal force and linear acceleration interference, so a smaller correction factor is used to reduce the risk of incorrect correction.

[0059] For the yaw attitude angle, since the gravitational acceleration vector does not provide a directional reference in the yaw direction, the accelerometer cannot directly correct the integral drift of the yaw angle. When the gimbal is stationary or nearly stationary, if both the pitch angular velocity and roll angular velocity are less than the stationary angular velocity judgment value, the integral increment of the yaw angular velocity is set to zero to keep the yaw attitude angle unchanged, so as to suppress the yaw drift in the stationary state. When the gimbal is in motion, the yaw attitude angle is updated only by the integral of the yaw angular velocity. At the same time, the attitude calculation unit periodically receives the absolute heading calibration value provided by the external heading reference device. When the calibration value is received, the yaw attitude angle is directly replaced with the calibration value or the yaw attitude angle is gradually pulled and adjusted in the direction of the calibration value.

[0060] Finally, the corrected pitch, yaw, and roll attitude angles are output as the attitude calculation results for this cycle. At the same time, the above three attitude angles are stored in the internal state register as the initial attitude for angular velocity integration in the next calculation cycle. The above reading, integration, prediction, comparison, correction, and output steps are repeated according to a fixed calculation cycle to continuously output the real-time attitude angle sequence of the gimbal relative to the reference coordinate system.

[0061] Each attitude angle data in the attitude angle sequence is accompanied by a corresponding timestamp. The timestamp is taken from the clock counter inside the attitude calculation unit and records the precise time of the attitude angle output, providing a unified time reference for subsequent time alignment with the infrared imaging frame.

[0062] The second sampling frequency refers to the number of times the angular velocity sensing unit and attitude calculation unit output data per unit time. The value of the second sampling frequency must be higher than the first frame frequency of the infrared imaging core to ensure that the dynamic change details of the gimbal attitude can be captured within the exposure duration of each frame of infrared image. Both the angular velocity component and the attitude angular component are accompanied by corresponding timestamps.

[0063] After acquiring the raw infrared frame sequence and attitude angle data, the timestamp of each raw infrared frame is aligned with the timestamp of the attitude angle data to establish the correspondence between frames and attitudes. The timestamp of the raw infrared frame refers to the precise moment recorded by the infrared imaging core when it completes the exposure or readout operation of each frame image.

[0064] The timestamp of attitude angle data refers to the precise moment recorded by the attitude calculation unit when outputting each set of attitude angle components and angular velocity components. Since the clock unit of the infrared imaging core and the gimbal clock unit may use independent clock sources, there is an initial clock deviation and clock drift between the two. It is necessary to unify the two time series to the same time reference through time alignment operation. The specific method of alignment operation is as follows: before the data acquisition begins, the UAV flight control system sends a synchronization pulse signal to the infrared imaging core and the gimbal attitude calculation unit at the same time. When the two units receive the synchronization pulse signal, they each record the count value of their local clock as the zero point reference of their respective time axis.

[0065] During subsequent data acquisition, the infrared imaging core and the gimbal attitude calculation unit each record timestamps according to their local clocks. After receiving the two data streams, the data processing unit converts the timestamps of the gimbal attitude data into equivalent timestamps on the time axis of the infrared imaging core based on the zero-point reference relationship established by the synchronization pulse signal. For each original infrared frame, based on its timestamp value, the unit searches for the attitude angle data group and angular velocity data group with the closest timestamps in the converted attitude data time series. This group of data is used as the attitude angle data and angular velocity data corresponding to the original infrared frame, thereby establishing the correspondence between each frame and the attitude.

[0066] If the timestamp of a certain original infrared frame falls exactly between two attitude sampling points, then the attitude angle and angular velocity are linearly interpolated according to the ratio of the difference between the timestamps of the two sampling points and the frame timestamp to obtain the attitude and angular velocity estimates at the corresponding moment of the frame.

[0067] S2. Based on the attitude angle data and angular velocity data, determine the motion blur direction parameter and motion blur amplitude parameter corresponding to each original infrared frame;

[0068] Furthermore, in S2, the exposure duration corresponding to the current original infrared frame is obtained, and the exposure duration is the length of the time window for the infrared imaging core focal plane array pixel to integrate infrared radiation.

[0069] Extract all pitch angular velocity samples from the attitude calculation unit between the start and end of the exposure, sum them up, and divide by the number of samples to obtain the average pitch angular velocity. Extract all yaw angular velocity samples, sum them up, and divide by the number of samples to obtain the average yaw angular velocity.

[0070] The first directional blur offset is obtained by multiplying the average pitch angular velocity by the exposure duration, and the second directional blur offset is obtained by multiplying the average yaw angular velocity by the exposure duration.

[0071] Calculate the square root of the sum of the squares of the first direction blur offset and the squares of the second direction blur offset, and use it as the motion blur amplitude parameter;

[0072] The arctangent of the ratio of the second-direction blur offset to the first-direction blur offset is calculated to obtain the motion blur direction parameter, which represents the angle between the image point movement trajectory and the first direction axis of the focal plane array.

[0073] It should be noted that motion blur is caused by the gimbal causing the infrared imaging core to change its spatial attitude during the exposure duration, resulting in the continuous movement of the corresponding image point position of the same object point on the focal plane array, thus forming a trailing trace along the direction of motion in a frame image.

[0074] The direction and magnitude of motion blur are determined by the magnitude and direction of the gimbal angular velocity during exposure. The greater the angular velocity and the longer the exposure duration, the greater the pixel displacement and the more severe the blur. Determining the motion blur direction parameter and motion blur magnitude parameter provides accurate blur parameters for subsequent frame-by-frame motion compensation processing, enabling the compensation filter to specifically weaken the pixel trailing caused by the gimbal movement and restore the image clarity. The determination process begins by obtaining the exposure duration corresponding to the current original infrared frame. The exposure duration refers to the length of the time window for each pixel in the focal plane array of the infrared imaging core to integrate infrared radiation. This parameter is automatically or fixedly set by the exposure control unit of the core according to the infrared radiation intensity of the target scene and the sensitivity of the core, and can be read from the working parameter register of the core.

[0075] In addition to obtaining the exposure duration, it is also necessary to obtain the average angular velocity of the gimbal in the pitch and yaw directions during the current raw infrared frame acquisition period. The current raw infrared frame acquisition period refers to the time interval from the start time of the exposure to the end time of the exposure. Since the angular velocity of the gimbal in the pitch and yaw directions may not be constant during the exposure period, but fluctuates slightly with time, it is necessary to calculate the average angular velocity within the time interval to represent the overall situation of the gimbal movement during the exposure period.

[0076] The average angular velocity is calculated as follows: extract all pitch angular velocity samples output by the attitude calculation unit between the start and end of the exposure, sum these pitch angular velocity samples, and then divide by the number of samples. The quotient is the average pitch angular velocity. In the same way, extract all yaw angular velocity samples within the time interval, sum them, and then divide by the number of samples to obtain the average yaw angular velocity. Since the angular velocity in the roll direction mainly causes the rotation of the image point around the optical axis, its contribution to the translational trailing of the image point is relatively small. Therefore, it can not be considered as a major factor in the calculation of motion blur direction and amplitude parameters.

[0077] After obtaining the average pitch angular velocity and the average yaw angular velocity, the average pitch angular velocity is multiplied by the exposure duration. The product is used as the first directional blur offset. The first directional blur offset represents the displacement of the object point in the target scene on the focal plane array along the pitch direction due to the rotation of the gimbal around the pitch axis during the exposure duration. The average pitch angular velocity is usually expressed in radians per second or degrees per second. The two are multiplied to obtain the angular displacement. The angular displacement is multiplied by the equivalent focal length of the imaging system and converted into the linear displacement on the focal plane, or the angular displacement is directly used as the angle of the first directional blur offset.

[0078] The average yaw angular velocity is multiplied by the exposure duration, and the product is used as the second-direction blur offset. The second-direction blur offset represents the displacement of the image point along the yaw direction due to the rotation of the gimbal around the yaw axis during the exposure duration. When the average pitch angular velocity is positive, it means that the gimbal tilts upward during the exposure and the image point moves downward towards the focal plane array. When the average pitch angular velocity is negative, it means that the gimbal tilts downward and the image point moves upward. When the average yaw angular velocity is positive, it means that the gimbal yaws to the left and the image point moves to the right. When the average yaw angular velocity is negative, it means that the gimbal yaws to the right and the image point moves to the left.

[0079] After obtaining the first directional blur offset and the second directional blur offset, the first directional blur offset and the second directional blur offset are combined to obtain the motion blur direction parameter and the motion blur amplitude parameter. The first directional blur offset and the second directional blur offset represent the displacement components of the image point in two orthogonal directions of the focal plane array, respectively. The specific method of the combination operation is as follows: the first directional blur offset is used as the first right-angled side, and the second directional blur offset is used as the second right-angled side to construct a right triangle. The motion blur amplitude parameter is equal to the length of the hypotenuse of the right triangle. The calculation method is as follows: calculate the sum of the square of the first directional blur offset and the square of the second directional blur offset, and take the square root of the sum. The result is the motion blur amplitude parameter.

[0080] The larger the motion blur amplitude parameter, the more severe the motion blur and the longer the trailing marks in the image. The motion blur direction parameter is determined by the ratio of the first direction blur offset and the second direction blur offset. Specifically, the arctangent of the ratio of the second direction blur offset to the first direction blur offset is calculated, and the resulting direction angle is the motion blur direction parameter. This angle represents the angle between the image point's movement trajectory and the first direction axis of the focal plane array. The motion blur direction parameter indicates the extension direction of the trailing marks in the image. The motion blur direction parameter and motion blur amplitude parameter corresponding to each original infrared frame are associated and stored with the image data of that frame to form a frame blur parameter pair.

[0081] S3. Based on the motion blur direction parameter and motion blur amplitude parameter, perform frame-by-frame motion compensation processing on the original infrared frame sequence to generate a compensated infrared frame sequence.

[0082] Furthermore, in S3, for each original infrared frame, the angle value represented by the motion blur direction parameter is mapped to the row and column coordinate system of the focal plane array, and the main axis direction of the arrangement of the compensation filter kernel weight coefficients is determined as the filtering direction, and the filtering direction is consistent with the image point trailing extension direction.

[0083] The pixel displacement represented by the motion blur amplitude parameter is converted into the number of pixels. When the displacement is an integer number of pixels, the filter length is an integer number of pixels. When the displacement contains a fractional part, the filter length is a minimum odd number of pixels not less than the displacement.

[0084] A compensation filter kernel with filtering direction and filtering length is constructed. The center weight of the compensation filter kernel is positive and the edge weight is negative. The center of the compensation filter kernel is aligned with the current pixel position to be processed in the original infrared frame. The pixel position and several pixels before and after it in the filtering direction are extracted and multiplied by the corresponding weight coefficient of the compensation filter kernel and then accumulated. The accumulated sum is used as the new gray value after the reverse filtering operation. The part of the image edge region that exceeds the boundary is filled by pixel value copying or mirror expansion.

[0085] After performing inverse filtering on each pixel position in the original infrared frame, a compensated infrared frame is obtained. The compensated infrared frames are then arranged in the order of the acquisition time of the original infrared frames to form a compensated infrared frame sequence.

[0086] It should be noted that when the compensation filter kernel performs the inverse filtering operation, in order to prevent the compensated pixel grayscale value from overflowing or generating a negative value due to the accumulation of negative weights, the amplitude clamping processing of the accumulated sum needs to be performed. If the accumulated sum exceeds the maximum grayscale value that the original bit depth of the infrared imaging core can express, the new grayscale value is truncated to the maximum grayscale value; if the accumulated sum is less than zero, the new grayscale value is set to zero.

[0087] For image edge regions, in addition to using pixel value duplication or mirror expansion for padding, an edge smoothing transition strategy can also be adopted. That is, in areas less than half the filter length away from the image edge, the edge weights of the compensation filter kernel are gradually attenuated according to the distance of the pixels from the edge, so as to suppress artifact ring bands caused by missing data at the edge.

[0088] It should be noted that for each original infrared frame, the first step of frame-by-frame motion compensation processing is to determine the filtering direction of the compensation filter kernel based on the motion blur direction parameter corresponding to that frame, and to determine the filtering length of the compensation filter kernel based on the motion blur amplitude parameter. The compensation filter kernel is a numerical array composed of multiple weight coefficients arranged in one-dimensional or two-dimensional order, used to perform neighborhood weighted calculation on the gray values ​​of each pixel in the original infrared frame. The filtering direction refers to the main axis direction of the weight coefficients in the compensation filter kernel. The main axis direction should be consistent with the extension direction of the image point trail, that is, the same as the direction of the image point movement trajectory indicated by the motion blur direction parameter.

[0089] The filtering direction is determined as follows: the angle value represented by the motion blur direction parameter is mapped to the row and column coordinate system of the focal plane array. If the direction angle is zero degrees or close to zero degrees, it means that the trailing extends along the row direction of the image. At this time, the weight coefficients of the compensation filter kernel are arranged along the row direction. If the direction angle is right angle or close to right angle, it means that the trailing extends along the column direction of the image. The weight coefficients of the compensation filter kernel are arranged along the column direction. If the direction angle is oblique angle, the weight coefficients of the compensation filter kernel are arranged along the corresponding oblique direction. Oblique filtering can be achieved by distributing the projection length of the filter kernel in the row and column directions according to the angle ratio.

[0090] The filter length refers to the number of weight coefficients contained in the compensation filter kernel. The filter length is determined as follows: the pixel displacement represented by the motion blur amplitude parameter is converted into the number of pixels. If the displacement is exactly an integer number of pixels, the filter length is taken as that integer number of pixel values. If the displacement contains a decimal part, the filter length is taken as the smallest odd number of pixel values ​​not less than that displacement, so as to ensure that the filter kernel has a clear center pixel position. The larger the filter length, the wider the neighborhood range involved in the compensation operation, and the stronger the compensation ability for long trails.

[0091] After determining the filtering direction and length of the compensation filter kernel, the original infrared frame is subjected to inverse filtering using the compensation filter kernel with the filtering direction and length. The basic principle of the inverse filtering operation is: the original infrared frame can be regarded as the result of the convolution operation between a clear image and a motion blur function. The motion blur function is determined by the direction and amplitude of the gimbal movement and is manifested as the diffusion of pixel energy along a specific direction and a specific length. The inverse filtering operation constructs a compensation filter kernel that has the opposite effect to the motion blur function. The compensation filter kernel is then subjected to spatial domain convolution operation with the original infrared frame to re-converge the dispersed pixel energy. The specific execution process is as follows: the center of the compensation filter kernel is aligned with the current pixel position to be processed in the original infrared frame. The gray values ​​of the pixel position and several pixels before and after it in the filtering direction are extracted. The extracted pixel gray values ​​are multiplied one by one with the weight coefficients of the corresponding positions in the compensation filter kernel. All the product results are then summed. The sum is used as the new gray value of the pixel position after the inverse filtering operation.

[0092] The weight coefficients in the compensation filter kernel are designed based on the physical characteristics of motion blur. Typically, the center weights are positive and larger, while the edge weights are negative and smaller. The combination of positive and negative weights achieves inverse compensation for diffused energy. The above inverse filtering operation is performed on each pixel position in the original infrared frame. If the filter kernel exceeds the image boundary in the image edge region, the required pixels are filled in by copying the boundary pixel values ​​or mirroring. After completing the inverse filtering operation for all pixels, a motion-compensated infrared image is obtained, in which the pixel trailing caused by the gimbal movement is effectively reduced.

[0093] After completing the inverse filtering operation for each original infrared frame, the infrared frames that have undergone the inverse filtering operation are arranged according to the original acquisition order to form a compensated infrared frame sequence. The original acquisition order refers to the chronological order in which each frame in the original infrared frame sequence was generated. This order reflects the spatial coverage relationship of the UAV continuously scanning the inspection area during its flight along the cruise route. The arrangement according to the original acquisition order is as follows: the first compensated infrared frame obtained after the inverse filtering operation of the first original infrared frame is placed at the beginning of the sequence, the second compensated infrared frame obtained after the inverse filtering operation of the second original infrared frame is placed at the second position, and so on, until all original infrared frames have been compensated and arranged in order. The frame sequence formed after the arrangement is completed is the compensated infrared frame sequence. The infrared image of each frame in this sequence has undergone motion compensation processing for its own motion blur parameters, and the time interval and spatial adjacency relationship between frames are consistent with the original infrared frame sequence.

[0094] S4. The compensated infrared frame sequence and the corresponding attitude angle data are sent to the ground cloud processing unit, which performs inter-frame spatial consistency reconstruction and global temperature field stitching to generate a wide-area infrared panoramic image covering the inspection area.

[0095] Furthermore, in S4, the ground cloud processing unit receives the compensated infrared frame sequence and corresponding attitude angle data, and maps the pixel coordinates of each infrared frame to a unified spatial coordinate system based on the UAV's spatial position, gimbal attitude angle, infrared imaging core field of view parameters and digital elevation model at the time of each frame acquisition.

[0096] For the first and second infrared frames with adjacent acquisition times, the spatial intersection of the ground coverage polygon in the same spatial coordinate system is obtained as the overlapping area. Within the overlapping area, the amplitude of the rate of change of pixel temperature field values ​​in the horizontal and vertical directions is calculated respectively. Pixels with a rate of change amplitude greater than the feature response judgment value are selected as the first temperature field feature point set and the second temperature field feature point set after being filtered by local non-maximum.

[0097] Extract the temperature field distribution pattern of the neighborhood around each feature point in the first temperature field feature point set. In the second temperature field feature point set, match the feature points with the most similar temperature field distribution patterns to establish several feature point pairs. Based on the feature point pairs, calculate the translation parameters of the second infrared frame relative to the first infrared frame in the row and column directions, the rotation parameters around the image center, and the scaling parameters, which together constitute the spatial transformation parameter group.

[0098] Based on the spatial transformation parameter set, scale, rotation and translation operations are performed on the second infrared frame to transform the second infrared frame into the coordinate space of the first infrared frame. In the overlapping area, the first weight is determined according to the distance of the pixel position from the center of the coverage area of ​​the first infrared frame and the second weight is determined according to the distance from the pixel position to the center of the coverage area of ​​the second infrared frame. The product of the temperature field value of the first infrared frame and the first weight and the product of the temperature field value of the second infrared frame and the second weight are added to obtain the fused temperature field value.

[0099] All infrared frames in the compensated infrared frame sequence are sequentially transformed and fused into a unified coordinate system to form a wide-area infrared panoramic image.

[0100] Based on the pre-established spatial location template of power equipment, the corresponding temperature field distribution sub-regions of each power equipment are cropped from the wide-area infrared panoramic image. The difference between the maximum temperature field value and the arithmetic mean of the temperature field in the sub-region is calculated as the first temperature difference. The maximum absolute value of the temperature field difference between adjacent pixels in the sub-region is calculated as the spatial change gradient.

[0101] When the first temperature difference exceeds the first temperature difference reference value and the spatial change gradient exceeds the first gradient reference value, the corresponding power equipment will be marked as abnormally heated equipment, and an equipment alarm message containing the equipment location coordinates and the abnormal temperature difference level will be generated.

[0102] It should be noted that a unified spatial coordinate system refers to a three-dimensional geographic coordinate system established with a fixed reference point in the ground inspection area as the origin, due north as the first horizontal axis, due east as the second horizontal axis, and the vertical upward direction as the third axis. The purpose of the mapping operation is to convert the coordinates of each pixel in the infrared image into ground coordinates in the geographic coordinate system based on the gimbal's pointing attitude and the UAV's own spatial position when each frame of infrared image is acquired. This allows the ground areas covered by different frames of infrared images to be correlated with each other in the same spatial coordinate system. The specific method of the mapping operation is as follows: obtain the longitude coordinates, latitude coordinates, and flight altitude of the UAV at the time of each frame acquisition. Combine the pitch and yaw attitude angles of the gimbal. Based on the field of view parameters and image resolution parameters of the infrared imaging core, establish the projection transformation relationship from the row and column coordinates of the infrared image pixels to the ground spatial coordinates. For each pixel in the infrared image, calculate the projection angle of the corresponding line of sight direction vector in the horizontal and vertical planes. Intersect the line of sight with the ground in the digital elevation model. The coordinates of the intersection point are the ground spatial coordinates corresponding to the pixel. The digital elevation model is pre-stored in the ground cloud processing unit and records the elevation values ​​of the surface of the inspection area in the form of a regular grid.

[0103] After each infrared frame is mapped to a unified spatial coordinate system, the ground-cloud processing unit extracts the overlapping area between adjacent infrared frames. Adjacent infrared frames refer to two infrared images that are acquired in adjacent time in the compensated infrared frame sequence. Since the UAV flies continuously along the cruise route and the infrared imaging core acquires data at a fixed frame rate, there is some overlap in the spatial coverage between adjacent frames. The method for extracting the overlapping area is as follows: for the first infrared frame and the second infrared frame, the ground coverage polygons of the first and second infrared frames are obtained respectively, and the spatial intersection area of ​​the two polygons is calculated. This intersection area is the overlapping area between the first and second infrared frames.

[0104] Within the overlapping area, the ground-cloud processing unit identifies the first temperature field feature point set in the first infrared frame and the second temperature field feature point set in the second infrared frame. Temperature field feature points refer to spatial location points in the infrared image that have significant temperature field changes, usually corresponding to structural parts with obvious temperature gradients or geometric contours, such as the edge of power equipment, insulator string connection points, conductor splicing pipes, and equipment clamps.

[0105] The feature point identification method is as follows: In the overlapping area, calculate the rate of change of the temperature field value of each pixel position in the horizontal and vertical directions, and determine the pixel position with the rate of change amplitude greater than the feature response judgment value as a candidate feature point. The candidate feature points are then filtered by local non-maximum, that is, in the neighborhood window centered on the candidate feature point, only the point with the largest rate of change amplitude is retained as a valid feature point.

[0106] The above operation is performed on the overlapping area of ​​the first infrared frame to obtain the first temperature field feature point set. The feature point set contains the pixel coordinates of each feature point in the infrared image. The same operation is performed on the overlapping area of ​​the second infrared frame to obtain the second temperature field feature point set. The feature points in the first temperature field feature point set and the second temperature field feature point set are corresponding according to their spatial distribution relative position relationship, that is, the same ground physical point corresponds to one feature point in the first infrared frame and one in the second infrared frame.

[0107] After acquiring the first and second temperature field feature point sets, the ground-to-cloud processing unit determines the spatial transformation parameter set between adjacent infrared frames based on the correspondence between the first and second temperature field feature point sets. The method for establishing the correspondence between feature points is as follows: for each feature point in the first temperature field feature point set, the temperature field distribution pattern in its surrounding neighborhood is extracted as descriptive information. The temperature field distribution pattern can be the order of temperature field values ​​of multiple sampling points in the neighborhood. In the second temperature field feature point set, the feature point with the most similar temperature field distribution pattern is found as the corresponding point. The degree of similarity is measured by comparing the consistency of the temperature field distribution patterns in the neighborhood of the two feature points. Through the above matching process, multiple pairs of corresponding feature points are selected from the first and second temperature field feature point sets.

[0108] Based on these feature point pairs, a set of spatial transformation parameters between the first infrared frame and the second infrared frame is calculated. The set of spatial transformation parameters includes translation parameters, rotation parameters, and scaling parameters. The translation parameter represents the overall offset of the second infrared frame relative to the first infrared frame in the row and column directions. The calculation method is as follows: for each pair of feature points, the difference between the row coordinates of the first feature point and the row coordinates of the second feature point, as well as the difference between the column coordinates, are calculated. The average value of the coordinate differences of all feature point pairs is then obtained to obtain the row direction translation parameter and the column direction translation parameter.

[0109] The rotation parameter represents the rotation angle of the second infrared frame relative to the first infrared frame around the image center. It is calculated as follows: select two pairs of non-collinear feature points from the feature point pairs, calculate the direction angle of the line connecting the two feature points in the first infrared frame and the direction angle of the line connecting the corresponding two feature points in the second infrared frame, and take the difference between the two direction angles as the rotation parameter. The scaling parameter represents the magnification or reduction ratio of the second infrared frame relative to the first infrared frame. It is calculated as follows: calculate the Euclidean distance between two feature points in the first infrared frame and the Euclidean distance between the corresponding two feature points in the second infrared frame, and take the ratio of the two distances as the scaling parameter.

[0110] After determining the spatial transformation parameter set, the ground-cloud processing unit transforms the second infrared frame into the coordinate space of the first infrared frame according to the spatial transformation parameter set, and performs weighted fusion of temperature field values ​​in the overlapping area. The specific transformation operation is as follows: for each pixel coordinate in the second infrared frame, firstly, a scaling transformation is performed with the image center as the origin, that is, its row and column coordinates are multiplied by the scaling parameter respectively. Then, the scaled coordinates are rotated around the image center by the angle indicated by the rotation parameter. Finally, the rotated coordinates are added to the row direction translation parameter and the column direction translation parameter respectively to obtain the corresponding coordinates of the pixel in the coordinate space of the first infrared frame. After the coordinate transformation of all pixels is completed, the second infrared frame is mapped into the coordinate space of the first infrared frame. In the overlapping area, the same ground position may have corresponding temperature field values ​​in both the first and second infrared frames. The two temperature field values ​​need to be fused to eliminate the splicing seam.

[0111] The weighted fusion method is as follows: For each pixel position in the overlapping area, the temperature field value of that position in the first infrared frame is obtained as the first temperature field value, and the temperature field value of the corresponding position in the transformed second infrared frame is obtained as the second temperature field value. The first weight and the second weight are determined according to the distance of the pixel position from the boundary of the overlapping area. The closer to the center of the coverage area of ​​the first infrared frame, the larger the first weight; the closer to the center of the coverage area of ​​the second infrared frame, the larger the second weight. The sum of the two weights is a fixed value. The first temperature field value is multiplied by the first weight, and the second temperature field value is multiplied by the second weight. The two products are added together, and the sum is the fused temperature field value. According to the above transformation and fusion method, all infrared frames in the compensated infrared frame sequence are stitched into a unified coordinate system in sequence to finally form a wide-area infrared panoramic image covering the inspection area. The wide-area infrared panoramic image presents the complete temperature field distribution of the entire inspection area in the form of a single large-size image.

[0112] After generating the wide-area infrared panoramic image, the ground-cloud processing unit extracts the temperature field distribution sub-regions corresponding to each power device from the wide-area infrared panoramic image. The temperature field distribution sub-regions corresponding to power devices refer to the connected pixel regions belonging to the main structure of the same power device in the panoramic image. The extraction method is as follows: a spatial location template for each power device in the inspection area is pre-established. The spatial location template records the outer boundary coordinates of each power device in the geographic coordinate system of the panoramic image. Based on the spatial location template of each power device, the sub-image within the corresponding boundary range is cropped from the wide-area infrared panoramic image as the temperature field distribution sub-region of the power device. For each temperature field distribution sub-region, the first temperature difference between the maximum value and the average value of the temperature field value in the sub-region is calculated, as well as the spatial change gradient of the temperature field value in the sub-region.

[0113] The first temperature difference is calculated as follows: traverse the temperature field values ​​of all pixels in the sub-region, find the maximum temperature field value, calculate the arithmetic mean of the temperature field values ​​of all pixels in the sub-region, subtract the arithmetic mean from the maximum temperature field value, and the difference is the first temperature difference. The first temperature difference reflects whether there are local abnormal hot spots on the surface of the power equipment and the degree of overheating. The spatial change gradient is calculated as follows: calculate the absolute value of the temperature field value difference between adjacent pixels in the sub-region, and take the maximum or average value of the absolute value of the temperature difference between all adjacent pixels as the spatial change gradient. The spatial change gradient reflects the degree of temperature field change in space. The larger the gradient, the more uneven the temperature field distribution, and the more likely there are local defects. When the first temperature difference exceeds the first temperature difference reference value and the spatial change gradient exceeds the first gradient reference value, it indicates that the power equipment meets the two conditions of significantly higher local temperature and uneven temperature field distribution. At this time, the power equipment is marked as abnormally heating equipment.

[0114] The first temperature difference reference value is determined based on the historical temperature difference statistical distribution of this type of power equipment under normal operating conditions. The first gradient reference value is determined based on the spatial uniformity statistical distribution of the temperature field of normal equipment. After marking the equipment as abnormally heating, an equipment alarm message containing the equipment location coordinates and the temperature difference abnormality level is generated. The equipment location coordinates are taken from the center coordinates of the equipment spatial location template. The temperature difference abnormality level is divided into three levels: slight abnormality, general abnormality, and severe abnormality based on the combination of the first temperature difference and the spatial change gradient. The alarm message is sent to the inspection and monitoring terminal through the alarm output interface of the ground cloud processing unit, prompting the operation and maintenance personnel to review and repair the abnormally heating equipment.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the described technical solution.

Claims

1. A method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI, characterized in that: Includes the following steps: S1. Obtain the original infrared frame sequence collected by the infrared imaging core on the UAV during cruise flight, and obtain the attitude angle data and angular velocity data of the UAV gimbal at each original infrared frame acquisition time. S2. Based on the attitude angle data and angular velocity data, determine the motion blur direction parameter and motion blur amplitude parameter corresponding to each original infrared frame; S3. Based on the motion blur direction parameter and motion blur amplitude parameter, perform frame-by-frame motion compensation processing on the original infrared frame sequence to generate a compensated infrared frame sequence. S4. The compensated infrared frame sequence and the corresponding attitude angle data are sent to the ground cloud processing unit, which performs inter-frame spatial consistency reconstruction and global temperature field stitching to generate a wide-area infrared panoramic image covering the inspection area.

2. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 1, characterized in that, In S1, the infrared imaging core mounted on the UAV continuously collects infrared radiation intensity data at the first frame frequency to form an original infrared frame sequence. The exposure timing of the infrared imaging core is synchronized with the integration time of the readout circuit inside the core, so that all pixels of the focal plane array start integration at the same time and end integration at the same time. The angular velocity and attitude angular components of the gimbal in the pitch, yaw and roll directions are obtained by the angular velocity sensing unit and attitude calculation unit attached to the gimbal at a second sampling frequency, which is higher than the first frame frequency.

3. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 2, characterized in that, Before data acquisition begins, the UAV flight control system simultaneously sends synchronization pulse signals to the infrared imaging core and the gimbal attitude calculation unit. The infrared imaging core and the gimbal attitude calculation unit record the local clock count value at the moment the synchronization pulse signal is received as the zero point reference of their respective time axes. For each frame in the original infrared frame sequence, the attitude angle data group and angular velocity data group with the closest timestamp are found in the attitude data time series according to the frame timestamp. If the frame timestamp is between two attitude sampling points, the attitude angle and angular velocity are linearly interpolated according to the time difference ratio to establish the correspondence between each original infrared frame and the corresponding attitude angle data and angular velocity data.

4. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 1, characterized in that, In S2, the exposure duration corresponding to the current original infrared frame is obtained. The exposure duration is the length of the time window for the infrared imaging core focal plane array pixel to integrate infrared radiation. Extract all pitch angular velocity samples from the attitude calculation unit between the start and end of the exposure, sum them up, and divide by the number of samples to obtain the average pitch angular velocity. Extract all yaw angular velocity samples, sum them up, and divide by the number of samples to obtain the average yaw angular velocity.

5. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 4, characterized in that, The first directional blur offset is obtained by multiplying the average pitch angular velocity by the exposure duration, and the second directional blur offset is obtained by multiplying the average yaw angular velocity by the exposure duration. Calculate the square root of the sum of the squares of the first direction blur offset and the squares of the second direction blur offset, and use it as the motion blur amplitude parameter; The arctangent of the ratio of the second direction blur offset to the first direction blur offset is calculated to obtain the motion blur direction parameter, which represents the angle between the image point movement trajectory and the first direction axis of the focal plane array.

6. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 1, characterized in that, In S3, for each original infrared frame, the angle value represented by the motion blur direction parameter is mapped to the row and column coordinate system of the focal plane array, and the main axis direction of the arrangement of the compensation filter kernel weight coefficients is determined as the filtering direction. The filtering direction is consistent with the image point trailing extension direction. The pixel displacement represented by the motion blur amplitude parameter is converted into the number of pixels. When the displacement is an integer number of pixels, the filter length is an integer number of pixels. When the displacement contains a fractional part, the filter length is a minimum odd number of pixels not less than the displacement.

7. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 6, characterized in that, A compensation filter kernel with filtering direction and filtering length is constructed. The center weight of the compensation filter kernel is positive and the edge weight is negative. The center of the compensation filter kernel is aligned with the current pixel position to be processed in the original infrared frame. The pixel position and several pixels before and after it in the filtering direction are extracted and multiplied by the corresponding weight coefficient of the compensation filter kernel and then accumulated. The accumulated sum is used as the new gray value after the reverse filtering operation. The part of the image edge region that exceeds the boundary is filled by pixel value copying or mirror expansion. After performing inverse filtering on each pixel position in the original infrared frame, a compensated infrared frame is obtained. The compensated infrared frames are then arranged in the order of the acquisition time of the original infrared frames to form a compensated infrared frame sequence.

8. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 1, characterized in that, In S4, the ground cloud processing unit receives the compensated infrared frame sequence and corresponding attitude angle data, and maps the pixel coordinates of each infrared frame to a unified spatial coordinate system based on the UAV's spatial position, gimbal attitude angle, infrared imaging core field of view parameters and digital elevation model at the time of each frame acquisition. For the first and second infrared frames acquired at adjacent times, the spatial intersection of the ground coverage polygons in a unified spatial coordinate system is obtained as the overlapping area. Within the overlapping area, the amplitude of the rate of change of pixel temperature field values ​​in the horizontal and vertical directions is calculated respectively. Pixels with a rate of change amplitude greater than the feature response judgment value are selected as the first temperature field feature point set and the second temperature field feature point set after being filtered by local non-maximum values.

9. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 8, characterized in that, Extract the temperature field distribution pattern of the neighborhood around each feature point in the first temperature field feature point set. In the second temperature field feature point set, match the feature points with the most similar temperature field distribution patterns to establish several feature point pairs. Based on the feature point pairs, calculate the translation parameters of the second infrared frame relative to the first infrared frame in the row and column directions, the rotation parameters around the image center, and the scaling parameters, which together constitute the spatial transformation parameter set.

10. The method for analyzing infrared images of power lines using unmanned aerial vehicles (UAVs) based on cloud computing and AI according to claim 9, characterized in that, Based on the spatial transformation parameter set, scale, rotation and translation operations are performed on the second infrared frame to transform the second infrared frame into the coordinate space of the first infrared frame. In the overlapping area, the first weight is determined according to the distance of the pixel position from the center of the coverage area of ​​the first infrared frame and the second weight is determined according to the distance from the pixel position to the center of the coverage area of ​​the second infrared frame. The product of the temperature field value of the first infrared frame and the first weight and the product of the temperature field value of the second infrared frame and the second weight are added to obtain the fused temperature field value. All infrared frames in the compensated infrared frame sequence are sequentially transformed and fused into a unified coordinate system to form a wide-area infrared panoramic image. Based on the pre-established spatial location template of power equipment, the corresponding temperature field distribution sub-regions of each power equipment are cropped from the wide-area infrared panoramic image. The difference between the maximum temperature field value and the arithmetic mean of the temperature field in the sub-region is calculated as the first temperature difference. The maximum absolute value of the temperature field difference between adjacent pixels in the sub-region is calculated as the spatial change gradient. When the first temperature difference exceeds the first temperature difference reference value and the spatial change gradient exceeds the first gradient reference value, the corresponding power equipment will be marked as abnormally heated equipment, and an equipment alarm message containing the equipment location coordinates and the abnormal temperature difference level will be generated.

Images