Photovoltaic power station intelligent inspection system based on AI vision

Abstract

The invention relates to the technical field of photovoltaic power station operation and maintenance, in particular to a photovoltaic power station intelligent inspection system based on AI vision, which comprises a video acquisition module, a geometric reference construction module, a tremor offset resolving module, a coordinate inverse correction module and a defect fine calibration module, the video acquisition module is used for acquiring a real-time video stream of an unmanned aerial vehicle polling photovoltaic array, and performing time-space synchronization calibration on the video stream to generate an original image sequence. According to the invention, the computer vision technology is utilized to calculate a current frame blanking point in a video picture as a tiny offset of a reference object relative to a reference position in real time, the displacement is deducted from a GPS coordinate, and a coordinate inverse correction module is utilized to inversely calculate a visual axis offset into a ground projection error. The shake amount and the shake direction of the camera at each moment can be accurately calculated, and then the GPS coordinates are corrected in turn, so that the geographic accuracy of defect positioning is improved, and the operation and maintenance personnel can accurately find a fault component.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power plant operation and maintenance technology, specifically to an intelligent inspection system for photovoltaic power plants based on AI vision. Background Technology

[0002] With the transformation of the global energy structure, the installed capacity of photovoltaic power generation continues to grow, and the construction scale of large-scale centralized photovoltaic power plants is expanding day by day. In order to ensure the efficient operation of the power plant, it is crucial to conduct regular inspections of photovoltaic modules and promptly detect defects such as hot spots, microcracks, and shading. Traditional manual inspections are inefficient and pose safety hazards, and are now gradually being replaced by intelligent inspections by drones equipped with visible light and infrared cameras.

[0003] Existing drone photovoltaic inspection technology typically adopts a front-end acquisition plus real-time image transmission and edge computing model. It uses deep learning algorithms to identify defect targets in aerial images and combines the drone's flight attitude data to calculate the geographic coordinates of the defects through photogrammetry principles, thereby generating an inspection report.

[0004] However, in practical engineering applications, existing inspection systems face the following severe technical challenges, making it difficult to meet the needs of refined operation and maintenance: On the one hand, affected by high-altitude unsteady airflow and motor vibration, relying solely on mechanical pan-tilt units is insufficient to completely eliminate high-frequency line-of-sight micro-vibrations. Small line-of-sight deviations can cause severe drift in ground positioning coordinates under long-distance perspective projection, affecting the accuracy of defect location in photovoltaic power plants. On the other hand, visual image stabilization is usually achieved using feature-point-based algorithms, but this method is prone to failure in scenarios with high repetition of textures on photovoltaic panels, leading to multiple alarms for the same defect in consecutive video streams and causing duplicate counting problems. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent inspection system for photovoltaic power plants based on AI vision, comprising:

[0006] The video acquisition module is used to acquire real-time video streams of drones inspecting photovoltaic arrays and to perform spatiotemporal synchronization calibration on the video streams to generate original image sequences.

[0007] The geometric reference construction module is used to identify the rectangular edge features of photovoltaic modules in the original image sequence, calculate the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame of the image based on the array parallel geometric constraints, and construct the reference blanking point position and reference tilt angle features in the no-flicker state.

[0008] The flutter offset calculation module is used to compare the offset of the current frame's blanking point coordinates and tilt features with the reference blanking point position and reference tilt features in real time. Through the perspective projection model, the blanking point displacement and tilt rotation are decoupled into pitch, yaw and roll three-axis flutter components.

[0009] The coordinate inverse correction module is used to calculate the displacement vector of the ground projection point caused by the line-of-sight offset based on the three-axis flutter components, and to perform inverse compensation on the original positioning coordinates of the UAV to generate the flutter-corrected component positioning coordinates.

[0010] The defect calibration module is used to perform photovoltaic module defect detection on the original image sequence, and associate the identified defect hot spot location with the vibration-corrected module positioning coordinates to generate a defect location mapping relationship.

[0011] Preferably, the process involves acquiring a real-time video stream of a drone inspecting a photovoltaic array, and performing spatiotemporal synchronization calibration on the video stream to generate an original image sequence, including:

[0012] Acquire real-time video streams collected by the UAV and UAV flight attitude data packets recorded synchronously with the real-time video streams, wherein the flight attitude data packets include timestamps, latitude and longitude, and raw inertial navigation data;

[0013] Keyframes are extracted from the real-time video stream, and the extracted keyframes are time-aligned with the flight attitude data packet using the timestamp to establish a frame-attitude association dataset.

[0014] The images in the frame-pose association dataset are subjected to distortion correction and denoising processing. Invalid frames with motion blur exceeding the preset range are removed to obtain a quality-standardized original image sequence.

[0015] Preferably, the rectangular edge features of the photovoltaic modules in the original image sequence are identified, and the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame are calculated based on the array parallel geometric constraints. This constructs the reference blanking point position and reference tilt angle features under a flutter-free state, including:

[0016] Edge detection is performed on each frame of the original image sequence, straight line segment features are extracted, parallel straight line clusters that conform to the arrangement pattern of photovoltaic modules are selected, and the average tilt angle feature of the parallel straight line clusters is calculated.

[0017] The intersection points of the extended parallel line clusters on the image plane are calculated based on the principles of perspective geometry to obtain the instantaneous hidden line point coordinates of a single frame image.

[0018] Temporal smoothing filtering is performed on the instantaneous blanking point coordinates and the average tilt angle features of multiple consecutive frames of images to remove high-frequency jitter noise. The convergence coordinates and convergence tilt angle under stable flight conditions are statistically obtained and determined as the reference blanking point position and reference tilt angle features under flutter-free conditions.

[0019] Preferably, edge detection is performed on each frame of the original image sequence, straight line segment features are extracted, parallel straight line clusters that conform to the arrangement pattern of photovoltaic modules are selected, and the average tilt angle feature of the parallel straight line clusters is calculated, including:

[0020] The original image sequence is subjected to adaptive histogram equalization to enhance the contrast between the photovoltaic module edge and the background, and the full set of line segment features in the image is extracted using a line segment detection algorithm.

[0021] Calculate the orientation angle of each line segment in the full set of line segment features, construct an orientation angle statistical histogram, and determine the main arrangement direction of the photovoltaic array through peak search;

[0022] Set an angle tolerance threshold and a length constraint threshold, filter out line segments whose direction angle is within the range of the main arrangement direction and the angle tolerance threshold, and whose length is greater than the length constraint threshold, remove messy textures and non-component edge noise, and obtain a cluster of parallel straight lines;

[0023] The slopes of all line segments in the parallel line cluster are calculated by weighted average, with the weights depending on the line segment lengths, to obtain the average tilt angle characteristic of the parallel line cluster.

[0024] Preferably, the offsets of the current frame's blanking point coordinates and tilt features with the reference blanking point position and reference tilt features are compared in real time. The blanking point displacement and tilt rotation are decoupled into pitch, yaw, and roll three-axis flutter components using a perspective projection model, including:

[0025] Obtain the instantaneous blanking point coordinates and average tilt angle features of the current frame image, calculate the difference between them and the reference blanking point position and reference tilt angle features, and obtain the blanking point displacement vector and tilt angle rotation offset respectively.

[0026] A perspective projection model is constructed by introducing the camera intrinsic parameter matrix, and the mapping relationship between the displacement vector of the hidden point on the image plane and the camera pitch angle and yaw angle, as well as the mapping relationship between the tilt rotation offset and the camera roll angle are established.

[0027] By using the mapping relationship, the displacement vector of the blanking point and the tilt rotation offset are solved in reverse to separate the rotational changes caused by camera shake and obtain the pitch, yaw and roll three-axis jitter components.

[0028] Preferably, the displacement vector of the ground projection point caused by the line-of-sight offset is calculated based on the three-axis jitter components, and the original positioning coordinates of the UAV are reverse-compensated to generate jitter-corrected component positioning coordinates, including:

[0029] Based on the three-axis jitter components, a rotation correction matrix is ​​constructed, and combined with the current relative flight altitude of the UAV, the offset distance and offset orientation of the line-of-sight center point on the ground projection plane are calculated.

[0030] The offset distance and offset azimuth are combined to form a ground projection point displacement vector, and the ground projection point displacement vector is converted to a geographic coordinate system.

[0031] Obtain the original UAV positioning coordinates corresponding to the current frame, subtract the displacement vector of the ground projection point for reverse compensation, and obtain the component positioning coordinates after vibration correction to eliminate jitter error.

[0032] Preferably, photovoltaic module defect detection is performed on the original image sequence, and the identified defect hotspot locations are correlated and mapped with the vibration-corrected module positioning coordinates to generate a defect location mapping relationship, including:

[0033] The current frame image is identified using a deep learning object detection algorithm to locate the center point of the bounding box of the defect hot spot in the image pixel coordinate system;

[0034] Based on the component positioning coordinates after the vibration correction, a transformation matrix between the image pixel coordinate system and the geographic coordinate system is established to convert the center point of the bounding box into geographic spatial coordinates.

[0035] The identified defect types, defect image fragments, and converted geospatial coordinates are associated and bound to generate a defect location mapping relationship that includes defect attributes and precise geographical locations.

[0036] Preferably, after generating the defect location mapping relationship containing defect attributes and precise geographical locations, the method further includes:

[0037] Construct a defect spatiotemporal buffer queue, and sequentially store the defect location mapping relationship generated from multiple consecutive frames of images into the defect spatiotemporal buffer queue;

[0038] Spatial clustering analysis is performed on the defect data in the defect spatiotemporal buffer queue. Based on the Euclidean distance of the geographic spatial coordinates, defect data that are determined to be the same target are grouped into the same defect cluster.

[0039] Calculate the weighted centroid of the geospatial coordinates of all defect data within the same defect cluster, where the weights are related to the confidence of defect detection in each frame, to obtain the globally unique geographic coordinates of the defect.

[0040] By binding the globally unique geographic coordinates with the corresponding defect attributes, a global defect distribution map of the photovoltaic power station is generated, thus completing the deduplication and fusion of duplicate detected defects.

[0041] Preferably, the system further includes:

[0042] The dynamic benchmark update module is used to verify the correction accuracy based on the defect location mapping relationship. When the positioning deviation exceeds a preset threshold, the benchmark blanking point position and benchmark tilt angle feature are recalibrated, and the updated benchmark is fed back to the geometric benchmark construction module.

[0043] Preferably, the correction accuracy is verified based on the defect location mapping relationship. When the positioning deviation exceeds a threshold, the reference blanking point position and reference tilt angle feature are recalibrated, and the updated reference is fed back to the geometric reference construction module, including:

[0044] Extract the geospatial coordinates of the same defect hotspot in multiple adjacent frames, calculate the coordinate dispersion of the same defect hotspot between different frames, and obtain the positioning deviation value;

[0045] The positioning deviation value is compared with a preset accuracy threshold. If the positioning deviation value is less than or equal to the accuracy threshold, the current reference blanking point position and reference tilt angle characteristics remain unchanged.

[0046] If the positioning deviation value exceeds the accuracy threshold, it is determined that the current flight attitude has undergone a macroscopic change, triggering a reset mechanism. The latest set of image sequences is collected to recalculate the convergence coordinates and convergence tilt angle, update the reference hidden point position and reference tilt angle features, and feed the updated reference back to the geometric reference construction module for offset calculation of subsequent frames.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] (1) This invention utilizes computer vision technology to calculate the tiny offset of the current frame blanking point in the video frame relative to the reference position in real time, and deducts this displacement from the GPS coordinates. The coordinate inverse correction module is used to calculate the line-of-sight offset into ground projection error, which can accurately calculate how much the camera shakes at each instant and in which direction. Then, the GPS coordinates are corrected in reverse, which improves the geographical accuracy of defect location and ensures that maintenance personnel can accurately find the faulty components.

[0049] (2) The present invention is equipped with a geometric reference construction module, which uses the neatly arranged edge lines of the components in the photovoltaic power station as the calculation reference to extract the global cluster of parallel straight lines and their average tilt angle features. Since this parallel line structure is very obvious and stable under any lighting conditions, even under conditions of changing lighting or simple texture, the system can still stably calculate the camera's attitude, ensuring the reliability of operation in complex environments.

[0050] (3) This invention introduces an intrinsic parameter matrix containing various parameters of the camera to construct a perspective projection model, thereby distinguishing the movement of the UAV position and the tilt of the lens angle, and calculating their impact on the picture separately. This is more comprehensive than simply correcting the position deviation, and can restore the true posture of the UAV during shooting to the greatest extent, thereby making the calculated ground coordinates more accurate and improving the inspection quality. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall system architecture in one embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1, please refer to Figure 1 This invention provides a technical solution: an intelligent inspection system for photovoltaic power plants based on AI vision, comprising:

[0054] The video acquisition module is used to acquire real-time video streams of drones inspecting photovoltaic arrays and to perform spatiotemporal synchronization calibration on the video streams to generate original image sequences.

[0055] The geometric benchmark construction module is used to identify the rectangular edge features of photovoltaic modules in the original image sequence, calculate the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame of the image based on the array parallel geometric constraints, and construct the benchmark blanking point position and benchmark tilt angle features in the no-flutter state.

[0056] The flutter offset calculation module is used to compare the offset of the current frame's blanking point coordinates and tilt features with the reference blanking point position and reference tilt features in real time. It decouples the blanking point displacement and tilt rotation into pitch, yaw and roll three-axis flutter components through a perspective projection model.

[0057] The coordinate inverse correction module is used to calculate the displacement vector of the ground projection point caused by the line-of-sight offset based on the three-axis flutter components, and to perform inverse compensation on the original positioning coordinates of the UAV to generate the flutter-corrected component positioning coordinates.

[0058] The defect calibration module is used to perform photovoltaic module defect detection on the original image sequence. It associates and maps the identified defect hotspot locations with the vibration-corrected module positioning coordinates to generate defect location mapping relationships.

[0059] In this invention, the video acquisition module is typically mounted beneath an industrial-grade drone. As a sensing front-end, it utilizes a timestamp alignment algorithm to resolve temporal discrepancies between high-frequency flight control data and low-frequency video streams, ensuring that each frame accurately indexes the corresponding drone attitude and position information. The geometric reference construction module leverages the parallel line features presented by the photovoltaic array under perspective projection. By calculating the disappearance points of these parallel lines on the image plane, it establishes a geometric reference system representing an ideal, flutter-free flight state. The flutter offset calculation module, based on monocular visual geometry principles, captures in real-time the minute displacements and rotations of the current frame's disappearance points relative to the reference reference system. Through a perspective projection model, it inversely solves these two-dimensional pixel changes into instantaneous physical jitter angles of the camera in the pitch, yaw, and roll dimensions. The coordinate inverse correction module takes the aforementioned jitter angles and, combined with the drone's altitude relative to the ground, calculates the ground projection point error vector caused by the line-of-sight offset through trigonometric geometric operations. It then performs inverse compensation on the original GPS coordinates to obtain accurate component positioning coordinates. The defect fine calibration module overlays deep learning recognition results onto the corrected geographic coordinate system to achieve precise defect localization.

[0060] In an optional embodiment, the video acquisition module acquires a real-time video stream of the photovoltaic array inspected by the drone, and performs spatiotemporal synchronization calibration on the video stream to generate an original image sequence, including:

[0061] Acquire real-time video streams collected by the UAV and UAV flight attitude data packets recorded synchronously with the real-time video streams. The flight attitude data packets include timestamps, latitude and longitude, and raw inertial navigation data.

[0062] Keyframes are extracted from the real-time video stream, and the extracted keyframes are time-aligned with the flight attitude data packets using timestamps to establish a frame-attitude association dataset.

[0063] Distortion correction and denoising processing are performed on the images in the frame-pose association dataset. Invalid frames with motion blur exceeding the preset range are removed to obtain a quality-standardized original image sequence.

[0064] It should be noted that the spatiotemporal synchronization calibration mentioned in this embodiment refers to the process of eliminating the deviation between the video stream and the flight attitude data on the time axis and unifying their spatial coordinate systems; keyframe extraction refers to the process of selecting representative image frames from a continuous high frame rate video stream according to specific rules in order to reduce data redundancy and ensure the overlap rate required for subsequent stitching or calculation; temporal alignment refers to the synchronization mechanism that solves the problem of asynchronous or inconsistent frequencies between the video stream acquisition clock and the flight control attitude recording clock; the frame-attitude association dataset refers to the mapping set of video frames and UAV attitude data aligned by timestamps, and its data structure is {video frame, timestamp, latitude and longitude, pitch angle, roll angle, yaw angle}; the quality-standardized original image sequence refers to the effective image set that has undergone distortion correction and denoising processing and has motion blur below the threshold, such as a Laplacian variance > 100, for subsequent feature extraction.

[0065] Specifically, the system obtains the absolute start time by reading the metadata of the video file, calculates the theoretical occurrence time of the Nth frame based on the frame rate, and searches within the flight attitude data packet containing timestamps, latitude and longitude, and inertial navigation data. For cases where the video frame timestamp and attitude data timestamp do not completely coincide, a linear interpolation algorithm is used to calculate the precise attitude at that moment in the video frame using attitude data from two consecutive frames. The calculation formula is as follows: In the formula, This represents the time proportion of the current video frame relative to the two preceding and following flight attitude sampling times. The value of this coefficient ranges from 0 to 1. infinitely close At this point, the time proportion coefficient approaches 0, and the interpolation result mainly depends on the attitude at the previous time step. ,when infinitely close At this point, the time proportion coefficient approaches 1, and the interpolation result mainly depends on the attitude at the next time step. , This represents the incremental change vector of flight attitude within the sampling interval, therefore This indicates the current video frame time relative to the initial sampling time. The cumulative change in attitude is calculated by adding this change to the base attitude. This establishes a high-precision frame-pose association dataset. The system calls pre-calibrated camera intrinsic parameter matrices and distortion coefficients, and uses a distortion correction algorithm to perform geometric correction on the images, ensuring that the edge lines of the photovoltaic modules conform to the straight-line characteristics of the pinhole camera model. The Laplacian variance algorithm is used to calculate the sharpness score of each frame. In the formula The second derivative Laplacian operator, representing the image, reflects the steepness of the edges. It removes motion-blurred frames with scores below a preset threshold, ensuring that subsequent algorithms only process high-quality, clear images.

[0066] In an optional embodiment, the rectangular edge features of the photovoltaic modules in the original image sequence are identified in the geometric reference construction module. Based on the array parallel geometric constraints, the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame of the image are calculated to construct the reference blanking point positions and reference tilt angle features in the flutter-free state, including:

[0067] Edge detection is performed on each frame of the original image sequence to extract straight line segment features, and parallel straight line clusters that conform to the arrangement pattern of photovoltaic modules are selected. The average tilt angle feature of the parallel straight line clusters is then calculated.

[0068] The intersection points of the extended parallel line clusters on the image plane are calculated based on the principles of perspective geometry, thus obtaining the coordinates of the instantaneous hidden line disappearance point of a single frame image.

[0069] Temporal smoothing filtering is performed on the instantaneous blanking point coordinates and average tilt angle features of multiple consecutive frames of images to remove high-frequency jitter noise. The convergence coordinates and convergence tilt angle under stable flight conditions are statistically obtained and determined as the reference blanking point position and reference tilt angle features under flutter-free conditions.

[0070] It should be noted that, in this embodiment, the rectangular edge feature refers to the straight edge shape of the photovoltaic module in the image, the array parallel geometric constraint refers to the geometric characteristics of the photovoltaic modules arranged in parallel to each other in physical space, the blanking point coordinates refer to the theoretical convergence point of the parallel lines in physical space on the perspective projection image plane, the parallel line cluster tilt angle feature refers to the statistical average of the slopes of multiple parallel line segments in the image, the reference blanking point position refers to the stable coordinates of the blanking point on the image plane when the UAV is flying smoothly without vibration, and the reference tilt angle feature refers to the stable tilt angle of the photovoltaic array edge lines when the UAV is flying smoothly.

[0071] Specifically, edge detection is performed on each frame of the original image sequence to extract straight line segment features. Parallel straight line clusters that conform to the arrangement pattern of photovoltaic modules are selected and the average tilt angle features of the parallel straight line clusters are calculated. Based on the principle of perspective geometry, the least squares method is used to solve for the extension intersection points of the parallel straight line clusters on the image plane to obtain the instantaneous blanking point coordinates of a single frame image. Temporal smoothing filtering is performed on the instantaneous blanking point coordinates and average tilt angle features of multiple consecutive frames to remove high-frequency jitter noise caused by airflow disturbance. The variance of the coordinates and tilt angle changes is statistically monitored. When the variance converges to a preset minimum range, the UAV is determined to be in a stable flight state. The converged coordinates and converged tilt angle at this time are determined as the reference blanking point position and reference tilt angle features in the no-flutter state, and used as the reference zero point for subsequent calculation of the flutter offset.

[0072] In an optional embodiment, the geometric reference construction module performs edge detection on each frame of the original image sequence, extracts line segment features, filters out clusters of parallel lines that conform to the arrangement pattern of photovoltaic modules, and calculates the average tilt angle features of the parallel line clusters, including:

[0073] Adaptive histogram equalization is applied to the original image sequence to enhance the contrast between the edges of the photovoltaic modules and the background. A line segment detection algorithm is used to extract the full set of line segment features in the image.

[0074] Calculate the orientation angle of each line segment in the full set of line segment features, construct an orientation angle statistical histogram, and determine the main arrangement direction of the photovoltaic array through peak search;

[0075] Set an angle tolerance threshold and a length constraint threshold, filter out line segments whose direction angle is within the range of the main arrangement direction and the angle tolerance threshold, and whose length is greater than the length constraint threshold, remove messy textures and non-component edge noise, and obtain a cluster of parallel straight lines;

[0076] The slopes of all line segments in a cluster of parallel lines are calculated by weighted average, with the weights depending on the line segment lengths, to obtain the average tilt angle characteristic of the cluster of parallel lines.

[0077] It should be noted that the adaptive histogram equalization process mentioned in this embodiment is an image enhancement technique that improves image details and suppresses noise by limiting the magnitude of local contrast enhancement. The full set of line segment features refers to the parameter set of all detected line segments in the image. The main arrangement direction refers to the main direction of the photovoltaic module edge line in the image determined by statistical histogram. The parallel line cluster refers to the set of line segments representing the edge of the photovoltaic module that are retained after being filtered by angle and length. The weighted average calculation refers to a mathematical method for calculating the average slope by allocating weights according to the line segment length.

[0078] Specifically, adaptive histogram equalization is applied to the original image to enhance the grayscale difference between the photovoltaic module edge and the backsheet or ground. A line segment detection algorithm is used to extract the full set of line segment features in the image. When calculating the array tilt angle, a weighted average algorithm based on line segment length is executed, and its calculation formula is as follows: In this formula The calculated macroscopic average tilt angle of the photovoltaic array. The total number of parallel line segments selected. For the first The pixel length of the distance a line segment spans on an image. For the first In image processing, the geometric angles of line segments on the image plane are considered. The edges of photovoltaic modules are typically represented as long, continuous line segments, while ground weeds, shadows, or gaps between modules are usually represented as broken, short line segments. Therefore... Here, it is used as a confidence weight; the longer the line segment, the higher its confidence score. The larger the value, the greater the direction angle of the line segment. The higher the proportion in the final result, the lower the proportion; conversely, when the line segment is short, fragmented, and noisy, its... When the value is small, its impact on the result is suppressed. This represents the total moment of inclination after length weighting of all detected line segments. The total length of all line segments is represented by the sum of the two lengths, and normalization is achieved by dividing the sum by the sum of the two lengths. This calculation method ensures that the calculated average inclination angle is accurate. It automatically moves towards the long, straight edges that occupy the main features of the image, enabling the extraction of macroscopic geometric features that represent the overall orientation of the photovoltaic array even in environments with background noise interference.

[0079] In an optional embodiment, the flutter offset calculation module compares in real time the offsets of the current frame's blanking point coordinates and tilt features with the reference blanking point position and reference tilt features. Using a perspective projection model, the blanking point displacement and tilt rotation are decoupled into pitch, yaw, and roll three-axis flutter components, including:

[0080] Obtain the instantaneous blanking point coordinates and average tilt angle features of the current frame image, calculate the difference between them and the reference blanking point position and reference tilt angle features, and obtain the blanking point displacement vector and tilt angle rotation offset respectively.

[0081] A perspective projection model is constructed by introducing the camera intrinsic parameter matrix, and the mapping relationship between the displacement vector of the hidden point on the image plane and the camera pitch angle and yaw angle, as well as the mapping relationship between the tilt rotation offset and the camera roll angle are established.

[0082] By using the mapping relationship, the displacement vector of the hidden point and the tilt rotation offset are solved in reverse to separate the rotational changes caused by camera shake and obtain the pitch, yaw and roll three-axis jitter components.

[0083] It should be noted that, in this embodiment, the blanking point displacement vector refers to the two-dimensional vector difference between the current frame blanking point coordinates and the reference blanking point coordinates; the tilt rotation offset refers to the angle difference between the average tilt angle of the current frame parallel line cluster and the reference tilt angle; the perspective projection model refers to the mathematical model describing the process of mapping three-dimensional spatial points to a two-dimensional image plane; and the three-axis jitter component refers to the instantaneous jitter angle of the camera relative to the reference attitude in the three degrees of freedom of pitch, yaw, and roll.

[0084] Specifically, the jitter migration calculation module calculates the average tilt angle of the cluster of parallel lines in the current frame image. relative to the reference tilt angle rotation difference This difference directly corresponds to the camera's roll angle, i.e., the jitter component, reflecting the image rotation caused by the left-right imbalance of the drone's arms. Using the focal length parameter in the camera's intrinsic parameter matrix, the pixel displacement of the hidden point on the image plane is decoupled into angular change. The calculation formulas are as follows: and ,in and These are the jitter components for the camera's yaw and pitch angles, respectively. and The x and y coordinates of the current frame's hidden line disappearance point are given. and The x and y coordinates of the baseline hidden point are given. and The horizontal and vertical focal lengths obtained from camera calibration, in pixels, are used to convert the horizontal movement of the hidden point on the image plane into the horizontal rotation angle of the camera, and the vertical movement into the vertical pitch angle of the camera, thereby accurately quantifying physical shake.

[0085] In an optional embodiment, the coordinate inverse correction module calculates the ground projection point displacement vector caused by the line-of-sight offset based on the three-axis jitter components, and performs inverse compensation on the original UAV positioning coordinates to generate jitter-corrected component positioning coordinates, including:

[0086] A rotation correction matrix is ​​constructed based on the three-axis flutter components. Combined with the current relative flight altitude of the UAV, the offset distance and offset azimuth of the line-of-sight center point on the ground projection plane are calculated.

[0087] Combine the offset distance and offset azimuth to form the ground projection point displacement vector, and then convert the ground projection point displacement vector to the geographic coordinate system.

[0088] Obtain the original UAV positioning coordinates corresponding to the current frame, subtract the ground projection point displacement vector for reverse compensation, and obtain the component positioning coordinates after vibration correction to eliminate jitter error.

[0089] It should be noted that, in this embodiment, the line-of-sight offset refers to the angular deviation of the camera's main optical axis from the vertically downward direction due to camera body vibration; the ground projection point displacement vector refers to the displacement vector of the actual landing point of the line-of-sight center on the ground relative to the ideal landing point; reverse compensation refers to the mathematical operation of subtracting the error vector from the original coordinates to restore the true position; and the component positioning coordinates after vibration correction refer to the geographical coordinates that can accurately correspond to the actual position of the photovoltaic module after the vibration error is eliminated.

[0090] Specifically, the coordinate inverse correction module performs geometric calculations to convert angular errors into geographic location errors, i.e., it performs coordinate inverse correction, and uses the drone's altimeter data as the relative flight altitude. Combined with the pitch calculated above and yaw The vibration angle is calculated by determining the offset distance of the camera's principal optical axis center point on the ground projection plane relative to the ideal landing point. The formula is: In the formula This represents the straight-line offset of the line of sight on the ground caused by vibration. This represents the combined total tilt angle by which the line of sight deviates from the vertically downward direction. In geometric projection relationships, the larger this combined total tilt angle, the farther the point of view deviates from the ideal center point. The part within the function represents the composite flutter angle, which is then decomposed into correction vectors in the latitude and longitude directions. And read the original GPS positioning coordinates of the UAV corresponding to the current frame. Execute the reverse compensation formula In the formula The corrected geographic coordinates of the photovoltaic modules. It is a displacement vector determined by the direction of the vibration. For example, if the vibration causes the camera to yaw... A negative value indicates a leftward shift, causing the line of sight to fall to the left side of the ground. The system needs to correct the coordinates to the right to obtain the true positioning coordinates pointing to the location of the photovoltaic module.

[0091] In an optional embodiment, the defect calibration module performs photovoltaic module defect detection on the original image sequence, and associates the identified defect hotspot locations with the vibration-corrected module positioning coordinates to generate a defect location mapping relationship, including:

[0092] The current frame image is identified using a deep learning object detection algorithm to locate the center point of the bounding box of the defect hot spot in the image pixel coordinate system;

[0093] Based on the component positioning coordinates corrected for tremor, a transformation matrix between the image pixel coordinate system and the geographic coordinate system is established to convert the center point of the bounding box into geographic spatial coordinates.

[0094] The identified defect types, defect image fragments, and converted geospatial coordinates are associated and bound to generate a defect location mapping relationship that includes defect attributes and precise geographical locations.

[0095] It should be noted that the deep learning target detection algorithm mentioned in this embodiment refers to an algorithm that uses convolutional neural networks such as YOLO to automatically identify specific targets in an image and output their position and category. The defect hot spot refers to the area on the photovoltaic module where the temperature rises abnormally due to a fault. The image pixel coordinate system refers to a two-dimensional coordinate system with the upper left corner of the image as the origin. The transformation matrix refers to a mathematical matrix that describes the mapping relationship between pixel coordinates and geographic coordinates. The defect location mapping relationship refers to the corresponding data structure of the visual attributes and geographic location attributes of the defect.

[0096] Specifically, this mapping process uses a trained deep learning object detection model, such as the YOLO series of algorithms, to identify defective objects in the image and outputs the center point of its bounding box in pixel coordinates. Using the corrected component positioning coordinates obtained in the preceding steps as the principal optical axis landing point, and combining the UAV altitude, camera field of view, and corrected attitude angle, a homography transformation matrix from the pixel plane to geographic space is constructed. and perform the operation. In this formula Let the latitude and longitude coordinates of the defect on the ground be in homogeneous form. It is a 3×3 transformation matrix that includes rotation, translation, and scaling parameters. The generated data structure, based on image pixel coordinates, includes the defect type, confidence level, and the high-precision latitude and longitude coordinates.

[0097] In an optional embodiment, after generating the defect location mapping relationship that includes defect attributes and precise geographical locations, the method further includes:

[0098] Construct a spatiotemporal buffer queue for defects, and store the defect location mapping relationship generated from multiple consecutive frames of images into the spatiotemporal buffer queue in sequence.

[0099] Spatial clustering analysis is performed on the defect data in the spatiotemporal buffer queue. Based on the Euclidean distance of the geographic spatial coordinates, defect data that are determined to be the same target are grouped into the same defect cluster.

[0100] Calculate the weighted centroid of the geospatial coordinates of all defect data within the same defect cluster, where the weights are related to the confidence of defect detection in each frame, to obtain the globally unique geographic coordinates of the defect.

[0101] By binding globally unique geographic coordinates with corresponding defect attributes, a global defect distribution map of the photovoltaic power station is generated, completing the deduplication and fusion of duplicate detected defects.

[0102] It should be noted that the defect spatiotemporal buffer queue mentioned in this embodiment refers to a memory structure used to temporarily store defect data detected within a certain time range; spatial clustering analysis refers to an algorithm that groups similar data points into one class based on spatial distance; weighted centroid refers to the geometric center that considers the weight of each data point; globally unique geographic coordinates refer to the unique coordinate point that represents the true location of the defect after fusion processing; and the global defect distribution map refers to a comprehensive view that displays the location and attributes of all defects in a photovoltaic power station.

[0103] Specifically, to address the issue of repeated detection of the same defect across multiple image frames, the system maintains a spatiotemporal defect buffer queue, storing newly generated defect mappings sequentially. A spatial distance-based clustering algorithm is employed; for example, multiple detection results with a geographical Euclidean distance less than a threshold (e.g., 0.5 meters) are grouped into the same defect cluster. When calculating the final coordinates, a confidence-based weighted centroid algorithm is used, with the calculation formula as follows: In the formula The merged result provides a globally unique geographic coordinate for this defect. This represents the total number of detection records within the cluster. For the first The latitude and longitude calculated from the second test For the first The confidence score of a detection typically ranges from 0 to 1. In deep learning detection, the clearer the target, the more complete the features, and the closer it is to the image center, the higher the confidence score output by the model. The higher the confidence level, the lower the confidence level. Conversely, if the target is located at the edge of the image or there is motion blur, the lower the confidence level in this formula. This represents a single geographic coordinate after a confidence-weighted calculation. As the sum of confidence levels, this weighted averaging mechanism makes the final fused coordinates... It favors observations with high confidence, i.e., high-quality imaging, and automatically reduces the negative impact of low-quality observations on positioning accuracy, thus eliminating random positioning errors.

[0104] In an optional embodiment, the system further includes:

[0105] The dynamic datum update module is used to verify the correction accuracy based on the defect location mapping relationship. When the positioning deviation exceeds the preset threshold, the datum blanking point position and datum tilt angle feature are recalibrated, and the updated datum is fed back to the geometric datum construction module.

[0106] In an optional embodiment, the dynamic datum update module verifies the correction accuracy based on the defect location mapping relationship. When the positioning deviation exceeds a threshold, the datum blanking point position and datum tilt angle characteristics are recalibrated, and the updated datum is fed back to the geometric datum construction module, including:

[0107] Extract the geospatial coordinates of the same defect hotspot in multiple adjacent frames, calculate the coordinate dispersion of the same defect hotspot between different frames, and obtain the positioning deviation value;

[0108] The positioning deviation value is compared with the preset accuracy threshold. If the positioning deviation value is less than or equal to the accuracy threshold, the current reference blanking point position and reference tilt angle characteristics remain unchanged.

[0109] If the positioning deviation exceeds the accuracy threshold, it is determined that the current flight attitude has changed macroscopically, triggering the reset mechanism. The latest set of image sequences is collected to recalculate the convergence coordinates and convergence tilt angle, update the reference hidden point position and reference tilt angle features, and feed the updated reference back to the geometric reference construction module for offset calculation of subsequent frames.

[0110] It should be noted that the positioning deviation value mentioned in this embodiment refers to the degree of dispersion between the geographic coordinates calculated for the same defect in different frames, the accuracy threshold refers to the maximum positioning error limit allowed by the system, the reset mechanism refers to the control logic of clearing the current benchmark and re-executing the benchmark construction process when the system error exceeds the limit, and the coordinate dispersion refers to the statistical quantity describing the degree of concentration of data distribution.

[0111] Specifically, this module acts as a monitoring and feedback mechanism to verify the correction accuracy based on the defect location mapping relationship. When the positioning deviation exceeds a preset threshold, it recalibrates the reference blanking point position and reference tilt angle characteristics. It continuously monitors the dispersion of the defect coordinate cluster after deduplication and fusion, and calculates the standard deviation of all coordinate points within the same defect cluster. The calculation formula is as follows: In the formula This represents the dispersion of the positioning results. For the number of observations, For single-time positioning coordinates, The coordinates of the cluster center Represents single positioning coordinates Relative to cluster center Euclidean distance deviation, This deviation is converted to the order of area to eliminate the influence of direction and amplify the weight of larger errors. This represents the mean square value of the positioning deviation, and the square root of the result is... The dimensions are then restored to distance units to quantify the spatial dispersion of the same physical defect across multiple frames of observation. When calculations reveal... When the accuracy exceeds a preset threshold, such as 2 meters, the system determines that the current geometric reference is no longer applicable, that is, the zero point has drifted due to the UAV's change of direction or terrain undulation. The system then triggers a reset mechanism, instructing the geometric reference construction module to clear the old reference and immediately collect the latest set of image sequences to re-perform edge detection and hidden point convergence calculation, and establish a reference hidden point position and reference tilt angle feature that are adapted to the current new course or new terrain.

[0112] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. An intelligent inspection system for photovoltaic power plants based on AI vision, characterized in that, include: The video acquisition module is used to acquire real-time video streams of drones inspecting photovoltaic arrays and to perform spatiotemporal synchronization calibration on the video streams to generate original image sequences. The geometric reference construction module is used to identify the rectangular edge features of photovoltaic modules in the original image sequence, calculate the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame of the image based on the array parallel geometric constraints, and construct the reference blanking point position and reference tilt angle features in the no-flicker state. The flutter offset calculation module is used to compare the offset of the current frame's blanking point coordinates and tilt features with the reference blanking point position and reference tilt features in real time. Through the perspective projection model, the blanking point displacement and tilt rotation are decoupled into pitch, yaw and roll three-axis flutter components. The coordinate inverse correction module is used to calculate the displacement vector of the ground projection point caused by the line-of-sight offset based on the three-axis flutter components, and to perform inverse compensation on the original positioning coordinates of the UAV to generate the flutter-corrected component positioning coordinates. The defect calibration module is used to perform photovoltaic module defect detection on the original image sequence, and associate the identified defect hot spot location with the vibration-corrected module positioning coordinates to generate a defect location mapping relationship.

2. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 1, characterized in that, Acquire real-time video streams of drone inspections of photovoltaic arrays, and perform spatiotemporal synchronization calibration on the video streams to generate original image sequences, including: Acquire real-time video streams collected by the UAV and UAV flight attitude data packets recorded synchronously with the real-time video streams, wherein the flight attitude data packets include timestamps, latitude and longitude, and raw inertial navigation data; Keyframes are extracted from the real-time video stream, and the extracted keyframes are time-aligned with the flight attitude data packet using the timestamp to establish a frame-attitude association dataset. The images in the frame-pose association dataset are subjected to distortion correction and denoising processing. Invalid frames with motion blur exceeding the preset range are removed to obtain a quality-standardized original image sequence.

3. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 2, characterized in that, Identify the rectangular edge features of the photovoltaic modules in the original image sequence, calculate the coordinates of the blanking points and the tilt angle features of the parallel line clusters in each frame of the image based on the array parallel geometric constraints, and construct the reference blanking point position and reference tilt angle features under the flutter-free state, including: Edge detection is performed on each frame of the original image sequence, straight line segment features are extracted, parallel straight line clusters that conform to the arrangement pattern of photovoltaic modules are selected, and the average tilt angle feature of the parallel straight line clusters is calculated. The intersection points of the extended parallel line clusters on the image plane are calculated based on the principles of perspective geometry to obtain the instantaneous hidden line point coordinates of a single frame image. Temporal smoothing filtering is performed on the instantaneous blanking point coordinates and the average tilt angle features of multiple consecutive frames of images to remove high-frequency jitter noise. The convergence coordinates and convergence tilt angle under stable flight conditions are statistically obtained and determined as the reference blanking point position and reference tilt angle features under flutter-free conditions.

4. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 3, characterized in that, Edge detection is performed on each frame of the original image sequence, straight line segment features are extracted, and clusters of parallel straight lines that conform to the arrangement pattern of photovoltaic modules are selected. The average tilt angle feature of the clusters of parallel straight lines is then calculated, including: The original image sequence is subjected to adaptive histogram equalization to enhance the contrast between the photovoltaic module edge and the background, and the full set of line segment features in the image is extracted using a line segment detection algorithm. Calculate the orientation angle of each line segment in the full set of line segment features, construct an orientation angle statistical histogram, and determine the main arrangement direction of the photovoltaic array through peak search; Set an angle tolerance threshold and a length constraint threshold, filter out line segments whose direction angle is within the range of the main arrangement direction and the angle tolerance threshold, and whose length is greater than the length constraint threshold, remove messy textures and non-component edge noise, and obtain a cluster of parallel straight lines; The slopes of all line segments in the parallel line cluster are calculated by weighted average, with the weights depending on the line segment lengths, to obtain the average tilt angle characteristic of the parallel line cluster.

5. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 3, characterized in that, The offset of the current frame's blanking point coordinates and tilt features from the reference blanking point position and reference tilt features is compared in real time. The blanking point displacement and tilt rotation are decoupled into pitch, yaw, and roll three-axis flutter components using a perspective projection model, including: Obtain the instantaneous blanking point coordinates and average tilt angle features of the current frame image, calculate the difference between them and the reference blanking point position and reference tilt angle features, and obtain the blanking point displacement vector and tilt angle rotation offset respectively. A perspective projection model is constructed by introducing the camera intrinsic parameter matrix, and the mapping relationship between the displacement vector of the hidden point on the image plane and the camera pitch angle and yaw angle, as well as the mapping relationship between the tilt rotation offset and the camera roll angle are established. By using the mapping relationship, the displacement vector of the blanking point and the tilt rotation offset are solved in reverse to separate the rotational changes caused by camera shake and obtain the pitch, yaw and roll three-axis jitter components.

6. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 5, characterized in that, Based on the three-axis flutter components, the displacement vector of the ground projection point caused by the line-of-sight offset is calculated. The original UAV positioning coordinates are then reverse-compensated to generate flutter-corrected component positioning coordinates, including: Based on the three-axis jitter components, a rotation correction matrix is ​​constructed, and combined with the current relative flight altitude of the UAV, the offset distance and offset orientation of the line-of-sight center point on the ground projection plane are calculated. The offset distance and offset azimuth are combined to form a ground projection point displacement vector, and the ground projection point displacement vector is converted to a geographic coordinate system. Obtain the original UAV positioning coordinates corresponding to the current frame, subtract the displacement vector of the ground projection point for reverse compensation, and obtain the component positioning coordinates after vibration correction to eliminate jitter error.

7. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 1, characterized in that, Photovoltaic module defect detection is performed on the original image sequence. The identified defect hotspot locations are correlated and mapped with the vibration-corrected module positioning coordinates to generate a defect location mapping relationship, including: The current frame image is identified using a deep learning object detection algorithm to locate the center point of the bounding box of the defect hot spot in the image pixel coordinate system; Based on the component positioning coordinates after the vibration correction, a transformation matrix between the image pixel coordinate system and the geographic coordinate system is established to convert the center point of the bounding box into geographic spatial coordinates. The identified defect types, defect image fragments, and converted geospatial coordinates are associated and bound to generate a defect location mapping relationship that includes defect attributes and precise geographical locations.

8. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 7, characterized in that, After generating the defect location mapping relationship containing defect attributes and precise geographical locations, the process also includes: Construct a defect spatiotemporal buffer queue, and sequentially store the defect location mapping relationship generated from multiple consecutive frames of images into the defect spatiotemporal buffer queue; Spatial clustering analysis is performed on the defect data in the defect spatiotemporal buffer queue. Based on the Euclidean distance of the geographic spatial coordinates, defect data that are determined to be the same target are grouped into the same defect cluster. Calculate the weighted centroid of the geospatial coordinates of all defect data within the same defect cluster, where the weights are related to the confidence of defect detection in each frame, to obtain the globally unique geographic coordinates of the defect. By binding the globally unique geographic coordinates with the corresponding defect attributes, a global defect distribution map of the photovoltaic power station is generated, thus completing the deduplication and fusion of duplicate detected defects.

9. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 1, characterized in that, The system also includes: The dynamic benchmark update module is used to verify the correction accuracy based on the defect location mapping relationship. When the positioning deviation exceeds a preset threshold, the benchmark blanking point position and benchmark tilt angle feature are recalibrated, and the updated benchmark is fed back to the geometric benchmark construction module.

10. The intelligent inspection system for photovoltaic power plants based on AI vision according to claim 9, characterized in that, The correction accuracy is verified based on the defect location mapping relationship. When the positioning deviation exceeds a threshold, the reference blanking point position and reference tilt angle feature are recalibrated, and the updated reference is fed back to the geometric reference construction module, including: Extract the geospatial coordinates of the same defect hotspot in multiple adjacent frames, calculate the coordinate dispersion of the same defect hotspot between different frames, and obtain the positioning deviation value; The positioning deviation value is compared with a preset accuracy threshold. If the positioning deviation value is less than or equal to the accuracy threshold, the current reference blanking point position and reference tilt angle characteristics remain unchanged. If the positioning deviation value exceeds the accuracy threshold, it is determined that the current flight attitude has undergone a macroscopic change, triggering a reset mechanism. The latest set of image sequences is collected to recalculate the convergence coordinates and convergence tilt angle, update the reference hidden point position and reference tilt angle features, and feed the updated reference back to the geometric reference construction module for offset calculation of subsequent frames.

Images