An unmanned aerial vehicle-based photovoltaic power station inspection image processing method and system
By constructing an inspection network architecture and dynamic flight control algorithm, combined with visual feature extraction and image quality assessment, the problems of missed inspections and duplicate inspections in traditional UAV inspections have been solved, achieving efficient and accurate photovoltaic power station inspections.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG COMM SERVICES
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional drone-based photovoltaic power station inspections lack multi-dimensional visual feature fusion and dynamic control, resulting in high rates of missed and duplicate inspections, inaccurate image quality assessment, and impact on the accuracy of inspection decisions.
By constructing an inspection network architecture, visual features of photovoltaic panel arrays are extracted, and high-precision visual positioning is achieved by combining SIFT and RANSAC algorithms. Edge detection and texture feature analysis are used to divide the dirt area, a dynamic flight control algorithm is constructed, and an image quality assessment model is built based on convolutional neural networks.
It achieves high-precision visual positioning of key nodes of photovoltaic panels, accurately delineates dirt areas, and dynamically adjusts the flight path, thereby improving the quality and efficiency of inspection.
Smart Images
Figure CN122265878A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual processing technology, specifically to a method and system for image processing of photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs). Background Technology
[0002] With the large-scale development of photovoltaic power plants, the efficiency and accuracy of photovoltaic panel inspection have become key factors restricting operation and maintenance costs. Traditional drone inspection route adjustments rely heavily on preset coordinates and lack visual positioning correction based on real-time images, resulting in high rates of missed and duplicate inspections. The quality of inspection images is judged only by basic indicators such as sharpness and brightness, without being correlated with defect identification effectiveness, making it difficult to guarantee the reliability of subsequent visual analysis and thus affecting the accuracy of inspection decisions.
[0003] Existing technologies lack an integrated solution for multi-dimensional visual feature fusion, dynamic control of visual guidance, and closed-loop optimization of recognition effect for photovoltaic panel arrays, resulting in the potential of visual technology not being fully explored.
[0004] Therefore, a method and system for image processing of photovoltaic power plant inspection based on UAVs are provided. Summary of the Invention
[0005] To address the aforementioned technical problems, the present invention aims to provide an image processing method and system for photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs).
[0006] To achieve the above objectives, the present invention provides the following technical solution: an image processing method for photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs), the method comprising: Image processing is performed on the initial inspection video of the corresponding photovoltaic power station to construct an inspection network architecture; based on the visual characteristics of key inspection nodes in the inspection network architecture, an initial inspection route is generated based on the positional relationship between the reference node and the key inspection nodes. Visual features are extracted from the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then an adaptive inspection route algorithm is constructed; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. A dynamic flight control algorithm is constructed by combining flight data collected in real time by UAVs and image data of photovoltaic panel surfaces. Based on the dynamic flight control algorithm and the inspection route optimization algorithm, the corresponding photovoltaic panel surface image data is obtained, and then an inspection image quality assessment model is constructed. The inspection image quality assessment model is used to evaluate the quality of photovoltaic panel surface image data collected in real time during the inspection process, so as to continuously improve the inspection quality and efficiency.
[0007] Furthermore, the process of image processing of the initial inspection videos of the corresponding photovoltaic power stations to construct the inspection network architecture includes: By pre-setting a custom initial reconnaissance route for the drone, initial inspection videos of the corresponding photovoltaic power station are obtained; point cloud data is acquired based on the actual layout data of the corresponding photovoltaic power station. Based on the preset illumination compensation algorithm, the initial inspection video is radiometrically corrected, and based on the ICP algorithm and point cloud data, the radiometrically corrected initial inspection video is geometrically corrected. Based on the semantic segmentation algorithm, the undulation feature image data of the photovoltaic panel array area, the initial position image data, the basic position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data of the initial inspection video after geometric correction are extracted. Based on the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data, obtain the image data obtained from the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data. Each inspection node and The inspection network architecture is composed of edges.
[0008] Furthermore, according to claim 2, the method for processing images of photovoltaic power plant inspection based on unmanned aerial vehicles is characterized in that the process of geometric correction of the initial inspection video after radiometric correction is as follows: Point cloud data is used as reference point cloud data, and corresponding source point cloud data is generated from the radiometrically corrected initial inspection video based on the reference point cloud data. Based on the source point cloud data and the reference point cloud data, a set of point pairs is obtained, and then a geometric correction target is set; based on the geometric correction target, the geometric correction result is determined.
[0009] Furthermore, the process of determining the positional relationship between the baseline node and the key inspection node is as follows: Based on the three-dimensional coordinates of the baseline node and the key inspection node, calculate the three-dimensional Euclidean distance between the key inspection node and the baseline node; select the key inspection node with the smallest three-dimensional Euclidean distance as the nearest key inspection node; and obtain the image data corresponding to the nearest key inspection node. Based on the intrinsic and extrinsic parameter matrices of the UAV camera, a projection matrix is obtained, and then the pixel coordinates of the reference node in the initial position image data and the pixel coordinates of the nearest key inspection node are obtained; a cropping radius is defined, and the corresponding pixel coordinates are used as the center, and cropping is performed according to the cropping radius to obtain the image regions of the reference node and the image regions of the key inspection nodes. Based on the SIFT algorithm, an initial set of matching point pairs is obtained for the image regions of the reference node and the key inspection node. Based on the RANSAC algorithm, the initial set of matching point pairs is eliminated to obtain a set of reliable matching point pairs. The average displacement vector of all reliable matching point pairs is calculated to determine the positional relationship between the reference node and the key inspection node.
[0010] Furthermore, the process of visual feature extraction for the corresponding photovoltaic power station's undulation feature image data is as follows: The corresponding points of photovoltaic panels between adjacent photovoltaic panel arrays in the undulation feature image data are obtained. According to the projection matrix, the pixel coordinates corresponding to different photovoltaic panel corresponding points between adjacent photovoltaic panel arrays in the undulation feature image data are obtained. The average disparity of the pixel coordinates corresponding to different photovoltaic panel corresponding points in the undulation feature image data is taken. According to the average disparity, the three-dimensional height between adjacent photovoltaic panel arrays is obtained, and then the three-dimensional height distribution between all adjacent photovoltaic panel arrays of the photovoltaic power station is obtained.
[0011] Furthermore, the process of combining the flight data collected in real time by the UAV and the image data of the photovoltaic panel surface to construct the dynamic flight control algorithm is as follows: The method acquires real-time flight data collected by UAVs and image data of photovoltaic panel surfaces. Based on gray-level co-occurrence matrix technology, it acquires a texture feature set of real-time acquired photovoltaic panel surface image data, including: calculating the gray-level co-occurrence matrix around each pixel of the photovoltaic panel surface image data by means of a sliding window, and extracting a series of texture feature values according to the gray-level co-occurrence matrix to construct a texture feature set. Based on an edge detection algorithm, the edge points between dirt and normal surfaces in the photovoltaic panel surface image data are determined. A sliding window is divided with these edge points as the center. Based on the texture feature set of the corresponding edge points in the photovoltaic panel surface image data, texture feature vectors within the window and baseline feature vectors of the normal areas in the photovoltaic panel surface image data are extracted. The corresponding cosine similarity is calculated based on the texture feature vectors and the baseline feature vectors. The corresponding region type is determined based on a preset region similarity threshold and cosine similarity. The pixel area of all dirt areas is counted and converted into physical area, which is recorded as the dirt area on the photovoltaic panel surface. Then, a dynamic flight control algorithm is constructed based on the dirt area on the photovoltaic panel surface.
[0012] Furthermore, the process of quality assessment of the photovoltaic panel surface image data collected in real time during the inspection includes: Acquire several sets of photovoltaic panel surface image data from historical acquisition periods, as well as standard photovoltaic panel surface image data; A training sample set was constructed based on photovoltaic panel surface image data from several historical acquisition periods and standard photovoltaic panel surface image data. A standard evaluation model is constructed based on convolutional neural networks. The training sample set is input into the standard evaluation model to train it, and the trained standard evaluation model is denoted as the inspection image quality evaluation model. The quality of all photovoltaic panel surface image data collected in real time during the same inspection is evaluated according to the inspection image quality evaluation model to obtain the corresponding evaluation results. The parameters of the inspection route optimization algorithm and the dynamic flight control algorithm are optimized according to the evaluation results.
[0013] A second aspect of the present invention also provides an image processing system for photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs), comprising: a data acquisition module, an image processing module, a visual secondary extraction module, a data fusion module, and an inspection optimization module; The data acquisition module is used to collect initial inspection videos of the corresponding photovoltaic power station, drone flight data, and photovoltaic panel surface image data. The image processing module is used to process the initial inspection video of the corresponding photovoltaic power station to construct the inspection network architecture; based on the visual characteristics of the key inspection nodes in the inspection network architecture, and based on the positional relationship between the reference node and the key inspection nodes, an initial inspection route is generated. The visual secondary extraction module is used to perform secondary visual feature extraction on the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then construct an adaptive inspection route algorithm; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. The data fusion module is used to combine and process the flight data collected in real time by the UAV and the image data of the photovoltaic panel surface to build a dynamic flight control algorithm. The inspection optimization module is used to obtain corresponding photovoltaic panel surface image data based on the dynamic flight control algorithm and the inspection route optimization algorithm, and then construct an inspection image quality assessment model; based on the inspection image quality assessment model, the quality of the photovoltaic panel surface image data collected in real time during the inspection process is assessed to continuously improve the inspection quality and efficiency.
[0014] Compared with the prior art, the beneficial effects of the present invention are: based on the semantic segmentation algorithm, the undulation features, edge and corner position image data of the photovoltaic panel array are extracted, and the SIFT feature matching and RANSAC algorithm are combined to remove mismatched points, thereby realizing high-precision visual positioning of key nodes of the photovoltaic panel; through edge detection and texture feature analysis, the boundary between dirt and normal areas is accurately delineated.
[0015] The three-dimensional height distribution of photovoltaic panels is extracted by visual features, and the height grid division and inspection height are automatically adjusted by combining image parallax calculation. The dynamic flight control algorithm is based on the cosine similarity between texture feature vector and reference vector to determine the region type in real time and link the flight speed and focal length adjustment.
[0016] Based on convolutional neural networks, an inspection image quality assessment model is constructed, which correlates image quality with defect recognition accuracy, and optimizes the inspection route and flight parameters in reverse based on the assessment results. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0018] Figure 1 This is a schematic diagram illustrating the steps of an image processing method for inspecting a photovoltaic power station based on unmanned aerial vehicles (UAVs).
[0019] Figure 2 This is a schematic diagram of a module for an image processing system for inspecting photovoltaic power plants based on unmanned aerial vehicles (UAVs). Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0022] like Figure 1 As shown, a method for image processing in photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs) includes the following steps: Step S1: Perform image processing on the initial inspection video of the corresponding photovoltaic power station to construct an inspection network architecture; based on the visual characteristics of key inspection nodes in the inspection network architecture, generate an initial inspection route based on the positional relationship between the reference node and the key inspection nodes. Step S2: Visual feature extraction is performed on the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then an adaptive inspection route algorithm is constructed; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. Step S3: Combine the flight data collected in real time by the UAV with the image data of the photovoltaic panel surface to construct a dynamic flight control algorithm; Step S4: Based on the dynamic flight control algorithm and the inspection route optimization algorithm, obtain the corresponding photovoltaic panel surface image data, and then construct an inspection image quality assessment model; Step S5: Evaluate the quality of the photovoltaic panel surface image data collected in real time during the inspection process according to the inspection image quality assessment model, so as to continuously improve the inspection quality and efficiency.
[0023] It should be further explained that, in the specific implementation process, the image processing of the initial inspection video of the corresponding photovoltaic power station, and the subsequent construction of the inspection network architecture, includes: Optionally, in this embodiment of the application, an initial inspection video of the corresponding photovoltaic power station is obtained by pre-setting a custom initial reconnaissance route of the drone; point cloud data is obtained based on the actual layout data of the corresponding photovoltaic power station; it should be noted that the actual layout data includes, but is not limited to, the location data of the photovoltaic power station, the arrangement of the photovoltaic panel array, the tilt angle of the photovoltaic panel array, the spacing of the photovoltaic panel array, the size of the photovoltaic panel, the height of the photovoltaic panel support, etc. Based on a preset illumination compensation algorithm, radiometric correction is performed on the initial inspection video. Then, based on the ICP algorithm and point cloud data, geometric correction is performed on the radiometrically corrected initial inspection video. Specifically: The point cloud data is used as the reference point cloud data and is denoted as follows: Based on reference point cloud data The radiometrically corrected initial inspection video is used to generate corresponding source point cloud data, denoted as . ; Define the initial rigid body transformation matrix, which is usually the identity matrix, i.e. ; For source cloud data Each point In reference point cloud data Find its nearest neighbors This forms a set of corresponding point pairs. ; Set geometric correction target ;in, This represents a rotation matrix used to control direction and orientation; This represents the translation vector, used to control position offset.
[0024] If the geometric correction target Less than or equal to the preset geometric correction target threshold If so, it is assumed that the initial inspection video has completed geometric correction; If the geometric correction target Greater than the preset geometric correction target threshold If the initial inspection video has not been geometrically corrected, then it will be obtained... Applied to source point cloud data Update its pose: until the geometric correction target is met. Less than or equal to the preset geometric correction target threshold ; It should be noted that the aforementioned ICP algorithm provides geometrically consistent input data for subsequent visual feature analysis by achieving precise alignment between the image and the real geographic space; the aforementioned radiometric correction refers to eliminating illumination differences.
[0025] Based on a semantic segmentation algorithm, image data of undulation features, initial position, base position, edge position, and corner position of the photovoltaic panel array are extracted from the geometrically corrected initial inspection video. It should be noted that the undulation feature image data refers to image data showing height variations in different areas of the photovoltaic panel array; the base position photovoltaic panel image data refers to the image data of each photovoltaic panel in the middle position of the array; the edge position photovoltaic panel image data refers to the image data of each photovoltaic panel at the edge position of the array; and the corner position photovoltaic panel image data refers to the image data of each photovoltaic panel at the corner position of the array.
[0026] Based on the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data, obtain the image data obtained from the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data. Each inspection node and Inspection network architecture composed of edges ;in, , represents the set of all inspection nodes; , represents the set of all edges; the inspection nodes include, but are not limited to, basic inspection nodes, key inspection nodes, and benchmark nodes.
[0027] It should be noted that the center of the photovoltaic panel corresponding to the basic position photovoltaic panel image data is set as the basic inspection node, and the center of the photovoltaic panel corresponding to the edge position photovoltaic panel image data and the corner position photovoltaic panel image data is set as the key inspection node. The key inspection nodes include edge key inspection nodes and corner key inspection nodes; the UAV cockpit position corresponding to the initial position image data is used as the reference node.
[0028] To further clarify, the photovoltaic panel array includes all the photovoltaic panels of the photovoltaic power station; when the photovoltaic panel array has only one row or one column of photovoltaic panels, the corresponding key inspection node is recorded as the corner key inspection node.
[0029] It should be further explained that, in the specific implementation process, based on the visual characteristics of key inspection nodes in the inspection network architecture, and based on determining the positional relationship between the baseline node and the key inspection nodes, the specific process of generating the initial inspection route includes: Optionally, in this embodiment, the preset inspection constraints are as follows: taking a reference node as the starting point, the visual features of the starting point and the nearest key inspection node are matched using the image data corresponding to the nearest key inspection node to determine the positional relationship between the reference node and the key inspection node. Specifically: The three-dimensional coordinates of the reference node are defined as follows: The three-dimensional coordinates of key inspection nodes are marked as follows: ; Calculate the three-dimensional Euclidean distance between the key inspection node and the reference node. ; The key inspection node corresponding to the minimum three-dimensional Euclidean distance is selected as the nearest key inspection node; and the image data corresponding to the nearest key inspection node is used. Construct the projection matrix based on the intrinsic and extrinsic parameter matrices of the UAV camera. Based on the initial position image data corresponding to the reference node and the image data corresponding to the nearest key inspection node, the corresponding pixel coordinates are calculated; the pixel coordinates ; This allows us to obtain the pixel coordinates of the reference node in the initial position image data. And the pixel coordinates of the most recent key inspection nodes ; Define the clipping radius In pixel coordinates and pixel coordinates The centers are respectively, based on the cutting radius. The corresponding image data is cropped to obtain the reference node image region. and key inspection node image areas ; Based on the SIFT algorithm, detect the reference node image region. and key inspection node image areas The key point set in the process is calculated, and the baseline node descriptor and key inspection node descriptor for the corresponding key points are calculated. Calculate the Euclidean distance between all key point pairs in the baseline node descriptor and the key inspection node descriptor for the baseline node image region. The set of key points in the image, obtain the corresponding key points. Initial set of matching point pairs; based on the RANSAC algorithm, outliers in the initial set of matching point pairs are removed to obtain a reliable set of matching point pairs; It should be noted that the RANSAC algorithm mentioned above is a classic algorithm in computer vision and machine learning used to robustly estimate model parameters from noisy data; its core objective is to find the correct model supported by the majority of data points in data with a large number of outliers. Furthermore, in the scenario of image registration and feature matching in UAV photovoltaic inspection, the RANSAC algorithm is used to remove abnormal matching point pairs to ensure the accuracy of the transformation matrix.
[0030] Calculate the displacement vector of each matching point pair in the set of reliable matching point pairs. Then, the average displacement vector of all reliable matching point pairs is calculated. ; like Then the positional relationship between the benchmark node and the key inspection node is determined to be a horizontal relationship; if If so, the positional relationship between the benchmark node and the key inspection node is determined to be a vertical relationship.
[0031] If the positional relationship is lateral, then the lateral direction is set as the initial inspection direction. This continues until the system moves along the initial inspection direction to the critical corner inspection node of the corresponding photovoltaic array, at which point the length of the inspection node is recorded. The drone turns left or right. Then move in the direction of turning. After the length of one inspection node reaches another critical corner inspection node, the drone turns left or right. And move according to the inspection constraints. After the length of one inspection node reaches another critical corner inspection node, the drone turns left or right. Then move in the direction of turning. The drone turns left or right when it reaches the edge critical inspection node adjacent to the critical inspection node at the corner. ,move After completing the required number of inspection nodes, upon reaching the corresponding basic inspection node, the drone turns left or right. ,move The drone turns left or right upon reaching the adjacent basic inspection node, based on the length of each inspection node. ,move After moving one inspection node length, reach the edge critical inspection node adjacent to the previous edge critical inspection node, and repeat the above process until the total length of each inspection node movement equals [the length of the previous movement]. The length of each inspection node, if If the number of inspection nodes is even, the inspection process ends directly, and the drone flies back to the base node; if... If the length of the inspection nodes is odd, the drone will turn left or right. After inspecting the remaining basic inspection nodes, the drone flies back to the base node and is recorded as the first initial inspection route. If the positional relationship is a vertical relationship, the corresponding inspection process in the first initial inspection route is executed. The specific inspection process will not be described in detail and is recorded as the second initial inspection route. The first initial inspection route and the second initial inspection route are denoted as the initial inspection route.
[0032] It should be noted that the inspection constraints include initial inspection height conditions, unique conditions for critical inspection nodes, and unique conditions for basic inspection nodes. The initial inspection height condition is set based on the highest photovoltaic panel support height in the photovoltaic array and a preset safe flight altitude threshold. The unique condition for critical inspection nodes means that all critical inspection nodes are allowed to be inspected only once. The unique condition for basic inspection nodes means that all basic inspection nodes are allowed to be inspected only once.
[0033] It should be further explained that, in the specific implementation process, the process of extracting visual features from the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then constructing the adaptive algorithm for the inspection route, includes: Optionally, in this embodiment of the application, the inspection route adaptive algorithm includes an inspection height adjustment algorithm and an inspection route adjustment algorithm; The inspection height adjustment algorithm specifically involves: extracting visual features from the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, specifically: Obtain the corresponding points of photovoltaic panels between adjacent photovoltaic panel arrays in the undulation feature image data, and then, based on the projection matrix... The pixel coordinates of corresponding points on different photovoltaic panels between adjacent photovoltaic panel arrays are obtained from the undulation feature image data; and the average disparity of the pixel coordinates of corresponding points on different photovoltaic panels in the undulation feature image data is taken. ;in, This indicates the horizontal pixel coordinate of one of the photovoltaic panels in the photovoltaic array; This indicates the horizontal pixel coordinates of the corresponding photovoltaic panel in another photovoltaic panel array; According to the average disparity Obtain the three-dimensional height between adjacent photovoltaic panel arrays. ;in, This indicates the camera focal length preset for the initial reconnaissance route of the drone, in pixels. This indicates the physical distance between the optical centers of the drone cameras; This represents the average disparity, expressed in pixels. It should be noted that the corresponding points of the photovoltaic panels refer to the corresponding edge points between adjacent photovoltaic panel arrays.
[0034] Based on the calculation formula for the three-dimensional height between adjacent photovoltaic panel arrays, the three-dimensional height distribution between all adjacent photovoltaic panel arrays in the photovoltaic power station is obtained; Based on the three-dimensional height distribution and the safe flight height threshold, the actual layout of the corresponding photovoltaic array is divided into different height grids in the vertical height. The height grids corresponding to photovoltaic panels at different heights are taken sequentially in the initial inspection direction. The center of the photovoltaic panels corresponding to two adjacent height grids is taken as the reference grid point. The reference grid points of two adjacent height grids are connected, and the height difference between the connected points is recorded as the pre-adjustment inspection height. The inspection height is automatically adjusted according to the pre-adjustment inspection height.
[0035] It should be noted that, in the embodiments of this application, the photovoltaic panels at different heights refer to the relative height of the photovoltaic panels with respect to the horizontal direction. The width of the height grid refers to the median value of the spacing between two columns of photovoltaic panels; the height of the height grid refers to the sum of the relative height of the photovoltaic panels with respect to the horizontal direction and the safe flight altitude threshold.
[0036] To further clarify, the inspection height of the highest photovoltaic panel is fixed at the safe flight altitude threshold.
[0037] The inspection route adjustment algorithm is as follows: the photovoltaic panel array is classified according to different three-dimensional height distributions, and the photovoltaic panel array is divided into several photovoltaic panel sub-arrays. The center of each photovoltaic panel at the edge and corner of each photovoltaic panel array is re-set as a key inspection node. The first inspection node of the photovoltaic panel array is recorded as the initial inspection node, and the last inspection node of the photovoltaic panel array is recorded as the end inspection node. The initial inspection route is adjusted based on the initial inspection node and the end inspection node.
[0038] It should be noted that the reconfigured critical inspection nodes also include the corresponding edge critical inspection nodes and corner critical inspection nodes.
[0039] It should be further explained that, in the specific implementation process, the process of adaptively adjusting the initial inspection route according to the aforementioned inspection route adaptive algorithm includes: Optionally, in this embodiment of the application, the initial inspection route is adaptively adjusted based on the inspection route adjustment algorithm in the inspection route adaptive algorithm, and the optimal positional relationship between the reference node and the initial inspection node of the photovoltaic panel array closest to the reference node in the initial inspection route is determined. If the optimized positional relationship is lateral, then the straight-line connection direction between the reference node and the corresponding initial inspection node of the photovoltaic panel array is recorded as the inspection optimization direction. After reaching the initial inspection node, the UAV turns left or right to optimize the inspection deviation angle and moves to the corner key inspection node of the corresponding photovoltaic panel array, recording the length of the inspection node as... , The drone turns left or right. Then move in the direction of turning. After the length of one inspection node reaches another critical corner inspection node, among which, The drone turns left or right. And move according to the inspection constraints. After the length of one inspection node reaches another critical corner inspection node, the drone turns left or right. Then move in the direction of turning. The drone turns left or right when it reaches the edge critical inspection node adjacent to the critical inspection node at the corner. ,move After completing the required number of inspection nodes, upon reaching the corresponding basic inspection node, the drone turns left or right. ,move The drone turns left or right upon reaching the adjacent basic inspection node, based on the length of each inspection node. ,move After moving one inspection node length, reach the edge critical inspection node adjacent to the previous edge critical inspection node, and repeat the above process until the total length of each inspection node movement equals [the length of the previous movement]. The length of each inspection node, if If the number of inspection nodes is even, the inspection of the corresponding photovoltaic panel array ends, and the end inspection node of the corresponding photovoltaic panel array is taken as the new reference node and recorded as the temporary reference node; if If the length of the inspection nodes is odd, the drone will turn left or right. After inspecting the remaining basic inspection nodes, the end inspection node of the corresponding photovoltaic panel array is taken as the new reference node and recorded as the temporary reference node. Optionally, in this embodiment of the application, the optimal positional relationship between the temporary reference node and the initial inspection node of the photovoltaic panel array closest to the temporary reference node that has not been inspected is determined, and the pre-adjusted inspection height between the corresponding photovoltaic panel arrays is obtained based on the inspection height adjustment algorithm in the inspection route adaptive algorithm, so as to realize the adjustment of the inspection height of the UAV, and repeat the inspection process of the inspected photovoltaic panel arrays mentioned above until all photovoltaic panel arrays have been inspected.
[0040] It should be noted that the inspection optimization deviation angle refers to the angle between the inspection optimization direction and the corresponding optimization position.
[0041] If the optimized positional relationship is a vertical relationship, the above-mentioned inspection process of the photovoltaic panel array is executed. The specific inspection process will not be described in detail.
[0042] It should be further explained that, in the specific implementation process, the real-time flight data collected by the UAV and the image data of the photovoltaic panel surface are combined and processed to construct a dynamic flight control algorithm; the specific process includes: Optionally, in this embodiment of the application, flight data collected in real time by the UAV and image data of the photovoltaic panel surface are obtained, and then a dynamic flight control algorithm is constructed; the dynamic flight control algorithm includes a dynamic focus adjustment model and a flight speed control model; It should be noted that the flight data includes, but is not limited to, real-time wind speed, real-time flight speed, etc.; the photovoltaic panel surface image data is image data after geometric correction processing. Optionally, in this embodiment of the application, based on an edge detection algorithm, edge calculation is performed on the real-time acquired photovoltaic panel surface image data to obtain the outline of the dirt on the photovoltaic panel surface, and then the area of dirt on the photovoltaic panel surface is obtained, specifically as follows: Based on gray-level co-occurrence matrix (GLCM) technology, a texture feature set is obtained from real-time acquired photovoltaic (PV) panel surface image data. This includes: calculating the GLCM around each pixel of the PV panel surface image data using a sliding window approach. Specific details include quantizing the gray levels of the PV panel surface image data into discrete gray levels, dividing the gray value range into several levels (e.g., 16, 32, and 64 levels for an 8-bit image), and defining the parameters required for the GLCM, including distance (d) and orientation (θ), which are determined according to actual application requirements. The specific steps are not detailed here. Subsequently, based on the calculated GLCM, a series of texture feature values are extracted to construct a texture feature set. The texture feature values in the texture feature set include, but are not limited to: Contrast ratio: A statistical feature describing the contrast ratio of pixels at different gray levels in an image. It is a feature that measures the roughness of the image texture and reflects the sharpness of the image and the depth of the texture grooves. The deeper the texture grooves, the greater the contrast ratio and the clearer the visual effect. Conversely, the lower the contrast ratio, the shallower the grooves and the blurrier the effect. Correlation: Describes the degree of correlation between pixels of different gray levels in an image. It measures the similarity of elements in the spatial gray-level co-occurrence matrix in the row or column direction. Therefore, the magnitude of the correlation value reflects the local gray-level correlation in the image. When the matrix element values are uniformly equal, the correlation value is large. Conversely, if the matrix pixel values differ greatly, the correlation value is small. If there is horizontal texture in the image, the correlation of the horizontal matrix is greater than the correlation of the other matrices. Energy: Describes the uniformity of pixel grayscale distribution in an image, measures the randomness contained in the image, and represents the complexity of the image. Entropy is maximized when all values of the co-occurrence matrix are equal or when pixel values exhibit the greatest randomness. Homogeneity: describes the similarity of gray levels between adjacent pixels in an image; it reflects the homogeneity of image texture and measures the amount of local variation in image texture. A large value indicates that there is little variation between different regions of image texture and that the local texture is very uniform. Entropy: Describes the degree of uncertainty in image texture, measures the randomness contained in the image, and represents the complexity of the image. Entropy is at its maximum when all values in the co-occurrence matrix are equal or when the pixel values exhibit the greatest randomness. Inverse variance reflects the clarity and regularity of the texture; textures are clear and have strong regularity.
[0043] It should be further explained that, in the specific implementation process, the specific process of obtaining the corresponding dirt area on the photovoltaic panel surface based on the edge detection algorithm and the texture feature set of each pixel in the photovoltaic panel surface image data includes: Based on an edge detection algorithm, the edge points between dirt and normal surfaces in the photovoltaic panel surface image data are determined. A sliding window is then divided around these edge points, and texture feature vectors within the window are extracted based on the texture feature sets of the corresponding edge points in the photovoltaic panel surface image data. and the baseline feature vector of the normal area in the photovoltaic panel surface image data. ; According to the texture feature vector and baseline feature vector Calculate the corresponding cosine similarity. ; Based on the preset regional similarity threshold ,like The area is determined to be a dirty area; if If so, the area is determined to be a normal area; The pixel area of all dirt areas is counted and converted into physical area, which is recorded as the dirt area on the photovoltaic panel surface.
[0044] Based on the real-time wind speed and the area of dirt on the photovoltaic panel surface, a flight speed control model is constructed. The flight speed control model is as follows: ;in, Indicates real-time flight speed; Indicates the base flight speed; This represents the dirt correction factor, which is obtained by technicians through experiments. This indicates the area of dirt on the surface of the photovoltaic panel; This represents the total area of the corresponding photovoltaic panel surface image data; This represents the wind speed correction factor, which is obtained by technicians through experiments. Indicates real-time wind speed; This indicates the maximum wind speed threshold for safe flight of a drone.
[0045] It should be noted that the basic flight speed The standard speed is preset by technicians, and the typical range is... Set according to the actual situation.
[0046] Optionally, in this embodiment of the application, a dynamic focus adjustment model is constructed based on the pre-adjusted inspection height and real-time wind speed. The dynamic focus adjustment model is as follows: ; This indicates dynamic adjustment of the focal length; Indicates the basic focal length; This represents the height deviation correction factor, which is obtained by technicians through experiments. This indicates the deviation between the actual flight inspection altitude and the pre-adjusted inspection altitude; Indicates the pre-adjustment inspection height; This represents the wind speed correction factor, which is obtained by technicians through experiments. Indicates real-time wind speed; This indicates the maximum wind speed threshold for safe flight of a drone; This represents the speed deviation correction factor, which is obtained by technicians through experiments. Indicates real-time flight speed; This indicates the base flight speed.
[0047] Based on the dynamic flight control algorithm and the inspection route optimization algorithm, the UAV is dynamically controlled to collect real-time image data of the photovoltaic panel surface.
[0048] It should be further explained that, in the specific implementation process, based on the dynamic flight control algorithm and the inspection route optimization algorithm, the corresponding photovoltaic panel surface image data is obtained, and then an inspection image quality assessment model is constructed. The specific process of assessing the quality of the photovoltaic panel surface image data collected in real time during the inspection includes: Optionally, in this embodiment of the application, several sets of photovoltaic panel surface image data from historical acquisition periods and standard photovoltaic panel surface image data are acquired; The specific process of constructing a training sample set based on photovoltaic panel surface image data from several historical acquisition periods and standard photovoltaic panel surface image data includes: Several sets of photovoltaic panel surface image data from historical acquisition periods, as well as standard photovoltaic panel surface image data, are grouped and labeled, denoted as . It is a natural number; Will Photovoltaic panel surface image data from historical acquisition periods and standard photovoltaic panel surface image data were used as sample data, and Less than The natural numbers, and using the sample data, the mean of the sample data is obtained, denoted as the sample set; The photovoltaic panel surface image data from the remaining historical acquisition periods and the standard photovoltaic panel surface image data are used as the test set; a training sample set is formed based on the sample set and the test set. A standard evaluation model is constructed based on convolutional neural networks. The training sample set is input into the standard evaluation model to train it, and the trained standard evaluation model is denoted as the inspection image quality evaluation model. The quality of all photovoltaic panel surface image data collected in real time during the same inspection is evaluated according to the inspection image quality evaluation model to obtain the corresponding evaluation results. The parameters of the inspection route optimization algorithm and the dynamic flight control algorithm are optimized according to the evaluation results. The specific optimization process is not described in this application.
[0049] like Figure 2 As shown, a photovoltaic power station inspection image processing system based on UAVs is disclosed. The system includes: a data acquisition module, an image processing module, a visual secondary extraction module, a data fusion module, and an inspection optimization module. The data acquisition module is used to collect initial inspection videos of the corresponding photovoltaic power station, drone flight data, and photovoltaic panel surface image data. The image processing module is used to process the initial inspection video of the corresponding photovoltaic power station to construct the inspection network architecture; based on the visual characteristics of the key inspection nodes in the inspection network architecture, and based on the positional relationship between the reference node and the key inspection nodes, an initial inspection route is generated. The visual secondary extraction module is used to perform secondary visual feature extraction on the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then construct an adaptive inspection route algorithm; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. The data fusion module is used to combine and process the flight data collected in real time by the UAV and the image data of the photovoltaic panel surface to build a dynamic flight control algorithm. The inspection optimization module is used to obtain corresponding photovoltaic panel surface image data based on the dynamic flight control algorithm and the inspection route optimization algorithm, and then construct an inspection image quality assessment model; based on the inspection image quality assessment model, the quality of the photovoltaic panel surface image data collected in real time during the inspection process is assessed to continuously improve the inspection quality and efficiency.
[0050] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0051] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0052] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.
[0053] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0054] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.
[0055] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0056] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0057] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for image processing in photovoltaic power plant inspection based on unmanned aerial vehicles (UAVs), characterized in that, The method includes: Image processing is performed on the initial inspection video of the corresponding photovoltaic power station to construct an inspection network architecture; based on the visual characteristics of key inspection nodes in the inspection network architecture, an initial inspection route is generated based on the positional relationship between the reference node and the key inspection nodes. Visual features are extracted from the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then an adaptive inspection route algorithm is constructed; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. A dynamic flight control algorithm is constructed by combining flight data collected in real time by UAVs and image data of photovoltaic panel surfaces. Based on the dynamic flight control algorithm and the inspection route optimization algorithm, the corresponding photovoltaic panel surface image data is obtained, and then an inspection image quality assessment model is constructed. The inspection image quality assessment model is used to evaluate the quality of photovoltaic panel surface image data collected in real time during the inspection process, so as to continuously improve the inspection quality and efficiency.
2. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of image processing of the initial inspection videos of the corresponding photovoltaic power station, and then constructing the inspection network architecture, includes: By pre-setting a custom initial reconnaissance route for the drone, initial inspection videos of the corresponding photovoltaic power station are obtained; point cloud data is acquired based on the actual layout data of the corresponding photovoltaic power station. Based on the preset illumination compensation algorithm, the initial inspection video is radiometrically corrected, and based on the ICP algorithm and point cloud data, the radiometrically corrected initial inspection video is geometrically corrected. Based on the semantic segmentation algorithm, the undulation feature image data of the photovoltaic panel array area, the initial position image data, the basic position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data of the initial inspection video after geometric correction are extracted. Based on the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data, obtain the image data obtained from the initial position image data, the base position photovoltaic panel image data, the edge position photovoltaic panel image data, and the corner position photovoltaic panel image data. Each inspection node and The inspection network architecture is composed of edges.
3. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, The process of performing geometric correction on the initial inspection video after radiometric correction is as follows: Point cloud data is used as reference point cloud data, and corresponding source point cloud data is generated from the radiometrically corrected initial inspection video based on the reference point cloud data. Based on the source point cloud data and the reference point cloud data, a set of point pairs is obtained, and then a geometric correction target is set; based on the geometric correction target, the geometric correction result is determined.
4. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The process of determining the positional relationship between the baseline node and the key inspection node is as follows: Calculate the three-dimensional Euclidean distance between the key inspection node and the benchmark node based on the three-dimensional coordinates of the benchmark node and the key inspection node. Select the critical inspection node corresponding to the minimum three-dimensional Euclidean distance as the nearest critical inspection node; And acquire the image data corresponding to the most recent key inspection node; Based on the intrinsic and extrinsic parameter matrices of the UAV camera, the projection matrix is obtained, and then the pixel coordinates of the reference node in the initial position image data and the pixel coordinates of the nearest key inspection node are obtained. Define a cropping radius and use the corresponding pixel coordinates as the center. Crop according to the cropping radius to obtain the reference node image region and the key inspection node image region. Based on the SIFT algorithm, an initial set of matching point pairs is obtained for the image regions of the reference node and the key inspection node. Based on the RANSAC algorithm, the initial set of matching point pairs is eliminated to obtain a set of reliable matching point pairs. The average displacement vector of all reliable matching point pairs is calculated to determine the positional relationship between the reference node and the key inspection node.
5. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The process of visual feature extraction for the undulation feature image data of the corresponding photovoltaic power station is as follows: The corresponding points of photovoltaic panels between adjacent photovoltaic panel arrays in the undulation feature image data are obtained. According to the projection matrix, the pixel coordinates corresponding to different photovoltaic panel corresponding points between adjacent photovoltaic panel arrays in the undulation feature image data are obtained. The average disparity of the pixel coordinates corresponding to different photovoltaic panel corresponding points in the undulation feature image data is taken. According to the average disparity, the three-dimensional height between adjacent photovoltaic panel arrays is obtained, and then the three-dimensional height distribution between all adjacent photovoltaic panel arrays of the photovoltaic power station is obtained.
6. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 5, characterized in that, The process of combining and processing real-time flight data collected by the UAV and image data of the photovoltaic panel surface to construct a dynamic flight control algorithm is as follows: The method acquires real-time flight data collected by UAVs and image data of photovoltaic panel surfaces. Based on gray-level co-occurrence matrix technology, it acquires a texture feature set of real-time acquired photovoltaic panel surface image data, including: calculating the gray-level co-occurrence matrix around each pixel of the photovoltaic panel surface image data by means of a sliding window, and extracting a series of texture feature values according to the gray-level co-occurrence matrix to construct a texture feature set. Based on an edge detection algorithm, the edge points between dirt and normal surfaces in the photovoltaic panel surface image data are determined. A sliding window is divided with these edge points as the center. Based on the texture feature set of the corresponding edge points in the photovoltaic panel surface image data, texture feature vectors within the window and baseline feature vectors of the normal areas in the photovoltaic panel surface image data are extracted. The corresponding cosine similarity is calculated based on the texture feature vectors and the baseline feature vectors. The corresponding region type is determined based on a preset region similarity threshold and cosine similarity. The pixel area of all dirt areas is counted and converted into physical area, which is recorded as the dirt area on the photovoltaic panel surface. Then, a dynamic flight control algorithm is constructed based on the dirt area on the photovoltaic panel surface.
7. The image processing method for photovoltaic power station inspection based on unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The process of quality assessment of photovoltaic panel surface image data collected in real time during inspection includes: Acquire several sets of photovoltaic panel surface image data from historical acquisition periods, as well as standard photovoltaic panel surface image data; A training sample set was constructed based on photovoltaic panel surface image data from several historical acquisition periods and standard photovoltaic panel surface image data. A standard evaluation model is constructed based on convolutional neural networks. The training sample set is input into the standard evaluation model to train it, and the trained standard evaluation model is denoted as the inspection image quality evaluation model. The quality of all photovoltaic panel surface image data collected in real time during the same inspection is evaluated according to the inspection image quality evaluation model to obtain the corresponding evaluation results. The parameters of the inspection route optimization algorithm and the dynamic flight control algorithm are optimized according to the evaluation results.
8. A photovoltaic power plant inspection image processing system based on unmanned aerial vehicles (UAVs), implementing the UAV-based photovoltaic power plant inspection image processing method described in any one of claims 1 to 7, characterized in that, include: The system includes a data acquisition module, an image processing module, a visual secondary extraction module, a data fusion module, and an inspection optimization module. The data acquisition module is used to collect initial inspection videos of the corresponding photovoltaic power station, drone flight data, and photovoltaic panel surface image data. The image processing module is used to process the initial inspection video of the corresponding photovoltaic power station and then construct the inspection network architecture. Based on the visual characteristics of key inspection nodes in the inspection network architecture, and by judging the positional relationship between the baseline node and the key inspection nodes, an initial inspection route is generated. The visual secondary extraction module is used to perform secondary visual feature extraction on the undulation feature image data of the corresponding photovoltaic power station to obtain the three-dimensional height distribution of the corresponding photovoltaic panel array, and then construct an adaptive inspection route algorithm; the initial inspection route is adaptively adjusted according to the adaptive inspection route algorithm. The data fusion module is used to combine and process the flight data collected in real time by the UAV and the image data of the photovoltaic panel surface to build a dynamic flight control algorithm. The inspection optimization module is used to obtain corresponding photovoltaic panel surface image data based on the dynamic flight control algorithm and the inspection route optimization algorithm, and then construct an inspection image quality assessment model; based on the inspection image quality assessment model, the quality of the photovoltaic panel surface image data collected in real time during the inspection process is assessed to continuously improve the inspection quality and efficiency.