A method for locating defects in photovoltaic modules based on a drone

Abstract

The application discloses a photovoltaic module defect positioning method based on a UAV, relates to the technical field of photovoltaic inspection, and comprises the following steps: performing instance segmentation on orthographic images of a photovoltaic station and assigning module codes, and constructing a module geographic information table; planning a UAV inspection path according to an electric parameter anomaly instruction of the station; collecting optical data along the path by the UAV, and recording real-time positioning data when a defect is identified; obtaining approximate defect coordinates through geometric attribution determination; and constructing a perspective transformation matrix through registration of module four-corner point image coordinates and latitude and longitude coordinates, and converting the defect image coordinates into accurate geographic coordinates. The application realizes accurate mapping of the defect coordinates by registering the perspective transformation matrix through the module four-corner point coordinates, and can greatly improve positioning accuracy and inspection efficiency.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic inspection technology, and in particular to a method for locating defects in photovoltaic modules based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Currently, photovoltaic (PV) module inspection and fault location mainly rely on SCADA data from the power plant. By collecting string-level operating electrical parameters through the SCADA system, preliminary fault inferences at the string level can be achieved. Based on the electrical parameter monitoring data, drones can be further used to conduct directional identification operations to complete module-level fault location. PV modules may develop defects such as microcracks, hot spots, and fissures during long-term operation, making them key monitoring targets during the operation and maintenance phase of PV power plants.

[0003] Existing methods for locating defects in photovoltaic modules generally have significant shortcomings. The location accuracy is relatively coarse, only able to pinpoint the string or module range, and manual inspections cannot provide precise location data. The location timeliness is significantly delayed, with large-scale inspections taking too long, and model-based inference has a broad location range. At the same time, the problem of missed defects is prominent, and the detection timeliness is limited, making it impossible to achieve comprehensive and efficient defect monitoring, which seriously restricts the quality of operation and maintenance of photovoltaic power plants. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for locating defects in photovoltaic modules based on unmanned aerial vehicles (UAVs) to solve the problem of low accuracy in locating defects in existing photovoltaic modules.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for locating defects in photovoltaic modules based on unmanned aerial vehicles (UAVs), comprising: The orthophoto data of the photovoltaic power station is segmented into instances, and the segmented photovoltaic module images are assigned module codes. Using the module codes as indexes, the photovoltaic module images are associated with and stored with their corresponding geographic coordinates to construct a geographic information table of the photovoltaic power station's modules. Based on the abnormal notification instructions issued by the power station monitoring device, the path planning is performed on the photovoltaic modules retrieved from the component geographic information table to obtain the inspection path of the drone; The drone is controlled to fly along the inspection path and collect real-time optical data of the photovoltaic module. When defect features are identified in the real-time optical data, the real-time positioning data of the drone is recorded. The real-time positioning data is compared with the spatial distribution of photovoltaic modules in the component geographic information table to determine the geometrical attribution, thereby obtaining the approximate defect location coordinates of the photovoltaic modules. Based on the approximate defect location coordinates, retrieve the latitude and longitude coordinates of the four corner points of the defective photovoltaic module and the corresponding photovoltaic module image from the component geographic information table, and perform spatial coordinate registration between the four corner point image coordinates in the photovoltaic module image and the latitude and longitude coordinates of the four corner points to construct the perspective transformation matrix of the photovoltaic module. The image coordinates of the defect features in the photovoltaic module image are substituted into the perspective transformation matrix to perform spatial transformation calculations, so as to obtain the target defect location coordinates of the photovoltaic module.

[0007] As a preferred embodiment of the UAV-based photovoltaic module defect location method described in this invention, wherein: Preferably, the process of constructing the component geographic information table of the photovoltaic power station is as follows: The orthophoto data of the photovoltaic power station is input into a pre-trained instance segmentation model for pixel-by-pixel mapping to obtain the instance attribution probability map of each pixel in the orthophoto data. Mask extraction is performed on the connected pixel regions in the instance attribution probability map to obtain the instance mask corresponding to the connected pixel regions, and the independent image region enclosed by the instance mask is cropped from the orthophoto data as the component image of the photovoltaic power station. A polygon approximation fitting is performed on the outer contour of the largest connected component in the component image, and the pixel coordinates of the four vertexes obtained by fitting are used as the image coordinates of the four corner points of the component image. The component images are assigned hierarchical component codes, which are generated sequentially according to the hierarchical order of the photovoltaic power station, transformer substation, string, and component. Using the component code as a unique primary key, obtain the position coordinates of the four corner points of the photovoltaic module corresponding to the component code in the geographic coordinate system, so as to obtain the latitude and longitude coordinates of the four corner points of the photovoltaic module; The component codes, the image coordinates of the four corner points, and the latitude and longitude coordinates of the four corner points are indexed and bound to construct the component geographic information table of the photovoltaic power station.

[0008] Preferably, the process of obtaining the drone's patrol path is as follows: The abnormal notification command issued by the power plant monitoring device is parsed to extract the string code of the abnormal string, the starting component code and the ending component code of the abnormal string. Using the starting component code and the ending component code as search criteria, extract the latitude and longitude coordinates of the four corner points of the starting component and the latitude and longitude coordinates of the four corner points of the ending component from the component geographic information table. Based on the latitude and longitude coordinates of the four corner points of the starting component, the spatial center position of the starting component is determined as the starting point of the UAV's patrol, and based on the latitude and longitude coordinates of the four corner points of the ending component, the spatial center position of the ending component is determined as the ending point of the UAV's patrol. Using the patrol start point as the starting point of the trajectory and the patrol end point as the ending point of the trajectory, the spatial center positions of the photovoltaic modules are connected according to the constraint rule of traversing all photovoltaic modules in the abnormal string without omission or repetition, so as to generate the patrol path of the UAV.

[0009] Preferably, the process of recording the real-time positioning data of the UAV is as follows: Control the drone to fly along the inspection path, and simultaneously trigger the visible light acquisition unit and infrared thermal image acquisition unit of the drone to acquire the visible light image sequence and infrared thermal image sequence of the photovoltaic module; The visible light image sequence and the infrared thermal image sequence are combined in a dual-stream manner to obtain the real-time optical data of the photovoltaic module; Crack edge identification is performed on each frame of the real-time optical data to obtain the defect features of the photovoltaic module; Lock the acquisition timestamp of the target frame image that generated the defect features; The UAV spatial position coordinates with the same acquisition timestamp are retrieved from the UAV's onboard positioning unit, the corresponding gimbal pitch angle and gimbal yaw angle are retrieved from the UAV's gimbal attitude sensor, and the camera focal length in the parameter storage area of ​​the visible light acquisition unit is extracted. The spatial coordinates of the UAV, the pitch angle of the gimbal, the yaw angle of the gimbal, and the focal length of the camera are associated and stored to obtain the real-time positioning data of the UAV.

[0010] Preferably, the process of obtaining the approximate defect location coordinates of the photovoltaic module is as follows: Using the spatial coordinates as the optical center position of the camera, the optical axis direction of the visible light acquisition unit is determined according to the gimbal pitch angle and gimbal yaw angle, and combined with the camera focal length, the image coordinates of the defect feature are converted into a spatial ray direction vector that originates from the optical center and passes through the defect point. Using the spatial coordinates as the starting point of the ray, extend the ray in a straight line along the direction vector of the ray until the straight line extends and intersects with the preset ground plane, and use the coordinates of the intersection point as the matching point of the photovoltaic module. Using the points to be matched as query conditions, traverse the set of latitude and longitude coordinates of the four corner points of the photovoltaic modules in the component geographic information table; Spatial inclusion verification is performed on the photovoltaic module. When the point to be matched falls within the closed spatial domain enclosed by the four corner points of the current photovoltaic module, the traversal is stopped and the current photovoltaic module is identified as a defective photovoltaic module. Extract the component code of the defective photovoltaic module from the component geographic information table, and determine the point to be matched as the approximate defect location coordinates of the photovoltaic module.

[0011] Preferably, the process of constructing the perspective transformation matrix of the photovoltaic module is as follows: Using the component code of the defective photovoltaic module as the retrieval primary key, the latitude and longitude coordinates of the four corner points of the defective photovoltaic module are extracted from the component geographic information table; Edge intersection detection is performed on the target frame image to obtain the image control point set of the photovoltaic module image; The pixel coordinates in the image control point set are coupled with the corresponding latitude and longitude coordinates in the four corner point latitude and longitude coordinate set to obtain four pairs of corresponding points of the photovoltaic module. Spatial transformation interpretation is performed on the four sets of corresponding point pairs to construct the perspective transformation matrix of the photovoltaic module.

[0012] Preferably, the process of interpreting the spatial transformation of the four sets of corresponding point pairs to construct the perspective transformation matrix of the photovoltaic module is as follows: The image pixel coordinates and geographic latitude and longitude coordinates contained in each of the four sets of corresponding point pairs are mapped and encapsulated to obtain four sets of mapping primitives for the four sets of corresponding point pairs. Relationship reconstruction is performed on the four sets of mapping primitives to obtain the projection relationship parameters of the four sets of mapping primitives; Based on a preset matrix structure, the projection relationship parameters are organized into an array to obtain the perspective transformation matrix of the photovoltaic module.

[0013] Preferably, the process of obtaining the target defect location coordinates of the photovoltaic module is as follows: Locate the continuous pixel region occupied by the defect features in the photovoltaic module image to obtain the defect region mask of the photovoltaic module; The distribution center of the defect region mask is estimated to obtain the defect image coordinates of the photovoltaic module; Using the coordinates of the defect image as input, a single-point spatial transformation is performed by applying the projection mapping rule from pixel coordinates to geographic coordinates defined by the perspective transformation matrix, so as to obtain the target defect location coordinates of the photovoltaic module.

[0014] The beneficial effects of this invention are as follows: By segmenting instances, hierarchically encoding components, and constructing a component geographic information table from orthophotos of photovoltaic power plants, this invention establishes a precise geographic information foundation for the components at the power plant. Combined with electrical parameter anomaly commands, it completes directional path planning, making UAV inspections more targeted and significantly improving the response speed and execution efficiency of inspection operations. Relying on the simultaneous acquisition of visible light and infrared dual modes by the UAV, it can accurately capture component defect features. With the help of an onboard high-precision positioning module to obtain real-time positioning data, the defective component is quickly located through geometric attribution determination. Then, by registering the image coordinates of the four corner points of the component with latitude and longitude coordinates to construct a perspective transformation matrix, it achieves accurate conversion of defect image coordinates to geographic coordinates, and stably controls the defect positioning accuracy within a high standard range, achieving centimeter-level accurate positioning of photovoltaic component defects.

[0015] This invention can quickly identify and locate defects in photovoltaic modules, significantly reducing defect location time, improving the overall inspection efficiency of photovoltaic power plants, and enabling continuous defect monitoring around the clock. Accurate location results directly guide maintenance personnel to efficiently handle faults, reducing manpower and time costs for defect investigation and repair, and promptly eliminating safety hazards caused by defects such as microcracks and hot spots, ensuring the long-term safe and stable operation of photovoltaic power plants. Simultaneously, the standardized module coding and geographic information management system enables standardized and digital control of power plant module information, providing solid data support and technical assurance for intelligent operation and maintenance of photovoltaic power plants, and continuously upgrading photovoltaic operation and maintenance work towards efficiency, precision, and intelligence. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the photovoltaic module defect localization method based on UAV in this embodiment; Figure 2 This is a flowchart of the UAV optical data acquisition and real-time defect feature determination in this embodiment; Figure 3 This is a flowchart of the geometric attribution determination and approximate defect coordinate acquisition in this embodiment; Figure 4 This is a flowchart illustrating the construction of the perspective transformation matrix and the precise conversion of the target defect coordinates in this embodiment. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0021] Reference Figure 1 This is one embodiment of the present invention, which provides a method for locating defects in photovoltaic modules based on unmanned aerial vehicles (UAVs), comprising the following steps: The orthophoto data of the photovoltaic power station is segmented into instances, and the segmented photovoltaic module images are assigned module codes. Using the module codes as indexes, the photovoltaic module images are associated with and stored with their corresponding geographic coordinates to construct a geographic information table of the photovoltaic power station's modules. In this embodiment of the invention, the process of constructing the component geographic information table of the photovoltaic power station is as follows: The orthophoto data of the photovoltaic power station is input into a pre-trained instance segmentation model for pixel-by-pixel mapping to obtain the instance attribution probability map of each pixel in the orthophoto data. Mask extraction is performed on the connected pixel regions in the instance attribution probability map to obtain the instance mask corresponding to the connected pixel regions, and the independent image region enclosed by the instance mask is cropped from the orthophoto data as the component image of the photovoltaic power station. A polygon approximation fitting is performed on the outer contour of the largest connected component in the component image, and the pixel coordinates of the four vertexes obtained by fitting are used as the image coordinates of the four corner points of the component image. The component images are assigned hierarchical component codes, which are generated sequentially according to the hierarchical order of the photovoltaic power station, transformer substation, string, and component. Using the component code as a unique primary key, obtain the position coordinates of the four corner points of the photovoltaic module corresponding to the component code in the geographic coordinate system, so as to obtain the latitude and longitude coordinates of the four corner points of the photovoltaic module; The component codes, the image coordinates of the four corner points, and the latitude and longitude coordinates of the four corner points are indexed and bound to construct the component geographic information table of the photovoltaic power station.

[0022] After acquiring orthophoto data covering the entire ground area of ​​the photovoltaic power station, this complete image data is imported into an instance segmentation model that has completed preliminary sample training and whose training convergence has reached the industry-standard criteria. The training of this instance segmentation model is customized for photovoltaic module instance segmentation tasks, with three progressive training levels: pixel feature level, module instance level, and power station scene level. The training dimensions include three core dimensions: spatial pixel dimension, appearance texture feature dimension, and edge morphology feature dimension. The specific training steps are as follows: first, orthophotos of photovoltaic power stations under different lighting conditions, different module density, and different power station scales are collected; then, the photovoltaic modules within the images are labeled pixel-by-pixel with category attributes and independent instance boundaries, constructing a standardized model training sample set. The total training sample set is divided into a model training subset and a fixed validation sample set according to a fixed ratio. Using the spatial pixel dimension data of orthophotos as the model input data, the basic visual features of the photovoltaic modules' appearance texture and edge morphology are extracted first at the pixel feature level. Then, at the module instance level, these basic features are aggregated to form complete instance features for a single photovoltaic module. Finally, at the site scene level, all module instance features are integrated to form the module feature distribution across the entire photovoltaic site. Training is conducted iteratively, with each iteration inputting the training subset data to the model to complete feature learning. After each iteration, the fixed validation sample set is input into the model to perform feature recognition verification, and the feature recognition error and accuracy are recorded simultaneously for each verification round. The model training convergence is determined when the feature recognition error on the fixed validation sample set no longer shows a continuously decreasing trend, and the recognition accuracy remains stable. Based on the various appearance texture and edge morphology features of photovoltaic modules pre-learned by the model, feature matching and category determination operations are performed on each pixel distributed within the orthophoto data. Based on the degree of overlap between the matching features of each pixel and the photovoltaic module features learned by the model, the corresponding classification tendency is divided, and a complete instance classification probability map is generated that can correspond to the position information of all pixels in the orthophoto data, and each pixel carries the classification tendency information.

[0023] The connectivity criterion is that pixels are directly adjacent in spatial position (top, bottom, left, and right) and have identical category affiliation. This criterion is based on the physical imaging pattern of photovoltaic modules presenting continuous, patchy pixel distributions in orthophotos. All pixel points within the instance affiliation probability map are traversed, and pixel clusters satisfying both the adjacent position relationship and consistent category affiliation are identified. For each identified connected pixel region, a pixel coverage boundary is defined along the outermost pixel point. Based on the defined boundary, the entire pixel distribution range of the corresponding region is completely delineated, generating a unique instance mask for each connected pixel region's boundary range. Using the pixel boundary range locked by the instance mask, the corresponding image range is selected in the original orthophoto data according to completely identical pixel coordinates. The image content within the selected range is completely captured and saved separately, forming non-overlapping photovoltaic module images that strictly correspond to individual photovoltaic equipment units.

[0024] Using the entire pixel distribution range of a single photovoltaic (PV) power plant module image as the traversal basis, the connected regions within the image whose pixel connectivity occupies a percentage of the total image area that meets a preset percentage standard are selected as the largest connected region. This preset percentage standard is set based on the conventional imaging ratio of the image area occupied by a single PV module in the corresponding cropped module image, and is used to filter out small connected regions formed by scattered noise pixels within the image. Along all edge pixels on the outermost side of the largest connected region, continuous point connections are made sequentially. Following the natural direction of the edge pixels, irregularly curved points on the edges are gradually regularized, and redundant, fragmented turning points at the edges that do not belong to the regular shape of the PV module are continuously removed until a regular closed quadrilateral outline that fits the actual shape of the PV module is formed. The pixel positions corresponding to the four corners of the regularized quadrilateral outline are locked, and the horizontal and vertical pixel arrangement information corresponding to each of the four corner positions is accurately recorded. These four position information are fixed and retained as the unique image coordinates of the four corner points corresponding to the boundary of the module image.

[0025] Following a fixed hierarchical arrangement of power plant, transformer substation, string, and module, this order is based on the actual equipment affiliation and management hierarchy of the photovoltaic power plant. The power plant, as the overall unit of photovoltaic power production, is the highest level, encompassing all transformer substations, strings, and modules within the plant. The transformer substation is the core unit responsible for power collection and conversion, belonging to the power plant and managing multiple strings. A string is a power supply unit composed of several modules connected in series under the transformer substation, belonging to the transformer substation and containing multiple modules. A module is the smallest independent unit of photovoltaic power generation, belonging to the string. This top-down hierarchical order perfectly matches the actual distribution, affiliation, and daily management processes of the photovoltaic power plant equipment, ensuring that the coding clearly reflects the hierarchical position of the equipment corresponding to each module image. First, the fixed coding content corresponding to the power plant level is determined, then the fixed coding content corresponding to the transformer substation level, then the fixed coding content corresponding to the string level, and finally the fixed coding content corresponding to the module level. The coding content of the four levels is then sequentially combined to assign a unique hierarchical module code to each module image.

[0026] Using the component code as a unique retrieval identifier, precise matching is performed in a dedicated geographic coordinate information database for the photovoltaic power station. This database is built by collecting and storing the latitude and longitude data of the four corner points of all photovoltaic modules through on-site positioning measurements of all photovoltaic modules using drones equipped with RTK positioning modules. The precise matching uses the component code as the unique retrieval identifier and performs a character-by-character complete consistency comparison with the component codes pre-stored in the geographic coordinate information database and bound to each photovoltaic module. When the comparison results are completely identical, the match is considered successful. The target photovoltaic module corresponding to the component code is found, and the original geographic coordinate data corresponding to the four corner points of the target photovoltaic module is retrieved. The longitude and latitude values ​​corresponding to each corner point are extracted from the original data, and the longitude and latitude values ​​of the four corner points are processed to obtain the latitude and longitude coordinates of the four corner points of the photovoltaic module.

[0027] Using the component code as the core retrieval index, a fixed one-to-one association is established between the image coordinates and latitude and longitude coordinates of the four corner points corresponding to a single photovoltaic module and the component code of that photovoltaic module. After completing the above information association operation for all photovoltaic modules in the photovoltaic power station, all the associated information is integrated and summarized to form a component geographic information table of the photovoltaic power station.

[0028] The beneficial effects are that, through standardized pixel division, edge detection, hierarchical encoding, coordinate matching and index binding operations, a component geographic information table containing component codes, four-corner point image coordinates, and four-corner point latitude and longitude coordinates is accurately constructed. This provides a unified and accurate data foundation for the positioning, retrieval and management of photovoltaic components, ensuring the accurate implementation of subsequent UAV inspections and defect location work, and realizing the standardized and digital management of photovoltaic power station component information.

[0029] Based on the abnormal notification instructions issued by the power station monitoring device, the path planning is performed on the photovoltaic modules retrieved from the component geographic information table to obtain the inspection path of the drone; In this embodiment of the invention, the process of obtaining the patrol path of the UAV is as follows: The abnormal notification command issued by the power plant monitoring device is parsed to extract the string code of the abnormal string, the starting component code and the ending component code of the abnormal string. Using the starting component code and the ending component code as search criteria, extract the latitude and longitude coordinates of the four corner points of the starting component and the latitude and longitude coordinates of the four corner points of the ending component from the component geographic information table. Based on the latitude and longitude coordinates of the four corner points of the starting component, the spatial center position of the starting component is determined as the starting point of the UAV's patrol, and based on the latitude and longitude coordinates of the four corner points of the ending component, the spatial center position of the ending component is determined as the ending point of the UAV's patrol. Using the patrol start point as the starting point of the trajectory and the patrol end point as the ending point of the trajectory, the spatial center positions of the photovoltaic modules are connected according to the constraint rule of traversing all photovoltaic modules in the abnormal string without omission or repetition, so as to generate the patrol path of the UAV.

[0030] The formula for calculating the spatial center position is as follows: ; In the formula, The spatial center position of the component. The x-coordinate of the top-left corner of the component. The x-coordinate of the top right corner of the component. The x-coordinate of the bottom left corner of the component. The x-coordinate of the bottom right corner of the component. The ordinate of the top-left corner of the component. The ordinate of the bottom right corner of the component. The ordinate of the bottom left corner of the component. The ordinate of the top right corner of the component. The elevation coordinates of the top left corner of the component. The elevation coordinates of the top right corner of the component. The elevation coordinates of the bottom left corner of the component. This refers to the elevation coordinates of the bottom right corner of the component.

[0031] The abnormal notification commands issued by the power plant monitoring device are analyzed and identified field by field. These commands are automatically generated by the monitoring device when it detects deviations from preset normal threshold ranges in the voltage, current, and power of the photovoltaic strings. They are standardized command messages containing abnormal string identifiers and abnormal component range identifiers. The preset normal threshold ranges are standard value ranges for string operating voltage, operating current, and output power, pre-defined based on the factory-rated electrical parameters of the photovoltaic modules, standard series configuration parameters of the photovoltaic strings, and the design and operation specifications of the photovoltaic power plant. Electrical parameters within this range indicate normal operation; values ​​exceeding this range are considered abnormal. The field-by-field analysis and identification process breaks down the command message into mutually exclusive segments according to preset fixed field formats and field separators. Independent field units are identified one by one based on field type characteristics. The preset fixed field format and field separator are derived from the industry-standard communication protocol followed by the photovoltaic power station monitoring system. They are pre-agreed upon and solidified into the system by the power station monitoring device and the back-end data processing unit. The fixed field format includes a fixed structure in sequence, including instruction type, string code, starting component code, ending component code, and abnormal occurrence time. The field separator is a pre-agreed special symbol used to separate different field units. The encoding fields used to identify the string identity, the encoding fields to identify the starting component identity, and the encoding fields to identify the ending component identity are locked in the instruction. The string code, starting component code, and ending component code of the abnormal string are completely extracted.

[0032] The extracted start component code and end component code are used as precise search conditions. The component code field content is checked row by row in the component geographic information table. After the code matching is completed, the coordinate data of the corresponding component is directly retrieved, and the latitude and longitude coordinate sets of the four corner points of the start component and the latitude and longitude coordinate sets of the four corner points of the end component are extracted.

[0033] The x-coordinates of the top left, top right, bottom left, and bottom right corners of the component, as well as the y-coordinates of the top left, top right, bottom left, and bottom right corners, and the elevation coordinates of the top left, top right, bottom left, and bottom right corners of the component, are all taken from the three-dimensional measured data collected from the four corner points of the component's spatial outline. By sampling and calibrating the points on the component's physical spatial boundary, the three-axis coordinate values ​​corresponding to each of the four feature corner points are directly obtained as the raw data for calculation.

[0034] The coordinate values ​​of the four feature corner points of the component in the same dimension are summed. All values ​​of the horizontal coordinate, vertical coordinate, and elevation coordinate are summed. Then, the summation results of the three dimensions are uniformly divided. The operation logic of solving the mean of the four equal divisions is used to complete the derivation of the center value of each dimension.

[0035] The spatial center position of the component is generated by the average of the three-dimensional coordinates of the four feature corner points. The summation of dimensions and the average division of the mean are completed based on the original sampled data of the three-axis coordinates of each corner point, which can accurately fit the overall spatial geometric center point of the component.

[0036] This calculation method, by collecting the three-dimensional coordinates of the four corners of the component and performing a mean conversion in the same dimension, can uniquely determine the spatial center position of the component, realize the quantitative calibration of the component's spatial geometric center, and provide a fixed numerical basis for the component's spatial position positioning and attitude benchmark establishment.

[0037] Retrieve the position information of the four corner points in the latitude and longitude coordinate set of the starting component. Determine the median value of the longitude and the median value of the latitude of the four corner points using the above calculation method. Determine the geographical location corresponding to the median value as the spatial center position of the starting component. Use this position as the starting point of the UAV's patrol. Retrieve the position information of the four corner points in the latitude and longitude coordinate set of the ending component. Determine the median value of the longitude and the median value of the latitude of the four corner points using the same calculation method. Determine the geographical location corresponding to the median value as the spatial center position of the ending component. Use this position as the ending point of the UAV's patrol.

[0038] The patrol start point is set as the starting point of the UAV flight path, and the patrol end point is set as the ending point of the UAV flight path. According to the physical arrangement order of the photovoltaic modules within the abnormal string, the spatial center position of each photovoltaic module is determined in turn. The constraint rule of traversing all photovoltaic modules in the abnormal string without missing any module or traversing any module repeatedly is strictly followed. The spatial center positions of all photovoltaic modules are connected in turn according to the arrangement order of the modules to finally generate the UAV patrol path.

[0039] The beneficial effects are that by accurately analyzing abnormal commands, directionally matching component coordinates, clarifying the source and calculation method of spatial center position parameters, scientifically determining the start and end points of inspections, and standardizing flight path planning, highly targeted UAV inspection paths can be generated quickly, avoiding large-scale aimless flight of UAVs, ensuring that all photovoltaic modules in the abnormal string can be inspected completely, effectively improving the response speed and execution efficiency of UAV inspections, and laying a solid path foundation for the subsequent accurate identification and location of photovoltaic module defects.

[0040] The drone is controlled to fly along the inspection path and collect real-time optical data of the photovoltaic module. When defect features are identified in the real-time optical data, the real-time positioning data of the drone is recorded. In this embodiment of the invention, the process of recording the real-time positioning data of the drone is as follows: Control the drone to fly along the inspection path, and simultaneously trigger the visible light acquisition unit and infrared thermal image acquisition unit of the drone to acquire the visible light image sequence and infrared thermal image sequence of the photovoltaic module; The visible light image sequence and the infrared thermal image sequence are combined in a dual-stream manner to obtain the real-time optical data of the photovoltaic module; Crack edge identification is performed on each frame of the real-time optical data to obtain the defect features of the photovoltaic module; Lock the acquisition timestamp of the target frame image that generated the defect features; Based on the acquisition timestamp, retrieve the UAV spatial position coordinates with the same acquisition timestamp from the UAV's onboard positioning unit, retrieve the corresponding gimbal pitch angle and gimbal yaw angle from the UAV's gimbal attitude sensor, and extract the camera focal length from the parameter storage area in the visible light acquisition unit. The spatial coordinates of the UAV, the pitch angle of the gimbal, the yaw angle of the gimbal, and the focal length of the camera are associated and stored to obtain the real-time positioning data of the UAV.

[0041] The drone maintains a constant and stable flight along the pre-planned inspection path. During flight, it continuously corrects its flight attitude based on the built-in navigation module to keep the path from deviating. At the same time, it sends a synchronization start signal to the visible light acquisition unit and the infrared thermal imaging acquisition unit on the drone. After receiving the signal, the visible light acquisition unit continuously acquires images of the photovoltaic module and arranges them in the order of acquisition time to form a visible light image sequence. After receiving the signal, the infrared thermal imaging acquisition unit continuously acquires images of the temperature distribution of the photovoltaic module and arranges them in the order of acquisition time to form an infrared thermal image sequence.

[0042] Using the same acquisition timestamp as the matching benchmark, each frame of the visible light image sequence is aligned and bound to the corresponding image at the same timestamp in the infrared thermal image sequence. All the aligned and bound image frames are integrated into a unified image data set in chronological order. This integrated image data set is the real-time optical data of the photovoltaic module.

[0043] The system iterates through each frame of the real-time optical data and performs edge contour tracking on all pixels in the image. Contours in the image whose contour length reaches the preset crack length benchmark and have continuous disconnected areas between adjacent pixels are identified as crack edges. The contour shape and pixel distribution information of the crack edge are extracted. The extracted information is the defect feature of the photovoltaic module.

[0044] In real-time optical data, an image frame containing defect features is located. The unique timestamp of the image frame, which is automatically generated and bound by the UAV acquisition system during acquisition, is read. This timestamp is the acquisition timestamp of the target frame image that produces the defect features.

[0045] Using the locked acquisition timestamp as the sole matching condition, records with completely identical timestamps are filtered out from the storage data of the UAV's onboard positioning unit. The UAV's spatial position coordinates in the record are extracted. Using the same acquisition timestamp as the matching condition, the corresponding gimbal pitch angle and gimbal yaw angle values ​​are retrieved from the storage data of the UAV's gimbal attitude sensor. The parameter storage area of ​​the visible light acquisition unit is opened, and the pre-fixed camera focal length values ​​stored in the area are directly read.

[0046] The obtained UAV spatial position coordinates, gimbal pitch angle, gimbal yaw angle and camera focal length are bound according to a fixed correspondence. The multiple sets of bound data are integrated into a single data set, which is the real-time positioning data of the UAV.

[0047] The beneficial effects are that by using a drone to fly stably along the inspection path and simultaneously triggering dual acquisition units to acquire corresponding image sequences, standardized real-time optical data is obtained by combining the two streams based on the timestamp. Based on the preset crack length benchmark, the crack edge is accurately identified to obtain defect features. After locking the corresponding acquisition timestamp, the data from the airborne positioning unit, gimbal attitude sensor, and visible light acquisition unit focal length parameters are accurately matched and retrieved. Multiple types of data are associated and stored to form complete real-time positioning data. The entire operation is standard and reproducible, which can provide complete and accurate raw data support for subsequent defect positioning, and improve the accuracy and reliability of photovoltaic module defect identification and positioning.

[0048] The real-time positioning data is compared with the spatial distribution of photovoltaic modules in the component geographic information table to determine the geometrical attribution, thereby obtaining the approximate defect location coordinates of the photovoltaic modules. In this embodiment of the invention, the process of obtaining the approximate defect location coordinates of the photovoltaic module is as follows: Using the spatial coordinates as the optical center position of the camera, and based on the gimbal pitch angle and gimbal yaw angle, the optical axis direction of the visible light acquisition unit is determined, and combined with the camera focal length, the image coordinates of the defect feature are converted into a spatial ray direction vector originating from the optical center and passing through the defect point. Using the spatial coordinates as the starting point of the ray, extend the ray in a straight line along the direction vector of the ray until the straight line extends and intersects with the preset ground plane, and use the coordinates of the intersection point as the matching point of the photovoltaic module. Using the points to be matched as query conditions, traverse the set of latitude and longitude coordinates of the four corner points of the photovoltaic modules in the component geographic information table; Spatial inclusion verification is performed on the photovoltaic module. When the point to be matched falls within the closed spatial domain enclosed by the four corner points of the current photovoltaic module, the traversal is stopped and the current photovoltaic module is identified as a defective photovoltaic module. Extract the component code of the defective photovoltaic module from the component geographic information table, and determine the point to be matched as the approximate defect location coordinates of the photovoltaic module.

[0049] The acquired UAV spatial coordinates are directly used as the optical center position of the camera of the visible light acquisition unit. The vertical orientation of the optical axis of the visible light acquisition unit is determined based on the gimbal pitch angle, and the horizontal orientation of the optical axis of the visible light acquisition unit is determined based on the gimbal yaw angle. The optical axis direction of the visible light acquisition unit is obtained by combining the vertical and horizontal orientations. Combined with the imaging reference distance corresponding to the camera focal length, the image coordinates of the defect feature are converted into a spatial ray direction vector that originates from the camera optical center and passes through the defect point, according to the physical rules of camera imaging projection.

[0050] The spatial coordinates corresponding to the optical center position of the camera are taken as the starting point of the ray. A continuous straight line is extended from the starting point along the direction vector of the spatial ray until the extended trajectory forms a unique intersection point with the pre-set standard horizontal ground plane. The position coordinates corresponding to the intersection point are extracted and used as the matching point of the photovoltaic module.

[0051] The identified matching points are used as fixed and unique query conditions. Strictly following the preset storage order of the four-level numerical ascending order of the component geographic information table (site code, transformer code, string code, and component code), the latitude and longitude coordinates of the four corner points of the photovoltaic components in each record are read sequentially, starting from the first component record in the table. The coordinate information of all photovoltaic components is completely traversed and checked one by one without skipping any component record.

[0052] For each photovoltaic module checked one by one during the traversal process, the longitude and latitude values ​​of the point to be matched are precisely compared with the closed quadrilateral area enclosed by the longitude and latitude coordinates of the four corner points of the module. It is verified one by one whether the point to be matched is within the internal boundary of the quadrilateral area. When the point to be matched falls completely within the closed space enclosed by the four corner points of the current photovoltaic module, the overall traversal operation is immediately terminated, and the photovoltaic module currently being checked is officially identified as a defective photovoltaic module.

[0053] The complete storage record corresponding to the defective photovoltaic module is accurately located in the component geographic information table. The component code, which is pre-bound and stored with coordinate information, is completely extracted from this record. At the same time, the latitude and longitude coordinates of the point to be matched are officially determined as the approximate defect location coordinates of the photovoltaic module.

[0054] The beneficial effects are that this process can efficiently complete the initial positioning of defective components by standardizing the extraction of horizontal projection coordinates under the geographic coordinate system, completely traversing the component geographic information table in a preset storage order of ascending four-level hierarchical coding, accurately performing spatial inclusion verification, and quickly locating defective components and extracting codes and approximate coordinates. This significantly narrows the scope of defect positioning and provides clear target components and reliable basic location basis for subsequent accurate geographic coordinate calculation of defects, effectively improving the overall efficiency and initial positioning accuracy of photovoltaic module defect positioning.

[0055] Based on the approximate defect location coordinates, the latitude and longitude coordinates of the four corner points of the defective photovoltaic module and the corresponding photovoltaic module image are retrieved from the component geographic information table. The image coordinates of the four corner points in the photovoltaic module image are spatially registered with the latitude and longitude coordinates of the four corner points to construct the perspective transformation matrix of the photovoltaic module. In this embodiment of the invention, the process of constructing the perspective transformation matrix of the photovoltaic module is as follows: Using the component code of the defective photovoltaic module as the retrieval primary key, the latitude and longitude coordinates of the four corner points of the defective photovoltaic module are extracted from the component geographic information table; Edge intersection detection is performed on the target frame image to obtain the image control point set of the photovoltaic module image; The pixel coordinates in the image control point set are coupled with the corresponding latitude and longitude coordinates in the four corner point latitude and longitude coordinate set to obtain four pairs of corresponding points of the photovoltaic module. Spatial transformation interpretation is performed on the four sets of corresponding point pairs to construct the perspective transformation matrix of the photovoltaic module.

[0056] The process of interpreting the spatial transformation of the four pairs of corresponding points to construct the perspective transformation matrix of the photovoltaic module is as follows: The image pixel coordinates and geographic latitude and longitude coordinates contained in each of the four sets of corresponding point pairs are mapped and encapsulated to obtain four sets of mapping primitives for the four sets of corresponding point pairs. Relationship reconstruction is performed on the four sets of mapping primitives to obtain the projection relationship parameters of the four sets of mapping primitives; Based on a preset matrix structure, the projection relationship parameters are organized into an array to obtain the perspective transformation matrix of the photovoltaic module.

[0057] The component code corresponding to the defective photovoltaic module is used as the unique retrieval key. The component geographic information table is stored in ascending order of site code, transformer code, string code, and component code. Starting from the first row of the component geographic information table, the component code field of each row is read sequentially. Each component code read is compared with the retrieval key character by character in the character order. When all characters are completely consistent, the match is confirmed to be successful. The longitude and latitude values ​​of the four corner points of the defective photovoltaic module (top left, top right, bottom left, and bottom right) are extracted from the successfully matched table row. The longitude and latitude values ​​of the four corner points are combined and arranged in a fixed directional order of top left, top right, bottom left, and bottom right to form the longitude and latitude coordinate set of the four corner points of the defective photovoltaic module.

[0058] Starting from the top-left pixel position of the target frame image, continuous edge tracking is performed by traversing all pixels row by row and column by column. Four contours with continuous pixel distribution and straight, unbent outlines are selected as the straight edges of the photovoltaic module. The adjacency relationship of the four straight edges is determined in a clockwise direction. The unique overlapping pixel position of each group of adjacent straight edges is determined as the edge intersection point. The horizontal and vertical pixel position values ​​of the four edge intersection points in the target frame image are read in sequence. The four pixel position values ​​are combined and arranged in a fixed orientation order of top left, top right, bottom left, and bottom right to form the image control point set of the photovoltaic module image.

[0059] The pixel coordinates within the image control point set are arranged in a fixed order of top left, top right, bottom left, and bottom right. Similarly, the geographic coordinates within the four corner point latitude and longitude coordinate set are arranged in the same fixed order. A one-to-one binding relationship is established between the top left pixel coordinate of the first position in the image control point set and the top left latitude and longitude coordinate of the first position in the four corner point latitude and longitude coordinate set. The top right pixel coordinate of the second position in the image control point set is also bound to the top right latitude and longitude coordinate of the second position in the four corner point latitude and longitude coordinate set. The bottom left pixel coordinate of the third position in the image control point set is bound to the bottom left latitude and longitude coordinate of the third position in the four corner point latitude and longitude coordinate set. Finally, the bottom right pixel coordinate of the fourth position in the image control point set is bound to the bottom right latitude and longitude coordinate of the fourth position in the four corner point latitude and longitude coordinate set. After all coordinates are bound, it can be ensured that the spatial positions of image pixel coordinates and geographical latitude and longitude coordinates correspond one-to-one. The four sets of independently bound coordinate combinations together constitute the four sets of corresponding point pairs of the photovoltaic module.

[0060] For each of the four pairs of corresponding points, the image pixel coordinates and geographic latitude and longitude coordinates contained in the pair are associated and integrated one-to-one to form an independent and unique coordinate mapping unit. The integration process of all four pairs is completed in the original order of the corresponding point pairs, and finally the four mapping primitives of the photovoltaic module are obtained.

[0061] Each of the four sets of mapping primitives is independently analyzed to determine their correspondences. Based on the spatial projection rules of camera perspective imaging, a unique correspondence between image pixel coordinates and geographic latitude and longitude coordinates within a single mapping primitive is established. This correspondence constitutes the association rule between the two sets of coordinates. Based on this established association rule, core fixed parameters describing the conversion of image pixel positions to geographic latitude and longitude positions are extracted from each mapping primitive. These parameters fully characterize the projection transformation logic between the two sets of coordinates. Following the original sequential arrangement of the four sets of mapping primitives, the core fixed parameters extracted from each set are sequentially aggregated and merged, completing the ordered aggregation of all parameters and ultimately forming the projection relationship parameters for the four sets of mapping primitives.

[0062] A preset 3x3 standard perspective transformation matrix structure is adopted. This matrix structure is a fixed matrix format specifically for the coordinate transformation of photovoltaic modules. The projection relationship parameters are sequentially filled into the corresponding positions of the 3x3 matrix according to the fixed arrangement rules of corresponding points of the upper left, upper right, lower left, and lower right, thus completing the normalized array arrangement of the projection relationship parameters and finally obtaining the perspective transformation matrix specifically for defective photovoltaic modules.

[0063] The beneficial effects are that this process establishes a precise mapping relationship between the pixel coordinates of photovoltaic module images and geographic latitude and longitude coordinates by accurately retrieving component information, standardizing the detection image control points, orderly constructing corresponding point pairs, completely encapsulating mapping primitives, accurately extracting projection relationship parameters, and standardizing and arraying parameters to generate a perspective transformation matrix. This provides a core transformation basis for the subsequent conversion of defect image coordinates into accurate geographic coordinates, and greatly improves the accuracy and stability of defect location.

[0064] The image coordinates of the defect features in the photovoltaic module image are substituted into the perspective transformation matrix to perform spatial transformation calculations, so as to obtain the target defect location coordinates of the photovoltaic module.

[0065] In this embodiment of the invention, the process of obtaining the target defect location coordinates of the photovoltaic module is as follows: Locate the continuous pixel region occupied by the defect features in the photovoltaic module image to obtain the defect region mask of the photovoltaic module; The distribution center of the defect region mask is estimated to obtain the defect image coordinates of the photovoltaic module; Using the coordinates of the defect image as input, a single-point spatial transformation is performed by applying the projection mapping rule from pixel coordinates to geographic coordinates defined by the perspective transformation matrix, so as to obtain the target defect location coordinates of the photovoltaic module.

[0066] The formula for calculating the target defect location coordinates is as follows: ; In the formula, The coordinates of the defect feature in the geographic coordinate system are: Here are the north coordinates of the defect feature in the geographic coordinate system. Locate the coordinates of the target defect. Let be the perspective transformation matrix. Let x be the x-coordinate of the defect feature in the pixel coordinate system of the photovoltaic module image. Let be the ordinate of the defect feature in the pixel coordinate system of the photovoltaic module image. The coordinates of the defect image are given.

[0067] In the photovoltaic module image, based on the defect features identified in the previous crack edge recognition, a full-coverage pixel traversal scan is carried out row by row and column by column starting from the upper left corner of the image. The pixel assignment is determined based on the fixed attributes such as grayscale and contour of the defect features. Continuous pixels that are completely consistent with the defect feature attributes and are connected without interruption are uniformly marked as defect pixels. The entire distribution range covered by the defect pixels is completely preserved. All background areas and normal photovoltaic module pixels in the image except for defect pixels are removed. Only the independent area composed of defect pixels is retained to form an independent pixel area that completely covers the defect features and has no extra pixel interference. Finally, the defect area mask of the photovoltaic module is obtained.

[0068] The horizontal coordinate of the defect feature in the photovoltaic module image pixel coordinate system is obtained by summarizing and statistically analyzing the horizontal pixel coordinates of all defect pixels inside the defect area mask, and calculating the arithmetic mean center value of all horizontal coordinates. This value accurately represents the horizontal center position of the defect feature in the image pixel coordinate system.

[0069] The vertical coordinate of the defect feature in the photovoltaic module image pixel coordinate system is obtained by summarizing and statistically analyzing the vertical pixel coordinates of all defect pixels inside the defect area mask, and calculating the arithmetic mean of all vertical coordinates. This value accurately represents the vertical center position of the defect feature in the image pixel coordinate system.

[0070] A complete statistical summary of the horizontal and vertical coordinates of all defect pixels within the defect mask is performed. The arithmetic mean center values ​​of the horizontal and vertical coordinates of all defect pixels within the mask are calculated. The pixel coordinate system is standardized with the top-left pixel of the photovoltaic module image as the origin, the horizontal direction to the right as the positive direction of the horizontal coordinate, and the vertical direction downwards as the positive direction of the vertical coordinate. The coordinates are combined in a fixed format with the horizontal coordinate value first and the vertical coordinate value last. The horizontal and vertical center values ​​are combined according to this fixed format to form the unique and precise location of the defect feature in the image, ultimately obtaining the defect image coordinates of the photovoltaic module.

[0071] The perspective transformation matrix uses the component code of the defective photovoltaic module as the retrieval primary key, retrieves the latitude and longitude coordinates of the image control points and four corner points from the component geographic information table, constructs four sets of corresponding point pairs in a fixed order, and generates the matrix after coordinate mapping encapsulation, projection relationship reconstruction, and 3x3 standard matrix array organization. It is the core carrier for realizing accurate conversion between image pixel coordinates and geographic latitude and longitude coordinates.

[0072] Using the coordinates of the defective image as the sole input data for spatial transformation, and strictly following the fixed projection mapping rules from image pixel coordinates to geographic latitude and longitude coordinates defined by the perspective transformation matrix, a single-point independent spatial transformation operation is performed on the coordinates of the defective image to completely convert the image coordinates in the pixel coordinate system to the position coordinates in the standard CGCS2000 geographic coordinate system.

[0073] The eastward coordinates of the defect feature in the geographic coordinate system are calculated by projecting the defect image coordinates through a perspective transformation matrix. It is the final eastward position value of the defect feature generated after converting the pixel coordinates to geographic coordinates.

[0074] The north coordinates of the defect feature in the geographic coordinate system are calculated from the spatial transformation of the defect image coordinates through the perspective transformation matrix. It is the final north position value of the defect feature generated after the pixel coordinates are converted into geographic coordinates.

[0075] The calculated eastward and northward coordinates are combined according to the geographic coordinate system standard to form complete on-site location information of the defect characteristics, and finally the target defect location coordinates of the photovoltaic module are obtained.

[0076] This calculation formula takes the defect image coordinates as input data and the perspective transformation matrix as the core transformation basis to perform single-point spatial transformation. It can directly output the geographic coordinate values ​​of the defect features. It is fully integrated with the entire process of defect area mask generation, defect image coordinate calculation, and perspective transformation matrix construction, realizing seamless and accurate conversion from defect image coordinates to actual geographic coordinates. It provides a unique and standardized calculation method for obtaining the target defect location coordinates.

[0077] The beneficial effects are that this process accurately marks the continuous pixels of defects through full-coverage pixel traversal scanning to form an interference-free defect area mask, obtains accurate defect image coordinates based on the arithmetic mean statistical center of defect pixels, and strictly performs coordinate space transformation based on a standardized perspective transformation matrix. This enables the image coordinates of defects to be accurately converted into actual geographic coordinates, achieving centimeter-level precise positioning of photovoltaic module defects. This provides photovoltaic power plant operation and maintenance personnel with defect location information accurate to specific points, significantly reducing the difficulty of defect investigation and ensuring the efficient and stable operation of defect repair and daily operation and maintenance of the power plant.

[0078] like Figure 2 The diagram shows the flowchart for UAV optical data acquisition and real-time defect feature determination. The UAV flies smoothly along a planned inspection path, simultaneously triggering visible light and infrared thermal imaging acquisition units to acquire continuous visible light images and continuous temperature distribution images of the photovoltaic module. The two types of images captured at the same time are integrated and processed to form real-time optical data that can be used for defect detection. Subsequently, detailed edge detection is performed on each set of optical data images to accurately identify defect features such as cracks on the surface of the photovoltaic module, determine the specific image where the defect occurred and the corresponding shooting time, and then, based on the shooting time, simultaneously retrieve the UAV's spatial position, the gimbal's pitch and yaw angles, and the camera's focal length parameters. All this data is uniformly organized and recorded to complete the retention of real-time positioning data.

[0079] like Figure 3 The diagram shows the flowchart for geometric attribution determination and approximate defect coordinate acquisition. The spatial location of the defect captured by the drone is used as the camera's center point. Combined with the gimbal's angle information, the shooting direction is determined. Then, using the camera's focal length parameters, the pixel position of the defect in the image is converted into a spatial ray pointing from the camera center towards the defect. Extending this ray towards the ground, the intersection point of the ray and the ground is obtained as the location to be matched. Subsequently, the geographical distribution range of all photovoltaic modules is checked sequentially in the module geographic information table to determine the photovoltaic module to which the matching point belongs, thus identifying the specific module with the defect. Finally, the coordinates of this ground intersection point are determined as the approximate location coordinates of the defect, and the unique code of the defective module is extracted.

[0080] like Figure 4The diagram illustrates the flowchart for constructing the perspective transformation matrix and accurately converting the target defect coordinates. Based on the defective component's code, the actual latitude and longitude coordinates of the four corner points of the component are retrieved from the component's geographic information table. Simultaneously, the pixel positions of the four corners of the component are precisely located in the image of the defect. The pixel coordinates of the four corners in the image are matched one-to-one with the latitude and longitude coordinates of the four corners in the actual geographic location, forming four sets of corresponding coordinate points. Through the spatial relationship calculation of these corresponding points, a perspective transformation matrix specific to this component is constructed. Next, the pixel region where the defect is located in the image is located, and the center pixel coordinates of the defect are calculated. These coordinates are substituted into the perspective transformation matrix for spatial transformation calculation, ultimately obtaining the precise location coordinates of the photovoltaic module defect in the actual geographic environment.

Claims

1. A method for locating defects in photovoltaic modules based on unmanned aerial vehicles (UAVs), characterized in that: include, The orthophoto data of the photovoltaic power station is segmented into instances, and the segmented photovoltaic module images are assigned module codes. Using the module codes as indexes, the photovoltaic module images are associated with and stored with their corresponding geographic coordinates to construct a geographic information table of the photovoltaic power station's modules. Based on the abnormal notification instructions issued by the power station monitoring device, the path planning is performed on the photovoltaic modules retrieved from the component geographic information table to obtain the inspection path of the drone; The drone is controlled to fly along the inspection path and collect real-time optical data of the photovoltaic module. When defect features are identified in the real-time optical data, the real-time positioning data of the drone is recorded. The real-time positioning data is compared with the spatial distribution of photovoltaic modules in the component geographic information table to determine the geometrical attribution, thereby obtaining the approximate defect location coordinates of the photovoltaic modules. Based on the approximate defect location coordinates, retrieve the latitude and longitude coordinates of the four corner points of the defective photovoltaic module and the corresponding photovoltaic module image from the component geographic information table, and perform spatial coordinate registration between the four corner point image coordinates in the photovoltaic module image and the latitude and longitude coordinates of the four corner points to construct the perspective transformation matrix of the photovoltaic module. The image coordinates of the defect features in the photovoltaic module image are substituted into the perspective transformation matrix to perform spatial transformation calculations, so as to obtain the target defect location coordinates of the photovoltaic module.

2. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 1, characterized in that, The process of constructing the component geographic information table for the photovoltaic power station is as follows: The orthophoto data of the photovoltaic power station is input into a pre-trained instance segmentation model for pixel-by-pixel mapping to obtain the instance attribution probability map of each pixel in the orthophoto data. Mask extraction is performed on the connected pixel regions in the instance attribution probability map to obtain the instance mask corresponding to the connected pixel regions, and the independent image region enclosed by the instance mask is cropped from the orthophoto data as the component image of the photovoltaic power station. A polygon approximation fitting is performed on the outer contour of the largest connected component in the component image, and the pixel coordinates of the four vertexes obtained by fitting are used as the image coordinates of the four corner points of the component image. The component images are assigned hierarchical component codes, which are generated sequentially according to the hierarchical order of the photovoltaic power station, transformer substation, string, and component. Using the component code as a unique primary key, obtain the position coordinates of the four corner points of the photovoltaic module corresponding to the component code in the geographic coordinate system, so as to obtain the latitude and longitude coordinates of the four corner points of the photovoltaic module; The component codes, the image coordinates of the four corner points, and the latitude and longitude coordinates of the four corner points are indexed and bound to construct the component geographic information table of the photovoltaic power station.

3. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 1, characterized in that, The process of obtaining the drone's patrol path is as follows: The abnormal notification command issued by the power plant monitoring device is parsed to extract the string code of the abnormal string, the starting component code and the ending component code of the abnormal string. Using the starting component code and the ending component code as search criteria, extract the latitude and longitude coordinates of the four corner points of the starting component and the latitude and longitude coordinates of the four corner points of the ending component from the component geographic information table. Based on the latitude and longitude coordinates of the four corner points of the starting component, the spatial center position of the starting component is determined as the starting point of the UAV's patrol, and based on the latitude and longitude coordinates of the four corner points of the ending component, the spatial center position of the ending component is determined as the ending point of the UAV's patrol. Using the patrol start point as the starting point of the trajectory and the patrol end point as the ending point of the trajectory, the spatial center positions of the photovoltaic modules are connected according to the constraint rule of traversing all photovoltaic modules in the abnormal string without omission or repetition, so as to generate the patrol path of the UAV.

4. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 1, characterized in that, The process of recording the real-time positioning data of the drone is as follows: Control the drone to fly along the inspection path, and simultaneously trigger the visible light acquisition unit and infrared thermal image acquisition unit of the drone to acquire the visible light image sequence and infrared thermal image sequence of the photovoltaic module; The visible light image sequence and the infrared thermal image sequence are combined in a dual-stream manner to obtain the real-time optical data of the photovoltaic module; Crack edge identification is performed on each frame of the real-time optical data to obtain the defect features of the photovoltaic module; Lock the acquisition timestamp of the target frame image that generated the defect features; The UAV spatial position coordinates with the same acquisition timestamp are retrieved from the UAV's onboard positioning unit, the corresponding gimbal pitch angle and gimbal yaw angle are retrieved from the UAV's gimbal attitude sensor, and the camera focal length in the parameter storage area of ​​the visible light acquisition unit is extracted. The spatial coordinates of the UAV, the pitch angle of the gimbal, the yaw angle of the gimbal, and the focal length of the camera are associated and stored to obtain the real-time positioning data of the UAV.

5. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 4, characterized in that, The process of obtaining the approximate defect location coordinates of the photovoltaic module is as follows: Using the spatial coordinates as the optical center position of the camera, and based on the gimbal pitch angle and gimbal yaw angle, the optical axis direction of the visible light acquisition unit is determined, and combined with the camera focal length, the image coordinates of the defect feature are converted into a spatial ray direction vector originating from the optical center and passing through the defect point. Using the spatial coordinates as the starting point of the ray, extend the ray in a straight line along the direction vector of the ray until the straight line extends and intersects with the preset ground plane, and use the coordinates of the intersection point as the matching point of the photovoltaic module. Using the points to be matched as query conditions, traverse the set of latitude and longitude coordinates of the four corner points of the photovoltaic modules in the component geographic information table; Spatial inclusion verification is performed on the photovoltaic module. When the point to be matched falls within the closed spatial domain enclosed by the four corner points of the current photovoltaic module, the traversal is stopped and the current photovoltaic module is identified as a defective photovoltaic module. Extract the component code of the defective photovoltaic module from the component geographic information table, and determine the point to be matched as the approximate defect location coordinates of the photovoltaic module.

6. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 5, characterized in that, The process of constructing the perspective transformation matrix of the photovoltaic module is as follows: Using the component code of the defective photovoltaic module as the retrieval primary key, the latitude and longitude coordinates of the four corner points of the defective photovoltaic module are extracted from the component geographic information table; Edge intersection detection is performed on the target frame image to obtain the image control point set of the photovoltaic module image; The pixel coordinates in the image control point set are coupled with the corresponding latitude and longitude coordinates in the four corner point latitude and longitude coordinate set to obtain four pairs of corresponding points of the photovoltaic module. Spatial transformation interpretation is performed on the four sets of corresponding point pairs to construct the perspective transformation matrix of the photovoltaic module.

7. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 6, characterized in that, The process of interpreting the spatial transformation of the four pairs of corresponding points to construct the perspective transformation matrix of the photovoltaic module is as follows: The image pixel coordinates and geographic latitude and longitude coordinates contained in each of the four sets of corresponding point pairs are mapped and encapsulated to obtain four sets of mapping primitives for the four sets of corresponding point pairs. Relationship reconstruction is performed on the four sets of mapping primitives to obtain the projection relationship parameters of the four sets of mapping primitives; Based on a preset matrix structure, the projection relationship parameters are organized into an array to obtain the perspective transformation matrix of the photovoltaic module.

8. The method for defect location of photovoltaic modules based on unmanned aerial vehicles as described in claim 4, characterized in that, The process of obtaining the target defect location coordinates of the photovoltaic module is as follows: Locate the continuous pixel region occupied by the defect features in the photovoltaic module image to obtain the defect region mask of the photovoltaic module; The distribution center of the defect region mask is estimated to obtain the defect image coordinates of the photovoltaic module; Using the coordinates of the defect image as input, a single-point spatial transformation is performed by applying the projection mapping rule from pixel coordinates to geographic coordinates defined by the perspective transformation matrix, so as to obtain the target defect location coordinates of the photovoltaic module.

Images