Method, system, and image processing device for capturing and / or processing electroluminescence images, and aircraft

Abstract

A method 400 for capturing and processing an electroluminescence (EL) image 1910 of a PV array 40 is disclosed herein. In the described embodiment, the method 400 includes controlling the aircraft 20 to fly along a route and capture EL images 1910 of corresponding PV array subsections 512 b of the PV array 40, deriving respective image quality parameters from at least some of the captured EL images, dynamically adjusting a flight speed of the aircraft along the route based on the respective image quality parameters for capturing the EL images 1910 of the PV array subsection 512 b, extracting multiple frames 1500 of the PV array subsection 512 b from the EL images 1910, determining a reference frame 2100 from the extracted frames 2100 having the highest image quality of the PV array subsection 512 b, performing image alignment of the extracted frames 2100 to the reference frame to generate an image-aligned frame 2130, and processing the image-aligned frame 2130 to produce an improved image 2140 of the PV array subsection 512 b having a higher resolution than the reference frame. A system, an image processing device, and an aircraft for this method are also disclosed.

Description

[Technical Field]

[0001] The present application relates to a method for capturing and processing electroluminescence (EL) images of a photovoltaic (PV) array, an aircraft for capturing EL images, and an image processing device for processing EL images. [Background technology]

[0002] Solar panels are widely used worldwide. However, due to high initial capital investment costs, site-installed solar panels must function properly and efficiently for a certain period of time to guarantee a return on investment. Therefore, maintaining the quality of site-installed solar panels is important. Due to the large-scale deployment of solar panels in solar farms (or photovoltaic (PV) factories in general) and their remote placement, such as on the roofs of houses, it is often difficult to monitor the performance of individual solar panels. Various imaging techniques are available to detect defects in solar panels, including visual, thermal (infrared), ultraviolet (UV) fluorescence, photoluminescence (PL), and electroluminescence (EL) imaging. For example, EL inspection is used during PV manufacturing for quality control.

[0003] For EL measurements, PV modules of a solar panel are connected to a power source and placed under forward bias. The emitted near-infrared light is captured by a camera sensitive in the near-infrared waveband. EL imaging has also been used on a sampling basis for on-site inspection. One common method for on-site EL inspection is using a mobile trailer. In this method, a mobile trailer carrying a darkroom is deployed at the site. PV modules are removed from their installed position for measurement in the darkroom inside the trailer. This method ensures that EL measurements of PV modules are performed in a controlled environment. However, because the PV modules must be removed, large-scale inspection using this method is time-consuming and not feasible. Furthermore, there is a risk of introducing defects during module handling.

[0004] Another method of EL inspection is performed using a tripod-mounted camera, either at night or during the day, with lock-in current control. While this method does not require removal of the PV module from the support frame, it is still time-consuming and highly labor-intensive. Furthermore, the limitation of capturing images from a tripod-mounted camera can result in perspective and intensity distortions in the captured images. Summary of the Invention [Problem to be solved by the invention]

[0005] It is therefore desirable to provide a solution that addresses at least one of the problems noted in the existing prior art and / or provide the public with a useful alternative. [Means for solving the problem]

[0006] According to a first aspect, there is provided a method for processing an electroluminescence (EL) image of a PV array, the method including the steps of: (i) extracting from the EL image a plurality of frames of a PV array subsection of the PV array, the PV array subsection including one or more PV modules of the PV array, (ii) determining from the extracted frames a reference frame of the PV array subsection having the highest image quality, (iii) performing image alignment of the extracted frames to the reference frame to generate an image-aligned frame, and (iv) processing the image-aligned frame to produce an enhanced image of the PV array subsection having higher resolution than the reference frame.

[0007] The described embodiments can take low-resolution, monochromatic images and videos under dim light conditions or in the absence of natural light, such as at night, and still produce images with improved resolution and reduced noise to identify defective PV modules. Generally, and when averaging is used, the magnitude of noise in the processed images can be reduced by roughly the square root of the number of images averaged. As a result, improved images with generally higher image quality can be achieved.

[0008] Specifically, the step of extracting frames from the image may include determining respective corner points of each PV module in the image, and constructing respective frames for each PV module based on the identified corner points of each PV module.

[0009] In a specific embodiment, determining the respective corner points of each PV module in the image may include clustering the respective corner points of a particular PV module that are repeated in different images, and calculating a respective average position for each cluster of the respective corner points.

[0010] Preferably, determining the reference frame having the highest image quality may include evaluating the image quality of each frame based on at least one of the frame's sharpness, signal-to-noise ratio, and completeness.

[0011] The method may also include arranging the extracted frames in a stacking array before performing the image alignment. Further, corner points of the PV modules may be stacked in the stacking array, and performing the image alignment may include aligning each corner point of each PV module in the extracted frames to a corresponding corner point of the PV module in the reference frame.

[0012] It is envisioned that the step of processing the image-aligned frames may include the steps of grouping the image-aligned frames according to the PV module in each frame, and performing image averaging on each group of image-aligned frames to obtain a respective improved frame for each PV module.

[0013] Furthermore, image averaging may be based on weighted image stack averaging and / or deep convolutional neural network architectures.

[0014] The method may further include associating each improved frame with a horizontal index and a vertical index according to the position of each PV module in the PV array subsection, and ordering the improved frames according to their horizontal and vertical indexes to produce an improved image of the PV array subsection.

[0015] The method may also include scaling a respective image intensity of each enhanced frame. Further, the method may also include mapping the enhanced image of the PV array subsection onto a base map of the PV array subsection. The base map may include the geolocation of each PV module.

[0016] In a specific embodiment, mapping the improved image onto the base map may further include orienting the improved image to align PV array subsections in the improved image with PV array subsections in the base map.

[0017] Further, each EL image of the PV array subsection may include an image identifier, and orienting the improved image may include associating the image identifier of the particular EL image with each improved frame in which that PV module appears in the particular EL image, and determining an orientation of the improved image based on the image identifier associated with each improved frame.

[0018] The geolocation may include GPS coordinates.

[0019] Moreover, the multiple frames may be consecutive frames of a PV array subsection.

[0020] According to a second aspect, there is provided an image processing device for processing an EL image of a PV array, the image processing device including an image processor configured to: extract from the EL image a plurality of frames of a PV array subsection of the PV array, the PV array subsection including one or more PV modules of the PV array, determine from the extracted frames a reference frame for the PV array subsection having the highest image quality, perform image alignment of the extracted frames to the reference frame to generate an image-aligned frame, and process the image-aligned frame to produce an improved image of the PV array subsection having higher resolution and a lower noise level than the reference frame.

[0021] The image processor may be further configured to extract frames from the image by determining respective corner points of each PV module in the image and constructing a respective frame for each PV module based on the identified corner points of each PV module.

[0022] Preferably, the image processor may be further configured to determine each corner point of each PV module in the image by clustering the respective corner points of a particular PV module, which may be repeated in different images, and calculating a respective average position for each cluster of respective corner points.

[0023] The image processor may also be further configured to determine the reference frame having the highest image quality by evaluating the image quality of each frame based on at least one of the frame's sharpness, signal-to-noise ratio, and completeness.

[0024] Additionally, the image processor may be further configured to arrange the extracted frames in a stacked array before performing the image alignment.

[0025] Furthermore, the respective corner points of the PV modules may be stacked in a stacked arrangement, and the image processor may be further configured to perform image alignment by aligning the respective corner points of each PV module in the extracted frame with corresponding corner points of the PV module in the reference frame.

[0026] In a specific embodiment, the image processor may be further configured to process the image-aligned frames by grouping the image-aligned frames according to the PV module in each frame, and to perform image averaging on each group of image-aligned frames to obtain a respective refined frame for each PV module.

[0027] Additionally, the image processor may be configured to perform image averaging based on weighted image stack averaging and / or a deep convolutional neural network structure.

[0028] The image processor may also be configured to associate each improved frame with a horizontal index and a vertical index according to the position of each PV module in the PV array subsection, and to arrange the improved frames according to their horizontal and vertical indexes to produce an improved image of the PV array subsection.

[0029] The image processor may be configured to scale the image intensity of each of the refinement frames.

[0030] The image processor may be further configured to map the improved image of the PV array subsection onto a base map of the PV array subsection. The base map may then include the geolocation of each PV module. Furthermore, the image processor may be configured to map the improved image onto the base map by orienting the improved image to align the PV array subsection in the improved image with the PV array subsection in the base map.

[0031] Further, each EL image of the PV array subsection may include an image identifier, and the image processor may be configured to orient the improved image by associating the image identifier of a particular EL image with each improved frame in which that PV module appears in the particular EL image, and determining an orientation of the improved image based on the image identifier associated with each improved frame.

[0032] The geolocation may include GPS coordinates. Further, the multiple frames may be consecutive frames of the PV array subsection.

[0033] According to a third aspect, there is provided a method for controlling movement of an aircraft having a camera for capturing EL images of a PV array, the method comprising the steps of controlling the aircraft to fly along a route to capture EL images of corresponding PV array subsections of the PV array, deriving respective image quality parameters from at least some of the captured EL images, and dynamically adjusting a flight speed of the aircraft along the route based on the respective image quality parameters for capturing the EL images of the PV array subsections.

[0034] Advantageously, by dynamically adjusting the flight speed of the aircraft according to image quality parameters derived from at least some of the captured EL images, the aircraft can adjust its flight speed to enable it to capture EL images with higher image quality, such as a better signal-to-noise ratio or image sharpness.

[0035] The image quality parameters may include an SNR scan factor and a motion blur scan factor. In a specific example, the SNR scan factor may depend on the target SNR, the measured SNR, and an estimated number of captured EL images that include a particular PV module of the PV array subsection.

[0036] Alternatively, the motion blur scanning factor may be the ratio of the measured object deflection to a predetermined maximum object deflection.

[0037] Preferably, the step of dynamically adjusting the flight speed of the aircraft along the route based on the respective image quality parameters may include the steps of deriving a target flight speed based on the minimum of an SNR scan factor and a motion blur scan factor, and dynamically adjusting the current flight speed of the aircraft to match the target flight speed.

[0038] Further, the step of deriving the target flight speed may include the steps of applying the minimum of the SNR scan factor and the motion blur scan factor to the current flight speed of the aircraft to derive the target scan speed, and if the target scan speed may be less than the maximum flight speed of the aircraft, selecting the target scan speed as the target flight speed.

[0039] The method may further include, if the target scan speed exceeds a maximum flight speed of the aircraft, selecting the maximum flight speed as the target flight speed.

[0040] The method may also include adjusting the target flight speed based on user input factors.

[0041] Additionally, the method may further include detecting an EL signal emitted from the PV array by one or more PV modules prior to controlling the aircraft to fly along the route to capture an EL image of a corresponding PV array subsection of the PV array.

[0042] The method may also include maneuvering the aircraft to an initial position, where the yaw axis of the aircraft and the optical axis of the camera may be perpendicular to the ground before detecting the EL signal.

[0043] Additionally, the method may include navigating the aircraft to a location of the EL signal.

[0044] In a specific embodiment, detecting the EL signal emitted by one or more PV modules of the PV array may include rotating the aircraft about the vehicle's yaw axis while simultaneously increasing the camera's optical axis angle until the EL signal can be detected.

[0045] The camera's optical axis angle may be increased from 0° to 70°. It is envisioned that the camera's optical axis angle may then be increased at a decreasing pitch rate. Preferably, the aircraft may rotate at a decreasing yaw rate.

[0046] The method may further include maneuvering the aircraft to a predetermined altitude before rotating the aircraft.

[0047] The method also includes determining respective key points of a reference PV module in the corresponding PV array subsection, deriving target alignment points from the respective key points for aligning a field of view (FOV) of a camera to the corresponding PV array subsection, performing a perspective transformation to align the respective key points to the target alignment points, and aligning the FOV of the camera to the corresponding PV array subsection by maneuvering the aircraft relative to the corresponding PV array subsection based on the perspective transformation.

[0048] Furthermore, aligning the camera's FOV with the corresponding PV array subsection may further include maneuvering the aircraft to an appropriate altitude, where the corresponding PV array subsection may be at a predetermined size ratio within the camera's FOV at the appropriate altitude.

[0049] The corresponding PV array subsection can occupy 80% to 90% of the camera's FOV at a given size ratio.

[0050] Additionally, the method may further include dynamically adjusting the focus of the camera according to the measured image sharpness.

[0051] The aircraft may further include a light source aligned with the optical axis of the camera, and the method may further include powering the light source except during capturing the EL image of the PV array.

[0052] According to a fourth aspect, there is provided an aircraft including a camera for capturing EL images of a PV array, a propulsion device for actuating movement of the aircraft, and a controller communicatively coupled to the camera and the propulsion device, wherein the controller is configured to control the aircraft to fly along a route to capture EL images of corresponding PV array subsections of the PV array, derive respective image quality parameters from at least some of the captured EL images, and dynamically adjust flight speed of the aircraft along the route based on the respective image quality parameters for capturing the EL images of the PV array subsections.

[0053] The image quality parameters may then include an SNR scanning factor and a motion blur scanning factor. Preferably, the SNR scanning factor may depend on the target SNR, the measured SNR, and an estimated number of captured EL images that include a particular PV module of the PV array subsection.

[0054] Alternatively, the motion blur scanning factor may be the ratio of the measured object deflection to a predetermined maximum object deflection.

[0055] Preferably, the controller may be further configured to dynamically adjust the flight speed of the aircraft along the route based on the respective image quality parameters by deriving a target flight speed based on the minimum of the SNR scan factor and the motion blur scan factor, and dynamically adjusting the current flight speed of the aircraft to match the target flight speed.

[0056] The controller may also be configured to derive a target flight speed by applying the minimum of the SNR scan factor and the motion blur scan factor to the current flight speed of the aircraft to derive a target scan speed, and if the target scan speed may be less than the maximum flight speed of the aircraft, selecting the target scan speed as the target flight speed.

[0057] Additionally, the controller may be configured to select the maximum flight speed as the target flight speed if the target scan speed exceeds the maximum flight speed of the aircraft.

[0058] The controller may also be configured to adjust the target flight speed based on user input factors.

[0059] Moreover, the controller may be further configured to detect an EL signal emitted from the PV array by one or more PV modules, the PV array having an array axis, and the one or more PV modules are aligned with the array axis and may include a plane; determine an array axis of the PV array; and control a camera to capture an EL image of the PV array along the array axis while dynamically adjusting the propulsion device to align the optical axis of the camera to be perpendicular to the plane of the one or more PV modules.

[0060] Further, the controller may be configured to set the aircraft to an initial position before locating the EL signal by dynamically adjusting the propulsion device to set the yaw axis of the aircraft to be perpendicular to the ground and dynamically adjusting the optical axis of the camera to be perpendicular to the ground.

[0061] The controller may also be configured to dynamically adjust the propulsion device to navigate the aircraft to the location of the EL signal.

[0062] The controller may be further configured to locate the EL signal emitted by one or more PV modules of the PV array by dynamically adjusting the propulsion device to rotate the air vehicle about the vehicle's yaw axis while simultaneously increasing the camera's optical axis angle until the EL signal can be located.

[0063] The camera's optical axis angle may be increased from 0° to 70°. Specifically, the camera's optical axis angle may then be increased at a decreasing pitch rate. Preferably, the aircraft may rotate at a decreasing yaw rate.

[0064] The controller may be further configured to dynamically adjust the propulsion device to maneuver the aircraft to a predetermined altitude before rotating the aircraft.

[0065] Further, the controller may be configured to align the camera's field of view (FOV) with the corresponding PV array subsection by determining respective key points of a reference PV module in the corresponding PV array subsection, deriving target alignment points from the respective key points for aligning a camera's FOV with the corresponding PV array subsection, performing a perspective transformation to align the respective key points with the target alignment points, and dynamically adjusting the propulsion device to steer the aircraft relative to the corresponding PV array subsection based on the perspective transformation.

[0066] The controller may also be configured to align the camera FOV with the corresponding PV array subsection by dynamically adjusting the propulsion devices to maneuver the aircraft to an appropriate altitude, where the corresponding PV array subsection may be at a predetermined size ratio within the camera FOV at the appropriate altitude.

[0067] The corresponding PV array subsection can occupy 80% to 90% of the camera's FOV at a given size ratio.

[0068] The controller may further be configured to dynamically adjust the focus of the camera according to the measured image sharpness.

[0069] Additionally, the aircraft may further include a light source aligned with the optical axis of the camera, and the controller may be further configured to power the light source except during capture of the EL image of the PV array.

[0070] According to a fifth aspect, there is provided a method for obtaining an improved image of a PV array subsection of a PV array from EL images of the PV array subsection captured by an aircraft having a camera, the method including: (i) controlling the aircraft to fly along a course to capture EL images of corresponding PV array subsections of the PV array, (ii) deriving respective image quality parameters from at least some of the captured EL images, (iii) dynamically adjusting a flight speed of the aircraft along the course based on the respective image quality parameters for capturing the EL images of the PV array subsection, (iv) extracting a plurality of frames of the PV array subsection from the EL images, (v) determining a reference frame having the highest image quality of the PV array subsection from the extracted frames, (vi) performing image alignment of the extracted frames to the reference frame to generate image-aligned frames, and (vii) processing the image-aligned frames to produce an improved image of the PV array subsection having a higher resolution than the reference frame.

[0071] According to a sixth aspect, there is provided a system for capturing and processing EL images of PV array subsections of a PV array. The system includes an aircraft and an image processing device. The aircraft includes a camera for capturing EL images of the PV array, a propulsion device for actuating movement of the aircraft, and a controller communicatively coupled to the camera and the propulsion device, the controller being configured to: control the aircraft to fly along a route and capture EL images of corresponding PV array subsections of the PV array; derive respective image quality parameters from at least some of the captured EL images; and dynamically adjust a flight speed of the aircraft along the route based on the respective image quality parameters for capturing the EL images of the PV array subsections. The image processing device includes an image processor configured to: extract from the EL image a plurality of frames of a PV array subsection of the PV array, where the PV array subsection includes one or more PV modules of the PV array; determine from the extracted frames a reference frame of the PV array subsection having the highest image quality; perform image alignment of the extracted frames to the reference frame to generate an image-aligned frame; and process the image-aligned frame to produce an improved image of the PV array subsection having higher resolution than the reference frame.

[0072] Exemplary embodiments will now be described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0073] [Figure 1] FIG. 1 illustrates an exemplary setup for a UAV to capture EL images of a PV array. [Figure 2] FIG. 2 is a block diagram illustrating the system architecture of a system for capturing and processing EL images, the system including the UAV of FIG. 1. [Figure 3]FIG. 3 shows an optical subsystem that is part of the system of FIG. 2. [Figure 4] 3 is a flowchart of an exemplary method for capturing and processing an EL image implemented by the system of FIG. 2. [Figure 5] FIG. 5 is a schematic diagram of a UAV implementing a POINT function as part of the exemplary method of FIG. [Figure 6] 5 is a schematic diagram of a UAV performing a first portion of a FIND function as part of the example method of FIG. 4. [Figure 7] FIG. 7 is a schematic diagram of a UAV implementing a second portion of the FIND function of FIG. 6. [Figure 8] FIG. 8 is a schematic diagram illustrating the process and results of the FIND function of FIGS. 6 and 7. [Figure 9] FIG. 5 is a perspective view of a UAV hovering over a PV string with the camera FOV unaligned before the ALIGN function is performed as part of the example method of FIG. [Figure 10] FIG. 5 is a perspective view of a UAV hovering over a PV string with its camera FOV aligned after the ALIGN function is performed as part of the example method of FIG. [Figure 11] FIG. 10 shows an EL image from the viewpoint of a camera with misaligned camera FOVs according to FIG. 9. [Figure 12] FIG. 11 shows an EL image from the viewpoint of the camera with the camera's FOV aligned according to FIG. 10. [Figure 13] 12 is a schematic illustration of an EL image from the viewpoint of the controller with misaligned camera FOVs according to FIG. 11; FIG. [Figure 14] 13 is a schematic illustration of an EL image from the viewpoint of the controller with the camera FOV aligned, according to FIG. 12; FIG. [Figure 15] 15A-15F show a series of six consecutive EL images of a PV string captured by a camera during a SCAN function as part of the example method of FIG. 4. FIG. [Figure 16]16 is a schematic diagram of the series of EL images of FIG. 15 after image alignment to the PV strings has been performed. [Figure 17] FIG. 16 is a schematic diagram of two successive EL images from FIG. 15 showing point-to-point deflection. [Figure 18A] 5 is a line graph showing the current flight speed of a UAV decreasing over time while performing a SCAN function as part of the example method of FIG. 4. [Figure 18B] 5 is a line graph illustrating the current flight speed of a UAV increasing over time while performing a SCAN function as part of the example method of FIG. 4. [Figure 19A] 5 is a schematic diagram illustrating a time course of a UAV performing an AUTO function as part of the exemplary method of FIG. 4. [Figure 19B] 5 is a schematic diagram illustrating a time course of a UAV performing an AUTO function as part of the exemplary method of FIG. 4. [Figure 19C] 5 is a schematic diagram illustrating a time course of a UAV performing an AUTO function as part of the exemplary method of FIG. 4. [Figure 19D] 5 is a schematic diagram illustrating a time course of a UAV performing an AUTO function as part of the exemplary method of FIG. 4. [Figure 19E] 5 is a schematic diagram illustrating a time course of a UAV performing an AUTO function as part of the exemplary method of FIG. 4. [Figure 20] FIG. 20 is a schematic diagram of the file structure for stored EL images captured during the AUTO function of FIG. 19. [Figure 21A] 5 is a schematic diagram of frames extracted from three consecutive EL images during the FREEZE function as part of the exemplary method of FIG. 4. [Figure 21B] FIG. 5 is a schematic diagram of frames extracted from three consecutive EL images during the FREEZE function as part of the exemplary method of FIG. 4. [Figure 21C] FIG. 5 is a schematic diagram of frames extracted from three consecutive EL images during the FREEZE function as part of the exemplary method of FIG. 4. [Figure 22]FIG. 22 is a schematic illustration of the image refinement steps performed on extracted frames as part of the FREEZE function of FIG. 21. [Figure 23] FIG. 23 shows a pixel intensity histogram of the improved frame obtained from the image improvement step of FIG. 22. [Figure 24] FIG. 24 is a schematic diagram of an improved EL image produced by the FREEZE function of FIGS. 21-23. [Figure 25A] FIG. 5 is a schematic diagram of an improved EL image where the PV modules are arranged in two rows and two columns during a first portion of the MAP function as part of the exemplary method of FIG. 4. [Figure 25B] FIG. 25B is an alternative schematic diagram to FIG. 25A with an improved EL image having PV modules arranged in two rows and three columns. [Figure 25C] FIG. 25B is an alternative schematic diagram to FIG. 25A with an improved EL image having PV modules arranged in two rows and five columns. [Figure 26] FIG. 25C is a schematic diagram of an enhanced EL image mapped onto the base map during a second portion of the MAP function of FIGS. 25A-25C. [Figure 27] FIG. 27 is a schematic diagram of the UAV during a SCAN function to approximate the center position of the camera's FOV along the PV string in the third part of the MAP function of FIG. [Figure 28A] FIG. 28 is a schematic diagram of an exemplary enhanced EL image mapped onto an exemplary base map using the string alignment method in the fourth part of the MAP function of FIG. 27. [Figure 28B] FIG. 28B is a schematic diagram of an exemplary enhanced EL image mapped onto an exemplary base map using a modular alignment method as an alternative to the fourth portion of FIG. 28A. DETAILED DESCRIPTION OF THE INVENTION

[0074] The following description includes illustrative examples. Those skilled in the art will appreciate that variations and modifications to these examples are possible and are within the scope of the present disclosure. The drawings and the following description of specific embodiments should not be taken away from the generality of the preceding summary.

[0075] FIG. 1 shows an exemplary EL inspection apparatus or setup 100 for capturing EL images of a PV array 10 installed on the roof of a building. In this embodiment, the PV array 10 includes three PV strings 12. Each PV string 12 includes two rows of five PV modules 14. The PV strings 12 are arranged in each row along the longitudinal axis of the PV string 12, such that the PV array 10 has an array axis 10a running along the longitudinal axis of the PV string 12. The PV strings 12 are connected to a combiner box 16 that combines the electrical outputs of the PV strings. The combiner box 16 is connected to an inverter box (not shown), which is then connected to the power grid. The inverter box converts the combined electrical output from DC to AC before feeding the combined electrical output to the power grid. In this way, electricity generated by the PV modules 14 is fed to the power grid. During EL inspection, the PV array 10 is disconnected from the power grid.

[0076] The setup 100 further includes a switcher box 32 including three channels 34. Each PV string 12 of the PV array 10 is connected to a respective channel 34 of the switcher box 32. The setup 100 further includes a power supply 36 connected to the switcher box 32. The power supply is configured to supply each PV string 12 with electricity of up to 1000 volts and a minimum current equal to 10% of the short-circuit current of the PV module 14. By selectively activating the channels 34, the field personnel 30 selectively supply current to the PV strings 12 from the power source 36, placing the PV strings 12 under a forward bias condition. When placed in a forward bias condition, one or more PV modules 14 in the PV string 12 emit light, sometimes known as electroluminescence (EL), thus producing an EL signal that is detected by an optical subsystem of the aircraft (e.g., unmanned aerial vehicle (UAV) 20).

[0077] 1 , a worker 30 notices that a normally operating PV array 10 is generating less electricity than expected. Neither visual nor infrared inspection indicates the reason for this. After disconnecting the PV array 10 from the power grid and electrically connecting the PV array 10 to a power source 36 via a switcher box 32, the worker 30 instructs an assistant 22 to position the UAV 20 to capture EL images of the PV strings 12 for EL inspection. The UAV 20 includes a main body 210, a propulsion device 230 attached to the main body 210 for flying the UAV 20, and an optical subsystem 220 mounted on the main body 210 for capturing EL images.

[0078] While an assistant 22 is provided in this embodiment, it is clear that the worker 30 may deploy the UAV 20 without the assistance of the assistant 22. Furthermore, it should be noted that multiple PV strings 12 may be connected to one channel 34. For example, all three PV strings 12 of the PV array 10 may be connected to a single channel 34. In this scenario, all three PV strings 12 are simultaneously placed under a forward bias condition, and an EL image of the entire PV array 10 is captured. Notably, the amount of current supplied by the power supply 36 is lower in this scenario compared to when each channel 34 is connected to a respective PV string 12, but this does not affect the PV strings 12 that are placed under a forward bias condition.

[0079] Additionally, larger PV arrays may include multiple combiner boxes 16, which are then connected to an inverter box (not shown). Alternatively, the PV array 10 may not include a combiner box 16, and instead the PV strings 12 are connected directly to the inverter boxes.

[0080] Preferably, each PV string 12 is supplied with 100% of the short-circuit current of the PV module 14. However, this need not be the case. For example, each PV string 12 may be supplied with a current equal to 60% of the short-circuit current of the PV module 14. Measurements of the same PV array subsection at multiple injected currents may be used to estimate the electrical properties of the PV module 14 and to identify current-dependent defects.

[0081] 2 shows the system architecture of a system 200 for capturing and processing images. The system 200 includes an unmanned aerial vehicle (UAV) 20 and an image processing device 260. In addition to an optical subsystem 220 and a propulsion device 230, the UAV 20 further includes an on-board processing subsystem 240 and a power source 242 (e.g., a battery). The power source 242 is connected to and powers the optical subsystem 220, the propulsion device 230, and the on-board processing subsystem 240. The on-board processing subsystem 240 is communicatively coupled to the optical subsystem 220 and the propulsion device 230 and is configured to control the optical subsystem 220 and the propulsion device 230 to perform various functions.

[0082] The optical subsystem 220 will first be described with reference to FIG. 3 . The optical subsystem 220 includes a camera 222 having an optical axis 222 a, which in this embodiment is a video camera operable to take monochrome video recordings. The camera 222 is sensitive in the near and / or short infrared (NIR, SWIR) EL wavebands and is suitable for capturing EL images in such wavebands. The camera 222 includes a focusing lens 223 that is also suitable for use in the NIR / SWIR EL wavebands. The lens 223 (e.g., a motorized focus lens, a voltage-controlled polymer lens, or a liquid lens) allows the on-board processing subsystem 240 to adjust the focus of the lens 223 depending on the distance from the lens 223 to the PV array 10. The camera 222 further includes a lens filter (not shown) to filter out any unwanted light spectrum.

[0083] The optical subsystem 220 further includes an optical distance measurement device, such as a light detection and ranging device (LIDAR) 224. The optical axis (224a) of the LIDAR is aligned with the optical axis 222a of the camera 222. The LIDAR 224 is operable to measure the distance from the PV array 10 to the optical subsystem 220.

[0084] The optical subsystem 220 further includes a focused light source (e.g., a laser 226). The optical axis (226a) of the laser is also aligned with the optical axis 222a of the camera 222. The laser 226 is arranged to emit light in the visible spectrum and has a beam divergence equal to or less than the field of view (FOV) of the camera to minimize optical interference from the laser. Additionally, the laser 226 allows for low-power operation, emits light in a narrow wavelength band, and creates a focused shape that is easily identified by the operator 30. The focused shape is asymmetric, which helps the operator 30 identify where the camera 222 is pointing and also identify rotation of the camera's FOV.

[0085] The optical subsystem 220 further includes a single-axis gimbal 228 (shown in FIG. 2) that mounts the optical subsystem 220 to the body 210 of the UAV 20. The on-board processing subsystem 240 controls the gimbal to raise and lower the optical axis 222a of the camera 222 in one degree of freedom (i.e., pitch).

[0086] Alternatively, the optical subsystem 220 may be mounted to the body 210 via a two- or three-axis gimbal to allow additional degrees of freedom (yaw, roll) for adjusting the optical axis 222a of the camera 222 and provide enhanced stability of the FOV. Additionally, the focusing shape created by the laser 226 may be symmetric. An LED may be used in place of the laser 226. The focus of the lens 223 is adjustable and may be driven either mechanically or electrically.

[0087] 2, the propulsion device 230 will now be described. The propulsion device 230 includes four sets of propellers 232 driven by respective motors 234 to fly the UAV and perform aerial maneuvers such as rotating about the aircraft's yaw axis 210a (see FIG. 1). The yaw axis 210a is a vertical axis that runs through the center of the body 210 when the UAV 20 is upright.

[0088] The on-board processing subsystem 240 includes a controller 250 and a memory unit 252. The controller is configured to perform five functions (FOCUS, POINT, FIND, ALIGN, SCAN, and AUTO) according to a set of instructions stored in the memory unit 252. The controller 250 receives information from the optical subsystem 220, including the distance from the PV array 10 to the LIDAR 224 and the camera's field of view. Using the information received from the optical subsystem 220, the controller 250 is configured to operate the optical subsystem 220 and the propulsion device 230 to perform the function POINT, FIND, FOCUS, ALIGN, SCAN, and AUTO algorithms. Once the EL image is captured, the UAV 20 returns to its base and transfers the EL image to the image processing device 260 for further processing.

[0089] Image processing device 260 is configured to perform the functions FREEZE and MAP. Image processing device 260 includes a frame extraction module 270, an image enhancement module 280, a mapping module 290, and an image processor 300. Image processing device 260 takes an EL image as input and outputs an enhanced EL image of PV array 10.

[0090] The operation of each aircraft component is described in more detail in the following sections.

[0091] FIG. 4 is a block diagram of an exemplary method 400 for capturing and processing EL images by system 200. Exemplary method 400 is described in conjunction with corresponding FIGS. 5-26B, where applicable. In this embodiment, UAV 20 is positioned to perform EL inspection of PV array 40, preferably at night or under low natural light conditions. PV array 40 is similar to PV array 10, except that PV array 40 includes more PV modules. Each set of combiner boxes (not shown) from PV array 10 is connected to a switcher box 532 controlled by field worker 30.

[0092] In step 410 of method 400, controller 250 executes the POINT function. FIG. 5 is a schematic diagram 500 of UAV 20 implementing the POINT function. Once UAV 20 is deployed, controller 250 is configured to control power source 242 to provide power to laser 226. Operator 30 locates area 530 illuminated by laser 226 and identifies where camera 222 is pointing. The laser has a light intensity within a safe range (laser: class 1 or 2) that ensures that field operator 30 will not suffer any eye injury even if there is unintentional direct eye exposure to laser 226. Laser 226 is switched on during most of UAV 20 operation. This allows operator 30 to quickly identify where camera 222 is pointing, especially when it is not clear which PV string is currently under forward bias. The laser 226 is turned off immediately before the UAV 20 performs the SCAN function so that the laser 226 does not appear in the EL image captured by the camera 222 .

[0093] The technician 30 consults a string connection diagram, which informs the technician 30 which PV strings are placed under forward bias according to the activated / open channels. In this embodiment, the string connection diagram contains an error, and the technician 30 is informed that for a particular channel, PV string 512a is placed under forward bias. In reality, another PV string 512b is placed under forward bias, and one or more PV modules 514b of PV string 512b emit an EL signal. PV strings 512a and 512b are part of the PV array 40 and are also referred to as PV array subsections 512a and 512b of the PV array 40.

[0094] After activating a particular channel, operator 30 manually guides UAV 20 to PV string 512a along track 520. Operator 30 notices that an EL signal is being emitted by PV string 512a and infers that there is an error in the string connection diagram. To determine the location of PV string 512b, which is under forward bias, i.e., emitting an EL signal, operator 30 instructs controller 250 to initiate a FIND function.

[0095] Note that operator 30 does not need to manually guide UAV 20 to PV string 512a. Operator 30 may initiate the FIND function immediately after placing UAV 20, thereby avoiding the POINT function. Alternatively, controller 250 may be configured to automatically initiate the FIND function once UAV 20 is placed.

[0096] In step 420 of method 400, controller 250 executes the FIND function. FIG. 6 is a schematic diagram 600 of UAV 20 performing a first portion of the FIND function. Note that operator 30 and illumination area 530 are not shown in FIG. 6a (and subsequent figures). The first portion of the FIND function involves setting UAV 20 to an initial position. Upon starting, controller 250 is configured to dynamically adjust propulsion device 230 to steer UAV 20 to an initial position in which UAV's yaw axis 210a is perpendicular to the ground. Controller 250 is further configured to dynamically adjust gimbal 228 so that camera optical axis 222a is also perpendicular to the ground, or in other words, the camera's field of view points to the area directly below UAV 20. In this position, optical axis 222a has an angle of 0°.

[0097] Additionally, the controller 250 is further configured to dynamically adjust the propulsion device 230 to steer the UAV 20 (along the UAV's yaw axis 210 a ) from the ground to a predetermined altitude 610 .

[0098] FIG. 7 is a schematic diagram 700 of UAV 20 performing a second portion of the FIND function. The second portion of the FIND function involves performing a sweep of the PV array 40 in a sweep path that begins in an area directly below the UAV 20 and spirals outward. To perform the sweep, controller 250 is configured to dynamically adjust propulsion device 230 to rotate UAV 20 (see arrow 710) about the UAV's yaw axis 210a. Controller 250 is further configured to simultaneously increase (720) the camera's optical axis angle, which causes the camera's FOV to move outward from the UAV 20. In combination with rotation 720, the camera's scan path forms a spiral 810. This is illustrated in FIG. 8, which shows a schematic diagram 800 illustrating the results of the second portion of the FIND function.

[0099] For example, in an initial position, the UAV hovers 10 m above the ground. When the camera's optical axis angle is rotated from 0° to 70°, a radiated area of 55 meters is observed (using the sine law relationship in equation (a) below). The camera 222 has a 60° angle of view. As a result, for a view pointing directly towards the ground, the field of view is 10 m (equation (b)). Therefore, three rotations are sufficient to cover the PV array 40, which has a radius of 25 m.

[0100]

number

[0101]

number

[0102] Note that the camera's optical axis angle increases with decreasing pitch rate. From the gimbal 228's perspective, the gimbal's pitch rate decreases with increasing pitch angle. Because the scan path 810 increases by 2π multiplied by the radius, the camera's FOV travels along a distance five times greater (2π·5 m≈31; 2π·25 m≈157 m) in its final rotation. As a result, the camera's rotation 710, or yaw rate, is adjusted to be five times lower in the final rotation.

[0103] The camera's yaw rate depends on the amount of motion blur that can be tolerated in the frame during the exposure time. For a maximum deflection during an exposure of 10 pixels, a 7 ms exposure time, and a horizontal sensor resolution of 640 px allows a yaw rate of 22 m / s (equation (c)). Using an approximated travel distance for a full spiral of 282 m (the sum of three circles with radii of 5 m, 15 m, and 25 m), method FIND can take up to 13 seconds if no forward-biased PV string is detected. While function FIND is in progress, controller 250 checks images captured by camera 222 for the presence of a characteristic of forward-biased PV string 512b. Function FIND stops when an EL signal is detected from PV string 512b. Once EL detection occurs, controller 250 is configured to dynamically adjust propulsion device 230 to steer UAV 20 to the EL signal. In this manner, the EL signal is used as an optical marker to guide UAV 20.

[0104]

number

[0105] At step 430 of method 400, controller 250 performs a FOCUS function. Controller 250 receives information from camera 222 regarding the distance to one or more PV modules in PV string 512b from LIDAR 224. Controller 250 is configured to dynamically adjust the focus of the camera according to the measured distance to match the distance between camera lens 223 and the focal point to the distance between camera lens 223 and the imaged target to maintain the focus of the camera lens.

[0106] In step 440 of method 400, controller 250 performs the ALIGN function, which is described next in conjunction with FIGS. 9-14. FIGS. 9 and 10 are perspective views 900, 1000 of UAV 20 hovering above PV string 512b with the camera's FOV misaligned and aligned with PV string 512b, respectively. FIGS. 11 and 12 are EL images 1100, 1200 from the perspective of camera 222 with the camera's FOV misaligned and aligned, respectively. FIGS. 13 and 14 are schematic diagrams 1300, 1400 of EL images 1100, 1200 from the perspective of controller 250.

[0107] 9, the camera's FOV 910 is not aligned with the PV string 512b. The camera's FOV 910 must be aligned with the PV string 512b (as shown in FIG. 10) before the method 400 can proceed to the SCAN function.

[0108] The controller 250 receives an EL image 1100 from the camera 222 (as shown in FIG. 11 ). The EL image 1100 includes a PV string 512b (or a portion of a PV string 512b) that appears brighter (has higher light intensity) due to the EL signal emitted by one or more PV modules 514b compared to a background 1110, i.e., the ground. The controller 250 applies an algorithm to determine that the camera's FOV 910 is misaligned with respect to the PV string 512b. The algorithm uses the intensity difference between the bright PV string 512b and the dark background 1110 to detect the position and orientation of the PV string 512b (or of the reference PV module 514b). Specifically, and with reference to FIG. 13 , the algorithm detects an edge 1310 of the PV string and derives key points 1320 (e.g., module corner points) from the edge 1310. The algorithm then determines a set of aligned points 1330 corresponding to each key point 1320 that minimizes misalignment and any angle or perspective distortion. The aligned points 1330 are set on upper and lower horizontal indicators 1340.

[0109] Additionally, the algorithm also determines an appropriate altitude of the UAV 20 relative to the PV string 512b that places the PV string 512b at a predetermined size ratio within the camera's FOV 910. The predetermined size ratio is set to maintain approximately 5-10% space between the top and bottom of the PV string 512b and the image boundary 1410 to allow for tolerance to position oscillations of the UAV 20. In other words, the PV string 512b occupies 80%-90% of the camera's FOV at the predetermined size ratio.

[0110] The algorithm then determines a perspective transformation to align key point 1320 with aligned point 1330 (as shown in FIG. 14 ). Controller 250 is then configured to make appropriate adjustments to gimbal 228 and propulsion device 230 based on the perspective transformation. As can be seen in FIG. 14 , camera FOV 910 is aligned with PV string 512b, which is at a predetermined size ratio within camera FOV 910. When aligned, the optical axis of the camera is perpendicular to the plane of the PV string. Because the PV string includes one or more PV modules 514b, the plane of the PV string consists of the plane of the one or more PV modules. The plane of the one or more PV modules is defined as the surface prepared to receive sunlight.

[0111] It should be noted that the controller 250 is configured to repeatedly perform the ALIGN function while the SCAN function is in progress. This ensures that the optical axis of the camera is perpendicular to the plane of the PV string while the EL image is being captured during the SCAN function. Advantageously, this minimizes perspective distortion and increases the image resolution of the EL image captured by the camera 222. Furthermore, it allows the camera 222 to capture the EL image with a more consistent focus across the EL image. Furthermore, the EL intensity from each PV module 514b is accurately captured, which is important for analysis purposes.

[0112] It should be noted that if the controller 250 does not detect the end 1210 of the PV string 512b in the EL image 1200, the controller 250 is configured to adjust the propulsion device 230 to steer the UAV 20 along the longitudinal axis 10a of the PV string (see FIG. 1) until the end 1210 of the PV string 512b enters the EL image 1200.

[0113] In step 450, controller 250 performs a SCAN function, which is now described with reference to Figures 15-17. Figure 15 includes Figures 15A-15F, each showing six consecutive EL frames of a series 1500 of PV string 512b captured by camera 222 at different locations along PV string 512b. It should be noted that the EL frames overlap such that a PV module 1510 is likely to appear multiple times, i.e., in Figures 15A-15E.

[0114] When considering the large PV installation and the limited flight time of the UAV 20, the scanning speed becomes a critical parameter in determining the system efficiency. A longer camera exposure time usually results in better image quality, i.e., a better signal-to-noise ratio (SNR). However, a long camera exposure time coupled with a fast scanning speed causes motion blur, which reduces image quality. On the other hand, a short camera exposure time results in an EL image with excessive noise, especially when the injection current is low, which also reduces image quality.

[0115] As the PV module 1510 appears in Figures 15A-15F, there are five frames of the PV module 1510 (also extracted at a later stage) available for image averaging (at a later stage). The SNR of the image average 1520 is approximately the number of frames available for image averaging (n frames ) In other words, the image noise of the image average 1520 decreases with the number of frames used to create the image average. The number of frames available for image averaging also depends on the scan rate and is calculated in real time by the controller 250.

[0116] Furthermore, even if the ALIGN function is repeatedly performed during the SCAN function, it is difficult for the camera's FOV to remain completely stable throughout the SCAN function. This is especially true when considering positional oscillations of the UAV 20 due to external forces acting on the UAV 20 (such as wind). This is evident in FIG. 16, which is a schematic diagram of a sequence 1500 of the EL images of FIGS. 15A-15F after image alignment to PV string 512b has been performed. The positional oscillations of the UAV 20 are evident in the point-to-point deflection between the EL images along PV string 512b. While the positional oscillations are mitigated (and largely corrected) by the ALIGN function, any calculation of the scan rate must take this deflection into account while limiting noise and motion blur.

[0117] Noise Limits: Controller 250 is configured to perform optical flow analysis (e.g., the Lucas-Kanade method) during the SCAN function. For each frame of Figures 15A-15F captured by camera 222, controller 250 calculates key points in the current frame and compares the key points in the current frame with key points in the previous frame to determine the length of the deflection vector.

[0118] 17 is a schematic diagram 1700 of two successive EL images (i.e., FIGS. 15D and 15E) of PV string 512b showing point-to-point deflection. The previous EL image in FIG. 15E is shown with a dotted line, and the current EL image F is shown with a thick line. The deflection of object 1710 at image center 1720 is calculated from the average deflection of detected key points in the EL images of FIGS. 15D and 15E. The length of this deflection vector is d f2f It is called.

[0119] The controller 250 calculates a line 1730 that passes through the image center 1720 and is the deflection angle. The line 1730 intersects with the image boundary 1740 at intersection points 1750, 1760. The distance between intersection points 1750, 1760 represents the object distance moved through the image plane. The controller 250 then calculates the length d of the deflection vector. f2f By taking the ratio of the distance between the intersection points 1750 and 1760, n frames Calculate.

[0120] The effect of noise on image quality can be quantified by the SNR.

[0121]

number

[0122] In this embodiment, the SNR is calculated as follows: The captured image of Otsu's method is the threshold (t Otsu ) is used to obtain the threshold, i.e., t Otsu The "Noise" value is obtained by averaging the intensity of all pixels brighter than 1000 kJ / s. The "Noise" value is obtained, prior to the EL measurement, from the average of the standard deviations of the pixels in several images taken in sequence with similar or comparable imaging parameters (e.g., exposure time, sensor temperature and gain).

[0123] The SNR-dependent scan rate factor (or simply the SNR scan factor), i.e., f SNR is applied to the current scanning speed using equation (2) to obtain the SNR of the image average 1520, i.e., SNR frame is sure to reach the target SNR, i.e., SNR target to match.

[0124]

number

[0125] For example, SNR target is set to 45 for lab measurements. The camera exposure time is frame is adjusted during the SCAN function to keep n at 5 (the minimum requirement for outdoor measurements). The controller 250 then determines whether 25 EL images are available for image averaging (n frame =25). Based on equation (1), the following relationship in equation (d) holds: In other words, the current scanning rate should be reduced to 56% of its current value. Essentially, reducing the scanning rate reduces the number of frames (n frames ) increases.

[0126]

number

[0127] Motion Blur Limits: To avoid the effect of motion blur, the object deflection (d exp ) is a predetermined maximum value (d exp_max ) should also be below 0.75 pixels per exposure time. A value of 0.75 pixels per exposure time is suggested. f2f ) is calculated by dividing the two frames (t f2f ) and exposure time (t exp ) to calculate the exposure time deviation (d exp )

[0128]

number

[0129] Motion blur dependent scanning factor (f blur ) is the maximum object deflection d as shown in equation (4). exp_max and the current object deflection d exp is equal to the ratio of

[0130]

number

[0131] As shown in equation (5), both factors (f SNR , f blur ) from the smallest of the scan rate factors (f scan ) is obtained. f scan =min(f SNR ,f blur ) (5)

[0132] To ensure high EL image quality, the maximum set scanning speed, i.e., v quality is obtained using equation (6). v quality =min(v max ,f scan ·v cur ) (6)

[0133] The maximum set scan speed, i.e., v quality defines the maximum scan speed at which high EL image quality can still be achieved.

[0134] According to equation (6), the scanning speed factor, i.e., f scan Let v be the current flight speed. cur The target scanning speed is calculated by multiplying by . If the target scanning speed is equal to the maximum flight speed of the UAV 20, i.e., v max If the target scan speed is less than v, the target scan speed is the maximum set scan speed, i.e., v quality In other words, the UAV 20 is selected as its maximum flight speed, i.e., v max Even if it is possible to move faster, this will reduce the image quality of the EL image, so it is recommended to use the maximum set scanning speed, i.e., v quality is the maximum flight speed v max is set to be less than

[0135] The maximum set scanning speed is the maximum flight speed of the UAV20, i.e., v max If it exceeds the maximum flight speed, i.e., vmax is the maximum set scanning speed, i.e., v quality is selected as.

[0136] The target flight speed, i.e., v target is the maximum set scanning speed, i.e., v quality , and the user input factor, i.e., f user is obtained according to equation (7). v target =v quality f user (7)

[0137] User input factor, i.e., f user is obtained from the deflection of a joystick remotely controlled by operator 30 and ranges from 0% to 100%. At 100%, the target flight speed, i.e., v target is simply the maximum set scan speed, i.e., v quality is.

[0138] To minimize jerky movements of the UAV 20, a smoothing technique is used to adjust the target flight speed, i.e., v target In this embodiment, the exponential moving average is applied to the set speed, i.e., v set is used to obtain the smoothing factor α, which ranges from 0 to 100%. v set =(α v cur )+((1-α)·v target ) (8)

[0139] For two exemplary embodiments of the SCAN function, the current flight speed of the UAV 20, i.e., v, decreases and increases over time according to the SCAN function, respectively. cur 18A and 18B, which are line graphs 1800a, 1800b illustrating the user input factor, f user is set to 100%, and the maximum flight speed of the UAV20, i.e., v max is said to be 9 m / s.

[0140] Referring to FIG. 18A, at time=1 s and the UAV 20 is at a current flight speed of 6 m / s, i.e., v cur When operating at , the controller 250 adjusts the scan rate factor to 50%, i.e., f scan1 is obtained from equation (5). If the scan rate factor is below 100%, this indicates that the UAV 20 is moving faster than it should. The controller 250 then determines that the target scan rate is 3 m / s. If the target scan rate is less than the maximum flight speed of the UAV 20, i.e., v max Since the target scanning speed is less than the maximum set scanning speed, i.e., v quality is selected as.

[0141] It should be noted that the user input factor, i.e., f user is 100%, the maximum set scanning speed, i.e., v quality Also, the target flight speed, i.e., v target is.

[0142] The controller 250 then calculates the target flight speed, i.e., v target The smoothing technique according to equation (8) is applied to minimize the jerking motion of the UAV 20, which reduces the current flight speed of the UAV 20, i.e., v, from 6 m / s (at time=1 s) to 3 m / s (at time=2 s). cur This can be seen in the smooth transition between

[0143] At time=2 s, the controller 250 sets the scan rate factor to 100%, i.e., f scan2 is obtained from equation (5). At this point, the current flight speed of the UAV 20, i.e., v cur is the maximum set scanning speed, i.e., v quality matches.

[0144] Referring to FIG. 18B, at time=1 s and the UAV 20 has a current flight speed of 3 m / s, i.e., v curWhen operating at , the controller 250 generates a scan speed factor of 150%, i.e., f scan1 is obtained from equation (5). If the scan rate factor is greater than 100%, this indicates that the UAV 20 can move 50% faster while still meeting the required image quality. The controller 250 then determines that the target scan rate is 4.5 m / s.

[0145] The target scanning speed is the maximum flight speed of the UAV20, i.e., v max Since the maximum flight speed is less than v max is the maximum set scanning speed, i.e., v quality The target scan rate is chosen to be the maximum set scan rate, i.e., v quality is selected according to equation (6).

[0146] Similarly, the user input factor, i.e., f user is 100%, the maximum set scanning speed, i.e., v quality Also, the target flight speed, i.e., v target The smoothing technique according to equation (8) is also applied to minimize jerky movements of the UAV 20.

[0147] The controller 250 then calculates the target flight speed, i.e., v target The aircraft's current flight speed, i.e., v cur At time=2 s, the controller 250 dynamically increases the scan rate factor, i.e., f scan2 is obtained from equation (5). At this point, the current flight speed of the UAV 20, i.e., v cur is the maximum set scanning speed, i.e., v quality matches.

[0148] It should be noted that the controller 250 continuously performs the SCAN function until the opposite end of the PV string 512b is detected, at which point the controller 250 terminates the SCAN function and the EL image is stored in the memory unit 252.

[0149] In step 460, the controller 250 is configured to execute the AUTO function. Figure 19, comprising Figures 19A-19E, is a schematic diagram illustrating a time course of the UAV 20 capturing EL video images of the PV array 40 (using the PV array 40 of Figure 5 as an example). The PV array 40 is shown to include two rows 1810, 1820 of PV strings. The controller 250 is communicatively coupled (via a wireless connection) to and can control the power source 36 and a switcher box 532, which is electrically connected to the PV array 40.

[0150] At t1, as shown in FIG. 19A, the UAV 20 starts at the end 1810a of the first row 1810 and captures an EL image of the first PV string 1812a in the scanning direction 1830. Upon detecting the end 1814a of the PV string 1812a, the controller 250 is configured to command the switcher box 532 to close the current channel and open the next channel for the next PV string, i.e., 1812b. Note that the current injected into PV string 1812b is maintained at the same level as that of PV string 1812a. The process continues until the controller 250 detects that the last PV string 1812f at the opposite end 1810b of the first row of PV strings 1810 has been reached at t2, as shown in FIG. 19B.

[0151] At t3, as shown in FIG. 19C , the controller 250 is configured to instruct the power supply 36 to reduce the current injected into the PV string 1812f. The purpose of reducing the injection current is to capture a low-current EL image for comparison with the higher current EL image. The controller 250 then dynamically adjusts the propulsion device 230 to move the UAV 20 in a scanning direction 1840 that is opposite to the scanning direction 1830. The UAV 20 moves along the scanning direction 1840 to capture EL images of the PV strings in the first row 1810 at the lower injection current.

[0152] 19D shows that at t4, the controller 250 detects that the UAV 40 has reached the end 1810a of the first row 1810 of PV strings. At this point, the controller 250 is configured to instruct the switcher box 532 to close the current channel and open the next channel to place the first PV string 1820a of the second row 1820 under forward bias. The controller 250 is further configured to perform a FIND function to locate the EL signal emitted by the PV string 1820a. It should be noted that the PV string 1820a is within the FOV of the camera during the FIND function, and the controller 250 is able to locate the PV string 1820a.

[0153] At t5 in FIG. 19E, the controller 250 is configured to navigate the UAV 20 to the PV string 1820a. When the UAV 20 reaches the PV string 1820a, the controller 250 is configured to perform the SCAN function again. EL images of the PV strings in the second row 1820 are captured using a process similar to that used to capture the EL images of the PV strings in the first row 1810 (detailed at t1-t4). Note that the controller 250 is configured to perform the ALIGN function throughout the duration of the SCAN function. The captured EL images are stored in the memory unit 252.

[0154] FIG. 20 is a schematic diagram 1900 of a file structure for a stored EL image. The file structure includes a stored EL image 1910 and an append block containing additional metadata 1920. The metadata 1920 is separated into a header 1930 and a body 1940. The header 1930 includes information about image correction methods applied to the EL image 1910 before it is saved. Image correction methods include dark current subtraction, flat-field correction, bad pixel replacement, and lens distortion removal. The body 1940 stores image-dependent data including camera exposure time and gain, UAV geolocation, camera orientation (yaw, pitch, roll), injection current, voltage, and channel information.

[0155] The camera 222 captures / digitizes the EL images 1910 at a bit depth greater than 8 bits (e.g., 14 or 16 bits). This allows the image intensity range to be resolved more precisely than the 255 brightness increments of a monochrome 8-bit sensor. To reduce file size, an image encoder based on 8-bit images is used. The upper and lower intensity ranges of each EL image 1910 are stored in the metadata 1920. The upper and lower intensity ranges are obtained from the available dynamic range of each EL image 1910 captured by the camera 222. The ranges can be used to scale any 8-bit EL image to the respective lower and upper intensity ranges of the original higher depth camera image.

[0156] When the UAV 20 returns to its base, the stored EL images are then transferred to the image processing device 260 for further processing.

[0157] In step 470, image processing device 260 performs the FREEZE function, which will now be described with reference to Figures 21-24. For simplicity, the FREEZE function will be described with reference to processing an EL image that includes only PV string 512b. It should be understood that the FREEZE function may similarly process any EL image.

[0158] The FREEZE function includes (i) a frame extraction step performed by a frame extraction module 270 and (ii) an image enhancement step performed by an image enhancement module 280 .

[0159] FIG. 21 includes FIGS. 21A-21C, which respectively show three consecutive EL frame images 2000a, 2000b, and 2000c of a PV string 512b that are processed as part of the frame extraction step. Image processor 300 instructs frame extraction module 270 to determine, from all three EL frame images 2000a, 2000b, and 2000c, a respective corner point 2010 of each PV module 514b in the PV string 512b. The detected corner points of each PV module 514b are shown as black dots 2012 in each EL frame image 2000a, 2000b, and 2000c. It should be noted that, in this embodiment, frame extraction module 270 cannot detect a specific corner point 2014 of a specific PV module 2016 in the first EL frame image 2000a of FIG. 21A.

[0160] All corner points 2010 detected in the first EL frame image 2000a of FIG. 21A are visualized as empty dots 2020 in the second EL frame image 2000b of FIG. 21B. It should be noted that empty dots 2030 are slightly shifted to the left compared to their corresponding black dots 2012 in the first EL frame image 2000a of FIG. 21A. This is due to the correction of the point-to-point bias. When the local corner point density exceeds a certain threshold (two detected corner points 2010 are sufficient in this embodiment), the image processor 300 controls the frame extraction module 270 to generate cluster points 2022. The location of the cluster points is obtained by averaging all corner points 2010 used to generate the cluster points 2022. This advantageously reduces the spatial detection error of individual corner points.

[0161] Referring to FIG. 21C , when a cluster point 2022 moves outside the boundary of an EL frame image, as shown by cluster point 2030 in EL image 2000c, the cluster point 2022 is stored. When too many EL frame images no longer add corner points 2010 to the cluster point, the cluster point 2022 within the boundary of the EL frame image is discarded. Cluster points 2022 within the boundary of an EL frame image, such as cluster point 2032 in EL image 2000c, are kept as long as the ratio of EL frame images that add new corner points 2010 to EL frame images that do not add corner points 2010 exceeds a certain threshold. All cluster points 2022 are then meshed by frame extraction module 270 to generate / construct quadrilaterals or frames, such as rectangles. Each frame is constructed from the cluster points 2022 of each PV module contained in the frame.

[0162] 22 shows that frames 2100 extracted from the frame extraction step are processed in the image refinement step. The image processor 300 instructs the image refinement module 280 to assign horizontal and vertical module indices 2110 to each frame 2100 according to its position in the PV string 512b. For example, the first frame associated with a particular PV module 2514 is located in the first row and first column of the PV string 512b and is assigned horizontal and vertical module indices [1,1].

[0163] The image processor 300 further controls the image refinement module 280 to group the frames 2100 according to the PV modules 514b included in each frame (or equivalently according to their module indexes 2110). Each frame 2100 includes four cluster points 2022 (marked "A" through "D" respectively). The frames 2100 within each group are then arranged in a stacked arrangement (called a stack) such that cluster points 2022 marked with the same alphabet ("A" through "D") are stacked on top of each other. An example stack 2120 with module index [1,1] is shown in FIG. 22.

[0164] The image refinement module 280 is further configured to discard areas 2124 that are not part of the frame 2100 .

[0165] Using the exemplary stack 2120 as an example, the image refinement module 280 is further configured to determine the reference frame with the highest image quality from the frames 2100 in the exemplary stack 2120. The image quality is evaluated based on the sharpness, SNR, and completeness of the PV modules 514b in the frame.

[0166] The image refinement module 280 is further configured to perform image alignment of the frame 2100. This is done using an image alignment algorithm such as "Parametric Image Alignment using Enhanced Correlation Coefficient." The image refinement module 280 aligns the remaining frames 2100 in the example stack 2120 to the reference frame. The image alignment is done by aligning the cluster points 2022 marked "A" in the remaining frames to the corresponding cluster points 2022 marked "A" in the reference frame to obtain an image-aligned frame 2130.

[0167] The image refinement module 280 is further configured to perform image averaging on the image-aligned frames 2130 to obtain an refined frame 2140 of the particular PV module 2514. The image averaging is performed using super-resolution routines such as weighted image stack averaging 2032 and / or a dedicated deep convolutional network structure 2034. The refined frame 2140 has a higher SNR (i.e., lower noise) and higher resolution (up to a resolution enhancement factor of 3) than the reference frame.

[0168] The same process is repeated for the remaining stacks to obtain respective improved frames for the remaining PV modules 514b in the PV string 512b. The image processor 300 further controls the image improvement module 280 to determine respective corner points 2010 of each improved frame and to remove any remaining perspective distortion in the improved frame.

[0169] The image processor 300 further controls the image improvement module 280 to order the improved frames according to their module indexes 2110 to produce improved EL images of the PV strings. If the distances between the PV modules 514b in the PV string 512b are similar, a single improved EL image is produced. If the distances vary due to a large air gap 2112 between the two PV modules (indicating that one of the PV modules belongs to a separate PV string 512c), separate improved EL images are produced for the separate PV strings 512c.

[0170] The image enhancement module 280 is further configured to scale the image intensity of each enhanced frame to reflect the intensity spectrum from the darkest to the brightest PV module 514b in the PV string 512b. Because the intensity scaling reduces the depth resolution of the intensity range of the PV modules, the enhanced frame for each PV module 514b is stored along with the enhanced EL image of the PV string.

[0171] During image processing, image intensities are represented by real or floating-point values. When the resulting image is visually displayed, the image intensities must be assigned brightness values between the darkest and brightest displayable values. To reduce the effect of pixels with excessive image (or pixel) intensity values, a brightness range containing a minimum number of pixels (previously called the effective dynamic range) is defined by the lowest pixel intensity bin (dotted line 2201a) and the highest pixel intensity bin (dotted line 2201b).

[0172] Figure 23 shows the pixel intensity histogram of the refined frame 2140. Two peaks 2200a, 2200b can be seen, corresponding to pixel intensities of the dark background and the bright EL signal, respectively. The brightness range between the lowest pixel intensity bin (dotted line 2201a) and the highest pixel intensity bin (dotted line 2201b) is defined to exclude pixel intensities that do not occur in many pixels. In Figure 23, pixel intensities below the lowest pixel intensity bin (left of dotted line 2201a) are set to the minimum value (0), and pixel intensities above the highest pixel intensity bin (right of dotted line 2201b) are set to the minimum value (0) for a chosen precision (2 for an 8-bit image). 8 The image intensity is scaled between the minimum (0) and maximum (255) values of the defined brightness range.

[0173] 24 shows an improved EL image 2200 of PV string 512b after the image improvement step is completed; note that the resolution of improved EL image 2200 is better than the resolution of the reference frame. Note that regardless of the scanning direction 2210, i.e., left-to-right, right-to-left, top-to-bottom, or any other combination, improved EL image 2200 is always aligned with the camera's yaw (i.e., optical axis 222a) because EL images 2000a, 2000b, 2000c are captured with the camera's FOV 910 aligned with PV string 512b. For ease of reference, a black triangle 2220 is used to indicate the lower left corner of improved EL image 2200, where improved EL image 2200 is aligned with the camera's FOV 910.

[0174] Existing known methods may then be used to process the improved EL image 2200 to identify any defective PV modules in the PV string 512b based on the EL imaging. Now that the defective PV modules have been identified, it may be useful to know the geolocation of the defective PV modules. For this purpose, the mapping module 290 may be used.

[0175] The image processor 300 controls the mapping module 290 to perform the MAP function as shown in Figures 25-28. The mapping module 290 is configured to map the improved EL image 2200 of the PV string 512b onto the base map of the PV string 2410. To identify the location of the PV module 514b in the PV string 512b (and therefore any defective PV modules) from the improved EL image 2200, frame-dependent timed geolocation (time, latitude, longitude, altitude, etc.) and camera orientation (e.g., yaw, pitch, roll) information is captured, processed, and stored during the SCAN function. This information is stored in the metadata 1920 of every captured EL image / video (see Figure 20).

[0176] Image processor 300 controls mapping module 290 to map the improved image onto the base map by orienting the improved image to align the PV array subsections in the improved image with the PV array subsections in the base map. If the geolocation of PV string 512b in improved EL image 2200 is known but its orientation (i.e., camera yaw) is unknown, there are four possible orientations (0°, 90°, 180°, and 270° rotation) with respect to black triangle 2220 to align the image with the PV string in the base map layer. Commercial PV modules typically have cell grids of 4×8, 6×10, or 6×12 cells and are generally rectangular, not square. In such cases, only two of the four orientations, namely, 0° and 180° rotation with respect to black triangle 2220, seem reasonable. This is explained in more detail with reference to Figures 25A, 25B and 25C, which show three modified EL images 2310, 2320, 2330 in which the PV modules are arranged in two rows and two, three and five columns, respectively.

[0177] The improved EL image 2310 in Figure 25A has the same number of rows and columns. Due to the shape of the PV module 2314, two of the four orientations (90° and 270° rotations relative to the black triangle 2220) will severely distort the improved EL image 2310 when it is mapped onto the base map of that PV string.

[0178] 25B has a square shape, the enhanced EL image 2320 has a different number of PV modules 2324 in its rows and columns. In two of the four orientations (90° and 270° rotations relative to the black triangle 2220), the number of PV modules 2324 in a row or column does not match the number of PV modules 2324 in the same row or column of that PV string in the base map.

[0179] The enhanced EL image 2330 in Figure 25C is shown with two orientations: 0° and 180° rotation relative to the black triangle 2220. As can be seen, only these two orientations result in an equally undistorted image matching its base map.

[0180] FIG. 26 shows an enhanced EL image 2200 mapped onto a base map 2410 of a PV string 512b. The PV string 512b has a vertical axis (i.e., the array axis 10a in FIG. 1 ), and the camera 222 is arranged to capture EL images of the PV string 512b at a predetermined size ratio, so that no EL image captures the entire PV string 512b. Specifically, in this embodiment, 11 EL images of the PV string 512b are captured. Each EL image is associated with a unique image identifier 2420. In this embodiment, the EL images are numbered 1 through 11 in ascending order.

[0181] When a particular EL image is used for image averaging to produce a refined frame, the image processor 300 further instructs the mapping module 290 to associate the image identifier 2420 of the particular EL image with the refined frame. For example, refined frame 2430 is associated with unique numeric identifiers "2," "3," "4," "5," and "6." In other words, corresponding frames extracted from EL images "2," "3," "4," "5," and "6" are used for image averaging to produce refined frame 2430.

[0182] The mapping module 290 can then determine the orientation of the improved image 2200 (from two available orientations, i.e., 0° and 180° rotation relative to the black triangle 2220) based on the image identifier 2420 associated with each improved frame. The improved frame associated with image identifier "1" represents a frame captured at the start of the SCAN function, as opposed to the improved frame associated with image identifier "11", which represents a frame captured at the end of the SCAN function.

[0183] Further indicators are discussed for the location and orientation of the enhanced EL image 2200 within the base map 2410. The approximate center position 2440 of the camera's FOV 910 along the PV string 512b can be calculated from the UAV's trajectory 2450 (including the flight start 2452 and flight end 2454), flight altitude, and camera orientation.

[0184] 27 is a schematic diagram 2500 showing a side view of the UAV 20 during a SCAN function to approximate the center position 2440 of the camera's FOV 910 along the PV string 512b. z The camera 222 has an altitude 2510 above the ground, denoted by α. The camera 222 is aligned at a pitch angle 2520 relative to the ground, denoted by α. The pitch angle 2520 is aligned to the tilt angle 2530 of the PV string 512b relative to the ground. The distance 2540 between the camera 222 and the PV string 512b is d L The distance 2540 is readily available from the LIDAR reading or as measured by the LIDAR 224 during the FOCUS function. The horizontal distance 2550 between the UAV 20 and the PV string 512b is d xy The horizontal distance 2550 is then calculated using equation (3), i.e. d xy,1 =d L cos(α) (9) Using d L Based on Eq. (4), i.e. d xy,2 =(d z -d pv )·tan(α) (10) Using d z and the height of the PV string 512b from the ground 2560, d pv It can be calculated based on either

[0185] Equation (9) is the height of the object to be imaged (d pv ) is known or estimated xy,2 This is preferred over equation (10) because it requires d. Furthermore, the UAV 20 is subject to errors due to longer flight times and weather changes. zis estimated using a barometer.

[0186] 27, the center position 2440 of the camera's FOV 910 is within PV string 512b while the UAV's track 2450 is slightly below. With high quality location data and simple PV string interconnections (as is the case for PV string 512b), the mapping module 290 can accurately map the enhanced EL image 2200 onto the base map 2410 without further input. With more complex PV string interconnections and / or lower quality location data, further input for the PV strings is obtained by comparing the number of measurements, switcher box channels, and information about which inverter box or combiner box 16 is connected to which switcher box channel and when.

[0187] Figure 28A shows an exemplary enhanced EL image 2610 being mapped onto an exemplary base map 2620 using the string alignment method. Figure 28B shows an exemplary enhanced EL image 2630 being mapped onto an exemplary base map 2640 using the module alignment method.

[0188] 28A , for the string alignment method, the corner points 2622 of the base map 2620 are known, and the improved EL image 2610 is roughly aligned on top of the base map 2620. If the base map contains multiple PV strings under the improved EL image 2610, the PV array 2620 that shares the most overlapping area with the improved EL image 2610 is chosen. The four corner points 2612 of the improved EL image 2610 are then affinely or perspectively aligned 2624 by the mapping module 290 to the corner points of the PV strings in the base map 2620 and 2622 according to their most similar orientation.

[0189] 28B , in this embodiment, the number of PV modules in the improved EL image 2630 does not match the number of PV modules in the base map 2640. This occurs when the improved EL image is generated from only a portion of the scanned PV strings or when the interconnections of the PV modules in the PV strings do not follow a regular pattern. In this case, the number of modules in each row and column of the PV string must be known, as well as the corner points 2642 of the base map 2640. Furthermore, the PV modules in the improved EL image 2630 must be approximately aligned with the PV modules of the base map 2640. The image processor 300 then instructs the mapping module 290 to obtain an affine or perspective image transformation from the deflection vectors of the corner points 2632 in the improved EL image 2630 to all corresponding corner points 2642 of the PV modules in the base map 2640. The image processor 300 further instructs the mapping module 290 to map the refined EL image 2630 onto the base map 2640 based on the greatest overlap of the corner points 2632 and 2642 and according to the most similar orientation.

[0190] Once the enhanced EL images 2200, 2610, 2630 are mapped onto the base maps 2410, 2620, 2640, information about the geolocation (such as GPS coordinates) of PV module defects identified in the enhanced EL images 2200, 2610, 2630 can be readily identified from the base maps 2410, 2620, 2640 for repair work and / or maintenance.

[0191] Advantageously, in view of the described embodiments, UAV 20 is capable of taking low-resolution, monochromatic video in dim light conditions, and further enhanced resolution and quality images can be produced for identifying defective PV modules and estimating PV module power loss. In particular, on-board processing subsystem 240 is capable of autonomously navigating UAV 20 and performing exemplary method 400.

[0192] Furthermore, since information such as frame-dependent timed geolocation (e.g., time, latitude, longitude, altitude, etc.) and camera orientation (e.g., yaw, pitch, roll) is processed, it is possible to reliably and accurately recreate the location of the PV strings for a particular EL image.

[0193] It should be noted that the various embodiments described herein should not be construed as limiting. For example, UAV 20 can be further equipped with an ultrasonic device for additional distance measurement. Furthermore, camera 222 can capture still EL images of the PV string under forward bias or alternatively record video of the PV string. Furthermore, color sensors can be used rather than monochromatic sensors. While the described embodiments use "PV strings" as an example, any other PV electrical connections can be used, and generally, the embodiments can be used with any PV array.

[0194] In addition to UAVs, other types of aircraft may be used, such as drones.

[0195] Although the exemplary method 400 is described as including all eight functions, namely, FOCUS, POINT, FIND, ALIGN, SCAN, AUTO, FREEZE, and MAP, it will be understood that the system 200 may perform any number of the functions in any reasonable order. For example, in an alternative embodiment, the on-board processing subsystem 240 may not perform the AUTO function because the operator 30 may desire greater control over which PV strings are inspected. In this case, the operator manually controls the switcher box 32 and power supply 36 after the SCAN function is completed, thus initiating the FIND or SCAN function. The image processing device 260 may also perform the FREEZE function without the MAP function.

[0196] Additionally, a FOCUS function may be performed at all times during method 400, especially while the SCAN function is in progress, to ensure that the captured EL images have high quality sharpness. Alternatively, if the distance between UAV 20 and PV array 10 can be kept within a narrow range, the FOCUS function need not be performed at all. In such an embodiment, a fixed focus lens without controlled focus adjustment may be used in place of focusing lens 223.

[0197] Additionally, in an alternative embodiment, UAV 20 can remotely transfer captured EL images to image processing device 260 without first returning to base.

[0198] Furthermore, a predetermined maximum value (d exp_max ) may be set to a maximum of 1.5.

[0199] In another example, if a lens filter is arranged to filter out any optical interference from the laser, the laser may not need to be turned off.

[0200] In yet another example, during the SCAN function, the default size ratio may be set to maintain approximately 15% to 20% (or even higher, e.g., 20% to 25%) of space between the top and bottom of the PV string 512b and the image boundary 1410 to allow for greater tolerance to position oscillations of the UAV 20, depending on how unstable the UAV 20 appears to be. [Explanation of symbols]

[0201] 10 PV arrays 12 PV strings 14 PV modules 16 Combiner Box 20 Unmanned Aerial Vehicle (UAV) 32 Switcher Box 34 channels 36 Power source, power source 40 PV arrays 100 Setup 200 systems 220 Optical Subsystem 222 Camera 223 Focusing lenses, lenses, camera lenses 224 Light Detection and Ranging Devices (LIDAR) 226 Laser 228 Single-axis gimbal, gimbal 230 Propulsion Device 232 Propeller 234 Motor 240 On-board Processing Subsystem 242 Power Source 250 Controller 252 Memory Unit 260 Image Processing Device 270 Frame Extraction Module 280 Image Enhancement Module 290 Mapping Module 300 Image Processor 512a PV string, PV array subsection 512b PV string, PV array subsection 512c PV string 514b PV module 532 Switcher Box 1812a PV string 1812b PV string 1812f PV string 1814f PV string 1820a PV string 2016 PV Module 2314 PV module 2324 PV module 2410 PV string, base map 2514 PV module 2620 PV array

Claims

1. 1. A method for processing an electroluminescence (EL) image of a PV array, comprising: extracting a plurality of frames of a PV array subsection of the PV array from the EL image, the PV array subsection including one or more PV modules of the PV array; determining a reference frame from the extracted frames that has the highest image quality for the PV array subsection; performing an image alignment of the extracted frame to the reference frame to generate an image-aligned frame, the step comprising: arranging the extracted frames in a stacked arrangement, wherein a corner point of each of the PV modules is stacked; and by aligning the respective corner points of each PV module in the extracted frame with corresponding corner points of the PV module in the reference frame; processing the image-aligned frame to produce an improved image of the PV array subsection having higher resolution than the reference frame.

2. The step of extracting the frame from the EL image comprises: identifying the respective corner points of each PV module in the EL image; and constructing a respective frame for each PV module based on the identified corner points of each PV module.

3. The step of determining each corner point of each PV module in the EL image comprises: clustering the respective corner points of a particular PV module that are repeated in different images; and calculating a respective average position for each cluster of each corner point.

4. 4. The method of claim 1, wherein determining the reference frame having the highest image quality comprises evaluating the image quality of each frame based on at least one of sharpness, signal-to-noise ratio, and completeness of the frame.

5. 5. The method of claim 1, wherein processing the image-aligned frames comprises: grouping the image-aligned frames according to the PV module in each frame; and performing image averaging on each group of image-aligned frames to obtain a respective improved frame for each PV module.

6. The method of claim 5 , wherein the image averaging is based on weighted image stack averaging and / or a deep convolutional neural network architecture.

7. 7. The method of claim 5 or 6, further comprising the steps of associating each improved frame with a horizontal index and a vertical index according to a position of each PV module in the PV array subsection, and ordering the improved frames according to their horizontal and vertical indexes to produce the improved image of the PV array subsection.

8. The method of any one of claims 5 to 7, further comprising the step of scaling the image intensity of each refinement frame.

9. 9. The method of claim 1, further comprising mapping the improved image of the PV array subsection onto a base map of the PV array subsection, the base map including the geolocation of each PV module.

10. 10. The method of claim 9, wherein mapping the improved image onto the base map includes orienting the improved image to align the PV array subsection in the improved image with the PV array subsection in the base map.

11. Each EL image of the PV array subsection includes an image identifier, and the step of orienting the improved image comprises: associating the image identifier of a particular EL image with each enhanced frame in which the PV module appears in the particular EL image; and determining an orientation of the refined image based on the image identifier associated with each refined frame.

12. The method of any one of claims 9 to 11, wherein the geolocation comprises a GPS coordinate.

13. The method of claim 1 , wherein the plurality of frames are consecutive frames of the PV array subsection.

14. 1. An image processing device for processing an EL image of a PV array, comprising: an image processor, the image processor comprising: extracting from the EL image a plurality of frames of a PV array subsection of the PV array, the PV array subsection including one or more PV modules of the PV array; determining a reference frame from the extracted frames that has the highest image quality for the PV array subsection; and performing an image alignment of the extracted frame to the reference frame to generate an image-aligned frame, the generating including: arranging the extracted frames in a stacked arrangement, wherein a corner point of each of the PV modules is stacked; and generating a reference frame by aligning the respective corner points of each PV module in the extracted frame with corresponding corner points of the PV module in the reference frame; an image processing device configured to process the image aligned frame to produce an improved image of the PV array subsection having higher resolution than the reference frame.

15. The image processor identifying respective corner points of each PV module in the EL image; and The image processing device of claim 14 , further configured to extract the frames from the EL image by constructing a respective frame for each PV module based on the identified corner points of each PV module.

16. The image processor clustering the respective corner points of a particular PV module that are repeated in different images; and The image processing device of claim 15 , further configured to determine a respective corner point of each PV module in the EL image by calculating a respective average position for each cluster of respective corner points.

17. 17. The image processing device of claim 14, wherein the image processor is further configured to determine a reference frame having the highest image quality by evaluating the image quality of each frame based on at least one of sharpness, signal-to-noise ratio, and completeness of the frame.

18. 18. The image processing device of claim 14, wherein the image processor is further configured to process the image-aligned frames by grouping the image-aligned frames according to the PV module in each frame, and to perform image averaging on each group of image-aligned frames to obtain a respective improved frame for each PV module.

19. 20. The image processing device of claim 18, wherein the image processor is further configured to perform image averaging based on weighted image stack averaging and / or a deep convolutional neural network structure.

20. 20. The image processing device of claim 18 or 19, wherein the image processor is further configured to associate each improved frame with a horizontal index and a vertical index according to a position of each PV module in the PV array subsection, and to arrange the improved frames according to their horizontal and vertical indexes to produce the improved image of the PV array subsection.

21. 21. An image processing device according to any one of claims 18 to 20, wherein the image processor is further configured to scale the image intensity of each refinement frame.

22. 22. The image processing device of claim 14, wherein the image processor is further configured to map the improved image of the PV array subsection onto a base map of the PV array subsection, the base map including the geolocation of each PV module.

23. 23. The image processing device of claim 22, wherein the image processor is further configured to map the improved image onto the base map by orienting the improved image to align the PV array subsection in the improved image with the PV array subsection in the base map.

24. Each EL image of the PV array subsection includes an image identifier, and the image processor: Associating the image identifier of a particular EL image with each refinement frame in which that PV module appears in the particular EL image; and The image processing device of claim 23 , further configured to orient the improved image by determining an orientation of the improved image based on the image identifier associated with each improved frame.

25. 25. An image processing device according to any one of claims 22 to 24, wherein the geolocation comprises GPS coordinates.

26. 26. The image processing device of claim 14, wherein the plurality of frames are consecutive frames of the PV array subsection.

27. 1. A method of controlling the movement of an aircraft having a camera for capturing EL images of a PV array, comprising: controlling the aircraft to fly along a route and capture EL images of corresponding PV array subsections of the PV array; deriving respective image quality parameters from at least some of the captured EL images, the image quality parameters including an SNR scanning factor and a motion blur scanning factor; dynamically adjusting a flight speed of the aircraft along the flight path based on the respective image quality parameters for capturing the EL images of the PV array subsections, deriving a target flight speed based on the minimum of the SNR scan factor and the motion blur scan factor; and dynamically adjusting a current flight speed of the aircraft to match the target flight speed.

28. 28. The method of claim 27, wherein the SNR scan factor depends on a target SNR, a measured SNR, and an estimated number of captured EL images that include a particular PV module of the PV array subsection.

29. 29. The method of claim 27 or 28, wherein the motion blur scanning factor is the ratio of a measured object deflection to a predetermined maximum object deflection.

30. The step of deriving the target flight speed includes: applying the minimum of the SNR scan factor and the motion blur scan factor to a current flight speed of the aircraft to derive a target scan speed; and selecting the target scan speed as the target flight speed if the target scan speed is less than a maximum flight speed of the aircraft.

31. 31. The method of claim 30, further comprising the step of: if the target scan rate exceeds the maximum flight speed of the aircraft, selecting the maximum flight speed as the target flight speed.

32. 32. The method of any one of claims 27 to 31, further comprising adjusting the target flight speed based on user input factors.

33. 33. The method of any one of claims 27 to 32, further comprising detecting, by one or more PV modules, an EL signal emitted from the PV array prior to controlling the aircraft to fly along the route to capture an EL image of a corresponding PV array subsection of the PV array.

34. 34. The method of claim 33, further comprising the step of maneuvering the aircraft to an initial position, wherein a yaw axis of the aircraft and an optical axis of the camera are perpendicular to the ground prior to detecting the EL signal.

35. 35. The method of claim 33 or 34, further comprising navigating the aircraft to the location of the EL signal.

36. 36. The method of any one of claims 33 to 35, wherein detecting the EL signal emitted by the one or more PV modules of the PV array comprises rotating the aircraft about an airframe yaw axis while simultaneously increasing an optical axis angle of the camera until the EL signal is detected.

37. 37. The method of claim 36, wherein the optical axis angle of the camera is increased from 0° to 70°.

38. 38. The method of claim 36 or 37, wherein the optical axis angle of the camera is increased with decreasing pitch speed.

39. 39. A method according to any one of claims 36 to 38, wherein the aircraft rotates at a decreasing yaw rate.

40. 40. The method of any one of claims 36 to 39, further comprising maneuvering the aircraft to a predetermined altitude before rotating the aircraft.

41. further comprising aligning a field of view (FOV) of the camera with the corresponding PV array subsection, the step comprising: determining a key point for each of the reference PV modules in the corresponding PV array subsection; deriving target alignment points from each of the key points for aligning the camera FOV with the corresponding PV array subsection; performing a perspective transformation to align each of the key points to the target alignment point; and 41. The method of any one of claims 27 to 40, performed by steering the aircraft relative to the corresponding PV array subsection based on the perspective transformation.

42. 42. The method of claim 41 , wherein aligning the camera FOV with the corresponding PV array subsection further comprises maneuvering the aircraft to an appropriate altitude, wherein the corresponding PV array subsection is at a predetermined size ratio within the camera FOV at the appropriate altitude.

43. 43. The method of claim 42, wherein the corresponding PV array subsection occupies 80% to 90% of the camera's FOV at the predetermined size ratio.

44. 44. The method of any one of claims 27 to 43, further comprising dynamically adjusting the focus of the camera according to the measured image sharpness.

45. 45. The method of any one of claims 27 to 44, wherein the aircraft further includes a light source aligned with the optical axis of the camera, the method further including powering the light source except during capturing the EL image of the PV array.

46. a camera for capturing an EL image of the PV array; a propulsion device for actuating the movement of the aircraft; a controller communicatively coupled to the camera and the propulsion device, the controller comprising: controlling the aircraft to fly along a route and capture an EL image of a corresponding PV array subsection of the PV array; deriving respective image quality parameters from at least some of the captured EL images, the image quality parameters including an SNR scanning factor and a motion blur scanning factor; dynamically adjusting a flight speed of the aircraft along the flight path based on the respective image quality parameters for capturing the EL images of the PV array subsections, wherein dynamically adjusting includes: deriving a target flight speed based on the minimum of the SNR scan factor and the motion blur scan factor; and dynamically adjusting a current flight speed of the aircraft to match the target flight speed.

47. 47. The aircraft of claim 46, wherein the SNR scan factor depends on a target SNR, a measured SNR, and an estimated number of captured EL images that include a particular PV module of the PV array subsection.

48. 48. An aircraft as described in claim 46 or 47, wherein the motion blur scan factor is a ratio of a measured object deflection to a predetermined maximum object deflection.

49. The controller is further configured to derive the target flight speed, which includes: applying the minimum of the SNR scan factor and the motion blur scan factor to a current flight speed of the aircraft to derive a target scan speed; and selecting the target scan speed as the target flight speed if the target scan speed is less than a maximum flight speed of the aircraft.

50. 50. The aircraft of claim 49, wherein the controller is further configured to select the maximum flight speed as the target flight speed if the target scan speed exceeds the maximum flight speed of the aircraft.

51. 51. The aircraft of any one of claims 46 to 50, wherein the controller is further configured to adjust the target flight speed based on user input factors.

52. The controller detecting an EL signal emitted from the PV array by one or more PV modules, the PV array having an array axis, the one or more PV modules being aligned with the array axis and including a plane; determining the array axis of the PV array; 52. The aircraft of any one of claims 46 to 51, further configured to: dynamically adjust the propulsion device to align the optical axis of the camera perpendicular to a plane of the one or more PV modules while controlling the camera to capture the EL image of the PV array along the array axis.

53. The controller is further configured to set the aircraft to an initial position before locating the EL signal, which includes: dynamically adjusting the propulsion device to set the yaw axis of the aircraft normal to the ground; and dynamically adjusting the optical axis of the camera so that it is perpendicular to the ground.

54. 54. The aircraft of claim 52 or 53, wherein the controller is further configured to dynamically adjust the propulsion device to navigate the aircraft to a location of the EL signal.

55. 55. The aircraft of any one of claims 52 to 54, wherein the controller is further configured to locate the EL signal emitted by the one or more PV modules of the PV array by rotating the aircraft about a fuselage yaw axis while dynamically adjusting the propulsion device to increase an optical axis angle of the camera until the EL signal is located.

56. 56. The aircraft of claim 55, wherein the optical axis angle of the camera is increased from 0° to 70°.

57. 57. An aircraft as described in claim 55 or 56, wherein the camera optical axis angle increases with decreasing pitch speed.

58. 58. An aircraft as claimed in any one of claims 55 to 57, wherein the aircraft rotates at a decreasing yaw rate.

59. 59. The aircraft of any one of claims 55 to 58, wherein the controller is further configured to dynamically adjust the propulsion devices to steer the aircraft to a predetermined altitude before rotating the aircraft.

60. The controller is further configured to align a field of view (FOV) of the camera with the corresponding PV array subsection, which includes: determining a key point for each of the reference PV modules in the corresponding PV array subsection; deriving target alignment points from each of the key points for aligning the camera FOV with the corresponding PV array subsection; performing a perspective transformation to align each of the key points to the target alignment point; and dynamically adjusting the propulsion devices to steer the aircraft relative to the corresponding PV array subsections based on the perspective transformation.

61. 61. The aircraft of claim 60, wherein the controller is further configured to align the camera FOV with the corresponding PV array subsection by dynamically adjusting the propulsion device to maneuver the aircraft to an appropriate altitude, the corresponding PV array subsection being at a predetermined size ratio within the camera FOV at the appropriate altitude.

62. 62. The aircraft of claim 61, wherein the corresponding PV array subsection occupies 80% to 90% of the camera's FOV at the predetermined size ratio.

63. 63. The aircraft of any one of claims 52 to 62, wherein the controller is further configured to dynamically adjust focus of the camera according to measured image sharpness.

64. 64. The aircraft of any one of claims 52 to 63, further comprising a light source aligned with an optical axis of the camera, the controller further configured to power the light source except during capture of the EL image of the PV array.

65. 1. A method for obtaining an enhanced image of a PV array subsection of a PV array from an EL image of the PV array subsection captured by an aircraft having a camera, the method comprising: controlling the aircraft to fly along a route and capture EL images of corresponding PV array subsections of the PV array; deriving respective image quality parameters from at least some of the captured EL images; dynamically adjusting a flight speed of the aircraft along the flight path based on the respective image quality parameters for capturing the EL images of the PV array subsections; extracting a plurality of frames of the PV array subsection from the EL image, the PV array subsection including one or more PV modules of the PV array; determining a reference frame from the extracted frames that has the highest image quality for the PV array subsection; performing an image alignment of the extracted frame to the reference frame to generate an image-aligned frame, the step comprising: arranging the extracted frames in a stacked arrangement, wherein a corner point of each of the PV modules is stacked; and by aligning the respective corner points of each PV module in the extracted frame with corresponding corner points of the PV module in the reference frame; processing the image-aligned frame to produce an improved image of the PV array subsection having higher resolution than the reference frame.

66. 1. A system for capturing and processing EL images of a PV array subsection of a PV array, comprising: a camera for capturing an EL image of the PV array; a propulsion device for actuating the movement of the aircraft; a controller communicatively coupled to the camera and the propulsion device, the controller comprising: controlling the aircraft to fly along a route and capture an EL image of a corresponding PV array subsection of the PV array; deriving respective image quality parameters from at least some of the captured EL images; dynamically adjusting a flight speed of the aircraft along the flight path based on the respective image quality parameters for capturing the EL images of the PV array subsections; and 1. An image processing device including an image processor, the image processor comprising: extracting from the EL image a plurality of frames of a PV array subsection of the PV array, the PV array subsection including one or more PV modules of the PV array; determining a reference frame from the extracted frames that has the highest image quality for the PV array subsection; and performing an image alignment of the extracted frame to the reference frame to generate an image-aligned frame, the generating including: arranging the extracted frames in a stacked arrangement, wherein a corner point of each of the PV modules is stacked; and generating a reference frame by aligning the respective corner points of each PV module in the extracted frame with corresponding corner points of the PV module in the reference frame; and an image processing device configured to process the image-aligned frame to produce an improved image of the PV array subsection having higher resolution than the reference frame.

Images