Solar panel inspection by unmanned aerial vehicle

Abstract

A method for detecting faults in solar panels in a solar farm is disclosed. In some examples, the method includes acquiring, by a computing system (106), a first image of a solar panel captured by an unmanned aerial vehicle (UAV) (102) during a first flight and a second image of the solar panel captured by the UAV (102) during a second flight. In some examples, the method also includes determining, by the computing system (106), a first score for the solar panel based on the first image, and determining, by the computing system (106), a second score for the solar panel based on the second image. In some examples, the method also includes determining, by the computing system (106), whether the solar panel has a fault based on the first score and the second score, and outputting an indication that the solar panel has a fault in response to determining that the solar panel includes a fault.

Description

Technical Field

[0001] The present disclosure relates to a system for solar panel inspection. Background Art

[0002] A solar farm may include many solar panels configured to generate electricity. To keep the solar panels in operating condition, workers may manually inspect the solar panels. Manual inspection may detect visible defects on the solar panels.

[0003] Workers in a solar farm can use an unmanned aerial vehicle (UAV) with a camera to capture images of solar panels to determine if the panels are defective. The camera can be configured to capture thermal imaging images that show temperature gradients across the solar panels. The temperature gradients can indicate if the solar panels are defective. Summary of the invention

[0004] In general, the present disclosure relates to a system for detecting faults in solar panels. The detection process may include one or more unmanned aerial vehicles (UAVs) having cameras configured to capture images of solar panels at different times. A cloud computing system may acquire images from the UAVs and determine a score based on each image. The score may indicate the condition of the solar panel at the time each image was captured. The cloud server computing system may be configured to determine whether the solar panel includes a fault based on the score of the image.

[0005] In one example, the present disclosure describes a method for detecting a fault in a solar panel in a solar farm. The method includes acquiring, by a computing system, a first image of the solar panel captured by a UAV during a first flight and a second image of the solar panel captured by the UAV during a second flight. The method also includes determining, by the computing system, a first score for the solar panel based on the first image, and determining a second score for the solar panel based on the second image. The method also includes determining, by the cloud computing system, that the solar panel includes a fault based on the first score and the second score. The method includes outputting, by the cloud computing system, an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0006] In another example, the present disclosure describes a cloud computing system including a memory configured to store a first image of the solar panel captured by a UAV during a first flight, and to store a second image of the solar panel captured by the UAV during a second flight. The cloud computing system also includes one or more processing circuits configured to determine a first score of the solar panel based on the first image, and to determine a second score of the solar panel based on the first image. The one or more processing circuits are further configured to determine whether the solar panel includes a fault based on the first score and the second score. The one or more processing circuits are further configured to output an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0007] In another example, the present disclosure describes a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, direct a cloud computing system to acquire a first image of the solar panel captured by a UAV during a first flight and a second image of the solar panel captured by the UAV during a second flight. The instructions, when executed, further direct the cloud computing system to determine a first score for the solar panel based on the first image and to determine a second score for the solar panel based on the first image. The instructions, when executed, further direct the cloud computing system to determine whether the solar panel includes a fault based on the first score and the second score. The instructions, when executed, direct the cloud computing system to output an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0008] The details of one or more examples of the present disclosure are set forth in the following drawings and the description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 An example unmanned aerial vehicle (UAV) system is shown that may be configured to implement the techniques of this disclosure.

[0010] Figure 2 is a block diagram illustrating example components of a UAV that may be used to implement the techniques of this disclosure.

[0011] Figure 3 is a block diagram illustrating exemplary components of a cloud computing system in accordance with techniques of this disclosure.

[0012] Figure 4 is a block diagram illustrating the autonomy of a solar panel inspection technique in accordance with the technology of this disclosure.

[0013] Figure 5A Solar panel inspection patterns in two directions according to the disclosed technique are shown.

[0014] Figure 5B An infrared camera image is shown for solar panel difference recognition and identification in accordance with the techniques of this disclosure.

[0015] Figure 6 is a diagram showing image capture angles according to the technology of this disclosure.

[0016] Figure 7 is a flow chart illustrating example operations for detecting faults in a solar panel in accordance with the techniques of this disclosure.

[0017] Figure 8 is a conceptual diagram illustrating a technique for automatically mapping a solar farm in accordance with the techniques of this disclosure.

[0018] Fig. 9 A decision diagram for ground mobility according to the techniques of this disclosure is shown.

[0019] Fig.10 is a flow chart illustrating a visible light image control algorithm for fault classification according to the techniques of this disclosure.

[0020] Fig.11 is a flow chart illustrating an infrared image control algorithm for fault classification according to the techniques of this disclosure.

[0021] Fig.12 is a flow chart illustrating mapping and combining visible light and infrared images for fault detection in accordance with the techniques of this disclosure.

[0022] Fig.13 is a flow chart illustrating task management for repair, maintenance, and tracking according to the techniques of this disclosure.

[0023] Fig.14 is a flow chart illustrating a verification control algorithm for two-dimensional / three-dimensional mapping techniques according to the techniques of this disclosure.

[0024] Fig.15 is a flow chart illustrating a procedure for electronic repair in accordance with the techniques of this disclosure.

[0025] Fig.16 is a flow chart illustrating a software algorithm for forming a model of an entire solar farm in accordance with the techniques of this disclosure.

[0026] Fig.17 A fault detection and identification process according to the techniques of this disclosure is shown.

[0027] Fig.18 A graphical user interface including a graphical icon indicating a solar panel having a fault is shown according to some examples of the present disclosure. DETAILED DESCRIPTION

[0028] The present disclosure describes systems, devices, methods, and techniques for detecting faults in solar panels based on images of the solar panels captured by an unmanned aerial vehicle (UAV). The technology may allow for detection of faults in all of the components of a solar farm, including solar panels, electrical wires, inverters, breaker boxes, transformers, towers, mechanical elements supporting the solar panels, vegetation surrounding the solar panels, and ground conditions (e.g., erosion). The term "solar panel assembly" may refer to any or all of these elements, including any other elements of a solar farm not listed herein. The detection techniques described herein may be performed manually (i.e., using an active operator), semi-autonomously, or fully autonomously. Semi-autonomous detection techniques may include less human involvement than manual detection.

[0029] Individual solar panel locations within a solar farm may be identified from captured images using applied global positioning system (GPS) data and UAV attitude information. Even if the solar farm does not provide its own location and array number, the computing system of the present disclosure may determine the (x, y, z) GPS location of an individual solar panel. The computing system of the present disclosure may be referred to as a "cloud computing system" or "cloud server computing system" to distinguish the computing system from the flight control (FC) system and the computing system onboard the UAV. However, in some examples, the cloud computing system may be merged with the processor onboard the FC system and / or the UAV.

[0030] A processor onboard the UAV may capture, store, and send image data to a cloud computing system. The cloud computing system may acquire the captured images and store them to a memory device. The cloud computing system may store image data with applied GPS and UAV attitude information collected by the processor onboard the UAV. The captured images, together with the GPS and attitude information, may constitute a big data database for a cloud application to perform analysis. The method may include using the GPS location and UAV attitude of each image to generate a map of the entire solar panel field, the size of which is reduced to as small as a single solar panel unit or another element of a solar panel assembly. The cloud application may integrate each image into a model of the solar farm, so that a user can access all of the data related to the solar panels, including the images, through the cloud application. Therefore, the cloud computing system can track individual solar panels very accurately.

[0031] Once the solar panel fault status, location, and score are determined by the cloud computing system, the cloud server and smartphone application technology can be configured to manage the entire solar farm or a portion thereof. The cloud application can associate a solar field worker with one or more elements of a solar panel assembly so that a worker can be assigned to the identified solar panel location to work on the solar panel assembly to maintain, repair, or replace the assembly or its elements. The cloud server computing system can also help the worker record the maintenance task and any details when the work is completed. If a solar panel is replaced, the worker can update the original solar panel manufacturing tracking number in the cloud application.

[0032] The cloud application can be configured to keep records on every single solar panel from the entire solar farm so that the cloud application will have full tracking capabilities for each solar panel. Using the tracking capabilities, the cloud application can track the status of each solar panel. The cloud application can update the status based on each new image and worker report. The cloud application can determine a score based on the new image and then determine whether the solar panel has a fault based on the new score and one or more previous scores (see, e.g. Fig. 9 ).

[0033] The method publishes the entire solar panel fault discovery information to the cloud by a cloud computing system. The cloud puts all the data into a cloud application so that solar field workers and supervisors can use the cloud application to find the location and fault status of the faulty solar panel. The cloud application can then be used to perform maintenance and repair tasks to become a seamless connection from the images captured by the UAV to the solar panel field map to the individual solar panels. The cloud application can also be configured to detect solar faults based on analyzing the captured images, determine the fault location on the solar panel, track faults on the solar panel and other components of the solar panel assembly, assist workers in maintenance / repair / replacement tasks and then allow workers to report maintenance issues.

[0034] The cloud application can provide task assignments to workers based on input data such as images. Once the cloud computing system receives the input data, the cloud computing system can be configured to operate autonomously without human intervention. For example, the cloud application can evaluate the image for faults, fault categories (major, minor, etc.), and then assign repair and maintenance tasks to workers. The cloud application can also track faults or the likelihood of faults over time, as one image may not be sufficient to conclusively show or show above a threshold confidence level that the solar panel has a fault. The cloud computing system can have a higher fault detection accuracy rate than a human manually browsing through images to find faults. The cloud computing system can also process hundreds or thousands of times more data and images than a human can view and analyze for faults. In addition, the cloud applications for visible images, infrared images, and electroluminescent test images shown in Figure 7 , Fig.10 , Fig.11 , Fig.12 , Fig.13 and Fig.14 The detection algorithms in the paper could improve fault detection and tracking processes at solar farms and other facilities.

[0035] Figure 1 An example of an unmanned aerial vehicle (UAV) system 100 is shown that can be configured to implement the techniques of this disclosure. The UAV system 100 includes a UAV 102, a controller device 104, and a cloud computing system 106. In the UAV system 100, the controller device 104 controls the flight path and data collection functions of the UAV 102. The cloud computing system 106 processes the data collected by the UAV 102. Figure 1 102 are shown as separate devices, but in some UAV systems, the functions of the controller device 104 and the cloud computing device 106 may be performed by a common device. UAV 102 may be part of a fleet of UAVs, where more than one UAV may capture images as described herein.

[0036] Although UAV 102 Figure 1102 is shown as a quadcopter, but UAV 102 can be any type of UAV, including but not limited to rotorcraft, fixed-wing aircraft, compound aircraft (such as tilt-rotor aircraft, X2 and X3), aerostat or any other type of UAV, including all vertical take-off and landing (VTOL) aircraft, stand-up landing aircraft, etc. UAV 102 can be configured to fly with different degrees of autonomy. In some examples, UAV 102 may be under constant or near-constant control of the user of controller device 104. In other examples, controller device 104 may deliver tasks (including flight plans) to UAV 102, and the onboard processor 116 on UAV 102 may be configured to perform tasks with little or no additional user input. The onboard processor 116 may include FC functions. In some examples, UAV 102 may use LIDAR to avoid collisions. The onboard processor 116 may be configured to autonomously navigate the flight path of UAV 102, for example, along a predefined route through a solar farm. The onboard processor 116 may also be configured to receive commands from the controller device 104 to control the movement of the UAV 102 .

[0037] Although the technology of this disclosure is not limited to any particular type of UAV, the UAV 102 may be, for example, a relatively small, low-altitude, low-speed UAV, where small in this case corresponds to less than 100 pounds, low altitude corresponds to an operating altitude of less than 3,000 feet above the ground, and low airspeed corresponds to an airspeed of less than 250 knots. Furthermore, it is contemplated that the UAV 102 may have a hovering capability, meaning that the UAV 102 may have the ability to maintain a generally constant position in the air.

[0038] The UAV 102 may have the following functions: automatic takeoff and landing, waypoint flight planning, return-to-home mode, waypoint activation, and automatic camera triggering functions. The UAV 102 may include a bent and / or curved leg structure to facilitate landing on an inclined or uneven surface. The return-to-home mode may be automatically or manually activated by one of the following events: user input, low battery power, overheating, weather changes, system failure, loss of line of sight between the user and the UAV, obstacle collision avoidance, and / or unacceptable uncertainty or abnormal behavior.

[0039] In some examples, the controller device 104 comprises a general purpose device, such as a personal digital assistant (PDA), a laptop or desktop computer, a tablet computer, a cellular or satellite radiotelephone, a "smart phone", or other such device. In examples where the controller device 104 is a general purpose device, the controller device 104 may be loaded with and configured to execute software designed to control the UAV 102. In other examples, the controller device 104 is a dedicated device specifically designed to control the UAV 102. The controller device 104 includes a FC control and navigation system 140 for controlling the movement of the UAV 102. The controller device 104 may include a handheld device and / or be part of a ground control base station.

[0040] The controller device 104 communicates with the UAV 102 via a communication link 108. The communication link 108 may be, for example, a direct link through a radio communication protocol such as WiFi, radio frequency wireless, Bluetooth, ZigBee, a proprietary protocol, or any other suitable protocol. In other examples, the communication link 108 may be a network-based link, in which the controller device 104 communicates with the UAV 102 through one or more intermediate devices such as a gateway, a router, a switch, a repeater, or other such network devices. The communication link 108 may include a wired connection or a mobile storage device. The wired connection may include a universal serial bus (USB) or RS-232. The mobile storage device may include a thumb drive, a secure digital (SD) card, or an external storage device (e.g., a solid-state memory or a hard drive).

[0041] The cloud computing system 106 includes one or more computing devices. The cloud computing system 106 may include one or more server computing devices for receiving and processing data and one or more mobile devices for presenting information to users, etc. For example, the cloud computing system 106 may include a general-purpose device such as a personal digital assistant (PDA), a laptop or desktop computer, a tablet computer, a smart phone, a server device, or another such device. The cloud computing system 106 may be loaded with and configured to execute software designed to process data collected by the UAV 102. In some examples, the UAV 102 is configured to stream data to the cloud computing system 106 in real time or near real time via, for example, a wireless communication link 110. In other examples, the UAV 102FC processor stores data while in flight and transmits the data to the cloud computing system 106 later, such as after the flight is completed.

[0042] One or more cameras 112 are mounted on UAV 102. According to the technology of the present disclosure, camera 112 may include one or more cameras capable of capturing infrared radiation images. The one or more cameras 112 may include a thermal imaging camera. In addition, in some examples of the present disclosure, camera 112 may include one or more cameras capable of capturing visible light images.

[0043] like Figure 1 As shown in the example of , camera 112 may be configured to capture an image of structure 114. Figure 1 114 is shown as a solar panel, but structure 114 may include a building, dam, wind turbine, monument, bridge, dike, seawall, pier, antenna, volcano, pumping station, gas or oil pipeline, power plant, manufacturing plant, power facility or other type of man-made or natural structure. Structure 114 may include a single solar panel or multiple solar panels mounted on the ground or mounted on a building, structure or mountain. In some examples, camera 112 may be configured to capture images of more than one solar panel at a time. Structure 114 may also include mechanical elements that support solar panels, wires, circuit breaker boxes, towers, and transformers. The inspection process may also include analyzing the state of the ground and vegetation around structure 114, as vegetation may block sunlight and ground erosion may affect the performance of solar panels.

[0044] According to the technology of the present disclosure, UAV 102 flies to a predetermined image capture location within a solar farm. The onboard processor 116 and / or the controller device 104 can control the movement of the UAV 102 to fly the UAV 102 to the predetermined location. At each position in the image capture location, the camera 112 of the UAV 102 captures a first set of one or more infrared images of the structure 114 and / or one or more visible light images of the structure 114. The onboard processor 116 can define and save each position in the image capture location based on x, y, and z coordinates and UAV attitude information, so that the cloud computing system 106 can use a predefined formula to determine the position of each individual solar panel. In some examples, the onboard processor 116 saves the captured images on a secure digital (SD) card or other type of memory card, and can also be transmitted online to a cloud-based web server using 3G, 4G, 5G, narrowband Internet of Things (NBIOT) or other wireless types of transmission technology.

[0045] In some examples, the UAV 102 is equipped with one or more differential global navigation satellite system (GNSS) devices to assist the UAV 102 in navigating to the image capture location. For example, the UAV 102 and / or the onboard processor 116 may be equipped for real-time kinematic positioning (RTK), which is a differential GNSS that can provide high positioning performance for the UAV 102 near a base station. In some examples, the accuracy of the GNSS device can be within 1 centimeter. The controller device 104 can be configured to determine the RTK and transmit it to the UAV 102, or the onboard processor 116 can determine the RTK to assist the flight of the UAV 102.

[0046] The first set of images may include infrared images and / or visible light images, and may show a condition of structure 114, such as temperature, color, appearance, visible defects, debris, etc. Onboard processor 116 stores the first set of captured infrared images for later analysis. Onboard processor 116 may transmit the images to cloud computing system 106, and onboard processor 116 and / or cloud computing system 106 may be configured to determine a score based on the first set of images captured at the first time.

[0047] During a subsequent flight, UAV 102 returns to the same predetermined image capture location and captures a second set of infrared images and / or visible light images of structure 114. In some examples, different UAVs in the fleet may fly to the same predetermined image capture location and capture a second set of infrared images and / or visible light images of structure 114. Cloud computing system 106 may store the second set of infrared images for later analysis. Cloud computing system 106 may also store the second set of captured images for later analysis. Cloud computing system 106 may be configured to determine a score based on the second set of images captured at a second time. In some examples, the second time may be at least the next day after the first time. Each of the first set of images and the second set of images may have a single image.

[0048] In various examples, the UAV 102 captures a second set of images during a second flight after various time periods have elapsed or after various events have occurred. For example, the UAV 102 may capture multiple sets of images daily, weekly, monthly, yearly, or after some other time period has elapsed. In some examples, the UAV 102 captures a set of images after an event such as a drop in power generated by the structure 114 or after a major weather event.

[0049] The predefined image capture locations may be aligned with multiple rows of solar panels in a solar farm. For example, a solar farm may have solar panels oriented south at an angle greater than sixty degrees from horizontal. For example, the predefined image capture locations may be located a few meters south of each row of solar panels to allow images to be captured at a desired angle (see Figure 6 ).

[0050] For each corresponding image in the second set of images, the cloud computing system 106 may be configured to attempt to identify the corresponding image in the first set of images. The corresponding image may be captured from the same image capture position as the image in the second set of infrared images. The cloud computing system 106 may identify the corresponding image in various ways. For example, the cloud computing system 106 may receive image capture position coordinates (e.g., coordinates in x, y, and z dimensions) for each image with UAV posture data. When the camera 112 captures an image, the image capture position coordinates of the image indicate the coordinates of the UAV 102. In this example, the cloud computing system 106 may determine that one image corresponds to another image based on the images associated with the same image capture position coordinates.

[0051] The cloud computing system 106 may be configured to determine a score for the structure 114 in the image based on the condition of the structure 114, as shown in the image. A higher score may indicate a greater likelihood of a fault in the structure 114. For example, the cloud computing system 106 may be configured to determine the score based on the infrared image by determining the temperature and / or color of the structure 114 and the color of the infrared image. The cloud computing system 106 may also determine a temperature and / or color difference of the structure 114, wherein a higher temperature and / or a greater color difference may indicate a greater likelihood of a fault in the structure 114. The temperature difference of the structure 114 may refer to the difference between the highest temperature on the surface of the structure 114 and the lowest temperature on the surface of the structure 114. For example, in the infrared image, the cloud computing system 106 may use the temperature difference on the image and the color of the image location to determine a fault state in the structure 114.

[0052] Cloud computing system 106 may be configured to determine a score based on visible defects on structure 114 and based on the visible light image. Examples of defects in the visible light image include cracks, spalling, warping or bending of structural elements, debris accumulation (e.g., dust, metal filings, iron filings, bird droppings), etc. In some examples, cloud computing system 106 may map the visible light image (which is associated with a specific image capture location in (x, y, z) space) to a 2-D and / or 3-D model of the structure. Thus, in some examples, cloud computing system 106 may associate various features of the structure, such as corners, with the visible light image. This may allow cloud computing system 106 and / or a user to identify and store data indicating locations in the structure that require maintenance or repair.

[0053] In addition to solar panels, cloud computing system 106 can monitor the state of the ground around solar panels for ground loss (e.g., erosion) based on visible images. Ground erosion around solar panels can be particularly important in hilly and mountainous terrain. In some examples, cloud computing system 106 can also use visible images to determine whether grass and other vegetation around solar panels are too high, because vegetation can shade solar panels, thereby reducing the power generation of solar farms. In some examples, cloud computing system 106 can use visible images to check the state of wired connections of solar farms. For example, if the wires are broken due to the bite of small animals such as mice or due to bird nests, cloud computing system 106 can use visible images of solar panels and surrounding areas to determine the condition of wires and other elements of solar panels so that cloud applications can assign maintenance workers to repair one or more fault conditions.

[0054] The cloud computing system 106 may also be configured to determine a score based on a change between the first image and the second image of the structure 114. In some examples, the cloud computing system 106 retrieves (e.g., automatically in response to user input, etc.) a visible light image corresponding to the infrared image in response to the cloud computing system 106 determining that the position of the structure has changed during the time interval based on the comparison of the infrared image. The user may use the retrieved visible light image to further perform an inspection of the structure and identify inspection targets. For example, where the structure includes a set of solar panels, the user may use the visible light image to identify cracks or jams in the solar panels. The use of the visible light image may help the user verify the analysis results of the infrared image and may help identify the location of damaged surfaces for maintenance and repair purposes. In some examples, the user may use the visible light image to determine maintenance or repair requirements in materials, processes, procedures, schedule estimates, and work order issuance.

[0055] The cloud computing system 106 and / or the onboard processor 116 may determine an overall score for the structure 114 based on the image. The onboard processor 116 may process the image after it is captured by the camera 112. In some examples, the onboard processor 116 may store the image to a memory device onboard the UAV 102 and then transmit the image to the cloud computing system 106 for processing and fault determination.

[0056] The total score may be used to assess whether structure 114 includes a fault. In some examples, to determine the total score, cloud computing system 106 and / or onboard processor 116 may determine the total score of the structure as the percentage of images in a group of images (e.g., the second group of infrared images described above) having scores greater than a particular threshold. For example, in one example, cloud computing system 106 may sort the images in a group of images according to their corresponding scores (e.g., in descending or ascending order). The cumulative alarm score curve is a curve in a graph that maps the sorted images to their scores. In this example, after sorting the images, cloud computing system 106 may determine the percentage of images having scores that fall into the most severe category. In this example, the percentage may be the total score of the structure. Images with the most severe category scores are referred to as substandard images. As discussed in detail below Fig. 9 Another example of how to determine whether a trigger point has been reached to perform maintenance on a structure, to repair a structure, or whether the structure must be evacuated or abandoned is shown.

[0057] In some examples, the onboard processor 116 adjusts the image to correct the yaw, attitude, and tilt differences between the image and the historical image. In some examples, the UAV 102 includes various sensors to detect the flight orientation of the UAV 102. Such sensors may include a compass and / or gyroscope for detecting yaw, a gyroscope for detecting attitude and tilt, and the like. The onboard processor 116 may send a set of historical images such as the orientation data of each image in the first set of infrared images discussed above. For example, when the camera mounted on the UAV 102 captures the image, the orientation data of the first image in the set of historical images may indicate that the UAV 102 is tilted by 2°. In this example, when the camera mounted on the UAV 102 captures the second image, the orientation data of the second image (e.g., the image in the second set of infrared images) may indicate that the UAV 102 is tilted by 5°. In addition, in this example, the onboard processor 116 may rotate the second image by -3° to align the first image and the second image. In this way, the first image and the second image may be aligned. In other examples, onboard processor 116 may apply a tilt effect to the photo to compensate for yaw and attitude differences. Cloud computing system 106 may perform a similar process on both the infrared light image and the visible light image received from onboard processor 116 .

[0058] The cloud computing system 106 may also perform various other types of pre-processing on the image. For example, the cloud computing system 106 may apply various filters to the image (e.g., to increase contrast, reduce noise, etc.). In some examples, the cloud computing system 106 may zoom in or out on the image to obtain a consistent view. In some examples, the cloud computing system 106 may zoom in on other special features in the image. For example, various environmental conditions such as clear weather, cloudy weather, rain, fog, etc. may affect the quality of infrared light images and visible light images captured by the UAV 102. In addition, wind may cause the UAV 102 to vibrate, which may cause the image to be blurred. The cloud computing system 106 may apply various effects to the image to compensate for environmental conditions. For example, the cloud computing system 106 may apply filters to remove blur and may zoom in or out. For infrared images, the cloud computing system 106 may add a contrast factor to emphasize and clarify the laser line.

[0059] In some examples, the onboard processor 116 overlays the infrared image with a concurrent visible light image in which GPS and UAV attitude information are embedded. The embedded information may allow the onboard processor 116 and / or the cloud computing system 106 to associate each image with an element of a solar panel assembly. For example, the onboard processor 116 and / or the cloud computing system 106 may use a control surface to apply a color layer mask to a visible light image to overlay a corresponding infrared image on the visible light image. The control surface may include a recognizable marker in the image that may be used to match corresponding positions in the visible light image and the infrared image. For example, the UAV 102 may capture an infrared image and a visible light image simultaneously. In this example, the onboard processor 116 and / or the cloud computing system 106 may overlay an infrared image on a visible light image so that the resulting image shows a laser beam emitted by a laser attached to a structure and a normal visible light image of the structure.

[0060] The cloud computing system 106 can be configured to output an alert indicating that the structure 114 has a fault based on the score for each image. The cloud computing system 106 can, for example, average the scores for the images of the structure 114 and compare the average score to a threshold score. The cloud computing system 106 can also look for structures with a single score that exceeds the threshold score and then analyze the other scores for the structure to determine whether the single score is an aberration. Scores that are aberrations may be inaccurate and may be due to weather conditions on the day of the corresponding image. The cloud computing system 106 can use, for example, Fig. 9 A technique is shown in which the determination of whether structure 114 has a fault is based on the percentage of visible light images that do not meet specifications and the percentage of infrared images that do not meet specifications.

[0061] In some examples, cloud computing system 106 can determine that structure 114 has a fault by determining whether to recommend maintenance, repair, or replacement of structure 114. A higher score can result in replacement (e.g., for a crack in structure 114), while a lower score can result in maintenance or repair. The lowest score range may not result in a recommendation.

[0062] Figure 2 An exemplary illustration of UAV 102 is shown. UAV 102 includes flight device 200, processor 202, memory 204, transceiver 206, antenna 208, navigation system 210, camera 212, sensor 214, and power supply 216. Communication channel 218 interconnects each of flight device 200, processor 202, memory 204, transceiver 206, antenna 208, navigation system 210, camera 212, sensor 214, and power supply 216 for inter-component communication (physically, communicatively, and / or operatively). In some examples, communication channel 218 includes a system bus, a network connection, an inter-process communication data structure, or any other method for transmitting data, including various types of wireless communication technologies. Power supply 216 can provide electrical energy to each of the other components of UAV 102. In some examples, power supply 216 is a battery.

[0063] The processor 202 is intended to represent all processing circuitry and all processing capabilities of the UAV 102, including the capabilities of the onboard processor 116. For example, the processor 202 may include one or more digital signal processors (DSPs), general purpose microprocessors, integrated circuits (ICs) or a group of ICs (e.g., a chipset), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. The processor 202 may also include FC functions, such as GPS positioning and attitude determination.

[0064] The memory 204 is intended to represent all of the various memory devices within the UAV 102. The memory 204 constitutes a computer-readable storage medium and may take the form of a volatile memory that does not retain stored content once the UAV 102 is turned off or a non-volatile memory that stores content for a longer period of time (including a period of time when the UAV 102 is in an unpowered state). Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), integrated random access memory (IRAM), thyristor random access memory (TRAM), zero capacitance random access memory (ZRAM), or any other type of suitable volatile memory. Examples of non-volatile memory include optical drives, magnetic disk drives, flash memory, read-only memory (ROM), electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM), or any other such type of non-volatile memory. The memory 204 may be configured to store the boundaries of a solar farm, a predefined route through the solar farm, and predefined flight locations within the solar farm for capturing images.

[0065] The functions of UAV 102 are implemented by hardware, software, firmware, or a combination thereof. Memory 204 may store software and firmware including multiple sets of instructions. Processor 202 and other hardware components of UAV 102 may execute instructions to perform the techniques of the present disclosure.

[0066] The transceiver 206 is configured to send and receive data using the antenna 208. The transceiver 206 may send and receive data according to any wireless communication protocol described elsewhere in this disclosure. For example, the transceiver 206 may be configured to receive navigation instructions. In addition, the transceiver 206 may be configured to send images and other data to a computing system, such as the controller device 104 ( Figure 1 ) or cloud computing systems ( Figure 1 The transceiver 206 may communicate with the controller device 104 (eg, the FC system 104 ) with respect to navigation, routing, location, and the like.

[0067] The navigation system 210 controls the flight path of the UAV 102. For example, the navigation system 210 may output a signal to the flight device 200 to instruct the UAV 102 to fly to a predetermined image capture location, land, or otherwise navigate to a location along the flight path of the UAV 102.

[0068] Camera 212 is configured to capture infrared images. Additionally, in some examples, camera 212 is configured to capture visible light images. In some examples, the infrared images and visible light images are captured with the same camera. In other examples, UAV 102 has separate cameras to capture infrared images and visible light images. Processor 202 may be configured to control camera 212.

[0069] Sensors 214 are intended to represent all of the various sensors included in UAV 102. UAV 102 may, for example, include one or more sensors used for flight management, such as accelerometers, gyroscopes, magnetometers, barometers, GNSS sensors, tilt sensors, inertial measurement sensors, velocity sensors, and the like.

[0070] Figure 3 is a block diagram illustrating example components of a cloud computing system 106 according to one or more techniques of this disclosure. Figure 3 In the example of FIG. 1 , cloud computing system 106 includes one or more processing circuits 300, power supply 302, user interface 303, memory 304, transceiver 306, and display 308. Communication channel 110 interconnects processing circuit 300, memory 304, transceiver 306, and display 308. Power supply 302 provides power to processing circuit 300, memory 304, transceiver 306, and display 308. Processing circuit 300, memory 304, transceiver 306, and display 308 can be similar to the above with respect to Figure 2 The processing circuit 202, the memory 204 and the transceiver 206 are implemented in the above manner. The display 308 may include various types of displays for outputting data, such as a liquid crystal display, a plasma display, a light emitting diode (LED) display, etc.

[0071] The user interface 303 may be configured to receive user input 305. The user interface 303 may include a keyboard, a mouse, a touch pad, a touch screen, etc. The user input 305 may include a user allocating a solar panel or a region of a solar farm to another user. The user input 305 may also include selecting an icon on a graphical user interface presented by the processing circuit 300 on the display 308.

[0072] exist Figure 3 In the example of FIG. 3 , the memory 304 stores the inspection unit 310 and the image archive 312. Figure 3 As shown in the example of , the inspection unit 310 includes an image modification unit 314 and an image analysis unit 316. The inspection unit 310, the image modification unit 314, and the image analysis unit 316 may include instructions that, when executed by the processing circuit 300, cause the cloud computing system 106 to perform actions attributed to the inspection unit 310, the image modification unit 314, and the image analysis unit 316 in the present disclosure.

[0073] exist Figure 3 In the example of , the inspection unit 310 can configure the transceiver 306 to receive a signal from the UAV 102 (e.g., Figure 1 and Figure 2 ) receive data. Thus, the inspection unit 310 may receive various types of data from the UAV 102. For example, the inspection unit 310 may receive image data, orientation data, image capture location coordinate data, and other types of data from the UAV 102. Thus, the transceiver 306 may be configured to receive an image captured by a camera mounted on the UAV, wherein the image has a structure.

[0074] In some examples, the image modification unit 314 performs image pre-processing functions on the image. For example, the image modification unit 314 may rotate or tilt an image received from the UAV 102 so that the image appears to be taken from the same angle as a historical image captured at the same image capture location. For example, if the historical images were all taken at a 0° tilt relative to a plane orthogonal to the gravity vector, but when the UAV 102 captures a new image, a gust of wind occurs that causes the new image to be taken at a 5° tilt relative to the plane, the image modification unit 314 may rotate the new image by -5° to ensure that the new image is from an angle consistent with the historical image. Similarly, historical images of a structure taken at a particular image capture location may have been taken straight at the structure bearing, but when a new image of the structure is taken at the same image capture location, the camera of the UAV 102 may yaw or pitch 4°. Therefore, in this example, the image modification unit 314 may apply a -4° tilt to the new image to correct for the yaw or pitch. The image modification unit 314 may determine the tilt, yaw, or pitch based on orientation data generated by the UAV 102 when the image is captured.

[0075] Image analysis unit 316 may analyze an image of a structure to determine whether the structure has changed position. For example, image analysis unit 316 may acquire a first infrared image and a second infrared image taken at different times at the same image capture location. In this example, image analysis unit 316 may determine a score for the second infrared image based on an angle between a beam from the laser shown in the first image and a beam from the laser shown in the second image. Additionally, image analysis unit 316 may determine, based on the score for the second infrared image, whether the position of the structure has changed during a time interval between capturing the first infrared image and capturing the second infrared image, wherein the laser remained attached to the structure during the time interval. Additionally, image analysis unit 316 may output an indication of whether the position of the structure has changed during the time interval.

[0076] Figure 4is a block diagram illustrating the autonomy of a solar panel inspection technique according to the technology of the present disclosure. The solar panel inspection technique may include a UAV (400) that captures infrared images and / or visible light images. The technique may include functionality to press one button to complete an entire inspection mission. The single button functionality may be based on UAV equipment and requirements as well as flight stability (402) and flight control and flight planning (404). The UAV may include a memory device configured to store a predefined route through a solar farm for the UAV to fly along during an inspection mission.

[0077] The memory device may also be configured to store predefined flight positions along the predefined route for capturing images. The predefined route and predefined flight positions may be based on two-dimensional and / or three-dimensional mapping and surveying techniques (406) of the solar farm. The UAV may automatically capture images at the predefined flight positions and automatically transmit the captured images to a cloud computing device. Using the mapping technology, the UAV may be configured to determine the position of the UAV when each image is captured. The onboard processor 116, the FC computing system, and / or another computing system may be further configured to determine the position of each solar panel on the inspection route.

[0078] The cloud computing system may be configured to automatically detect faults in the solar panels based on the visible light images and / or infrared images (408). The cloud computing system may also be configured to execute electronic maintenance and electronic repair procedures (410). The electronic maintenance and electronic repair procedures may include automatically assigning maintenance and repair tasks to workers. The electronic maintenance and electronic repair procedures may also include tracking the condition and maintenance status of the solar panels. The cloud computing system may be further configured to automatically analyze maintenance issues and power generation of the solar farm (412). The cloud computing system may execute electronic reports, through which the cloud computing system generates quarterly, semi-annual, and annual reports for company leadership or government officials. The cloud computing system may generate reports based on the scores of the solar panels and the measured power generated by the solar panels.

[0079] The cloud computing system 106 may be configured to automatically generate reports without user input. The cloud computing system 106 may first create a report template and place it in a desired network directory. The template file may include the following information: the time and date of the inspection, the person performing the inspection or the UAV that captured the images, the temperature and other environmental information during the inspection, the total inspected area, the power generation rate for the inspection day, and any other noteworthy events for the inspection day. The cloud computing system 106 may copy this information to the template file to populate the report document. The cloud computing system 106 may compare the current report with the previous report and highlight any differences between the two reports. The cloud computing system 106 may combine reports and data to form long-term reports, such as monthly and annual reports. The cloud computing system 106 may create a separate report for each inspection flight.

[0080] The cloud computing system may also be configured to store images and other data on a cloud network so that users can access data from multiple devices (414). The cloud computing system may also run an application that displays relevant data to the user. The application may display the location and condition of the solar panels to the user to aid in inspection, repair, and maintenance. The application may display faulty areas in red or pink. Yellow may indicate moderate temperatures, and green may indicate lower or normal operating temperatures. The application may also allow the user to zoom in on the image of the solar panel so that the user can verify and inspect faults. The application may allow the user to choose whether to view the image in color or in black and white.

[0081] Figure 5A Solar panel inspection patterns in two directions according to techniques of the present disclosure are shown. Longitudinal inspection flight pattern 502 may originate from starting location 500 and travel in an east-west direction through solar farm 520. Transverse inspection flight pattern 512 may originate from starting location 510 and travel in a north-south direction through solar farm 520. Along both inspection flight patterns 502 and 512, the UAV may pass by all of the solar panels in solar farm 520, including solar panel 514. The memory device may be configured to store predefined flight locations 540-548 along inspection flight patterns 502 and 512 for the UAV to capture images.

[0082] The predefined flight positions 540-548 may be aligned with a row of solar panels (including solar panel 514) in solar farm 520. As the UAV flies along longitudinal flight pattern 502, the UAV may pass through predefined flight positions 540-548 as the UAV flies over the row of solar panels. When the UAV reaches each of the predefined flight positions 540-548, the UAV may automatically capture an image of the solar panels in the row. In some examples, a user may override the movement and / or image capture of the UAV.

[0083] The onboard processor 116 and / or the FC system 140 may store the predefined flight positions 540 to 548 as three dimensions: latitude, longitude, and altitude. The onboard processor 116 and / or the FC system 140 may also store the position of the solar panels as three dimensions (x, y, z) because the solar farm 520 may not be on level ground. All or part of the solar farm 520 may be built on a hillside or slope. Therefore, the onboard processor 116 and / or the FC system 140 may cause the UAV 102 to rise or fall as the UAV 102 flies over the solar farm 520 so as to maintain a desired altitude above the solar panels. The onboard processor 116 and / or the FC system 140 may store altitude data for each of the predefined flight positions 540 to 548 to control the vertical position of the UAV 102 when the UAV 102 captures images at each of the predefined flight positions 540 to 548.

[0084] The onboard processor 116 and / or the FC system 140 may determine a predefined route for the UAV by first determining a boundary 530 around the solar farm 520. The boundary 530 may form a geo-fence, wherein the control system of the UAV prevents the UAV from traveling outside the geo-fence. The onboard processor 116 and / or the FC system 140 may be further configured to determine a predefined route through the solar farm 520 within the boundary 530. The predefined route may follow one of the inspection flight modes 502 and 512. The onboard processor 116 and / or the FC system 140 may also be configured to determine predefined flight positions 540 to 548 along the predefined route of the UAV to capture images of the solar panels. One of the predefined flight positions 540 to 548 may be located near the solar panel 514. The onboard processor 116 and / or FC system 140 may then be configured to transmit the predefined route and predefined flight positions 540 to 548 to the UAV via a wired connection (such as a USB), a storage device (such as a thumb drive or SD card), and / or a wireless connection (such as 3G, 4G, 5G, Wifi, or Bluetooth).

[0085] The UAV may be configured to perform inspections manually, semi-automatically, or fully automatically. For manual inspections, a user on the ground may direct the UAV to an image capture location. The user may control the capture of images, or the UAV may be programmed to automatically capture images when the UAV is in the appropriate position. During semi-automatic inspections, a processor on the UAV may control the flight path, but the user may override the automatic control of the UAV by using a remote control. During automatic inspections, a processor on the UAV may command the UAV control to follow the flight path. Automatic or semi-automatic inspections may be based on GNSS navigation and / or visual ranging. Navigation may include a certain height above the ground (e.g., ten, twenty, or thirty meters), ground speed using GNSS signals from a three-dimensional Kalman filter, and / or compass and GNSS heading angles.

[0086] Figure 5B An infrared camera image for solar panel difference recognition and identification according to the techniques of the present disclosure is shown. Solar panel 514 includes seventy-eight segments arranged in six rows and thirteen columns. All of the segments have a temperature between twenty-five degrees Celsius and thirty degrees Celsius, except for one of the segments which has a temperature of forty-eight degrees Celsius. The significantly higher temperature of the segment may indicate a fault.

[0087] Depending on the severity and extent of the fault, the cloud computing system 106 may recommend repairs or maintenance on the solar panel 514. The cloud computing system 106 may be configured to determine that there is a fault in the high temperature section based on the infrared image of the solar panel 514. The cloud computing system 106 may be configured to determine the score of the image of the solar panel 514 based on the temperature and / or color of the solar panel 514 or the temperature and / or color of only a single section of the solar panel 514. The cloud computing system 106 may be configured to determine the fault based on the temperature and / or color of one section exceeding the threshold temperature and / or threshold color. If the cloud computing system 106 determines that the temperature of the solar panel 514 is too high, the cloud computing system 106 may direct the UAV back to the solar panel 514 to capture another infrared image to confirm the fault. The cloud computing system 106 may also verify the determination of the fault based on the amount of electrical power generated by a string of solar panels including the solar panel 514.

[0088] The cloud computing system 106 can use template matching and machine learning to determine faults based on temperature and / or color differences in the image. The cloud computing system 106 can determine the average temperature of the solar panel and the image color. Infrared can show temperature differences through color, and the cloud computing system 106 can determine irregular temperature patterns based on template thermal patterns from stored infrared images.

[0089] Figure 6is a diagram showing image capture angles according to the technology of the present disclosure. An image captured by a UAV directly above a solar panel may not have good resolution of the solar panel due to light reflections. Similarly, an image captured by a UAV at a low angle may not have good resolution of the solar panel due to light reflections. Images captured at forty degrees above horizontal to eighty degrees above horizontal may have good resolution. Figure 6 is one example of a desired image capture angle. In some examples, it may be desirable for the image capture angle to be less than forty or forty-five degrees above horizontal, or greater than eighty or eighty-five degrees above horizontal. Generally speaking, solar panels in the northern hemisphere may be positioned at an angle of twenty to forty degrees from vertical toward the south.

[0090] Figure 7 is a flow chart illustrating an exemplary operation for detecting a fault in a solar panel according to the techniques of the present disclosure. For example, Figure 7 The operation can be used to detect the structure 114 ( Figure 1 ) and solar panels 514( Figure 5A and Figure 5B ) in the fault.

[0091] exist Figure 7 In the example of , computing system 106 may obtain information generated by a UAV (e.g., Figure 1 and Figure 2 The cloud computing system 106 may also obtain a first image of a structure 114, which may be a solar panel (700), captured by the UAV 102 during a first flight. The cloud computing system 106 may also obtain a second image of the structure 114 captured by the UAV during a second flight (702). The structure 114 may include one or more elements of a solar panel assembly, including mechanical elements, electrical wires, transformers, breaker boxes, towers, ground, and nearby vegetation. The cloud computing system 106 may obtain the image in a variety of ways. For example, the cloud computing system 106 may obtain the image from a local or remote image archive such as the image archive 312 ( Figure 3 ) to retrieve images. In some examples, cloud computing system 106 can obtain images directly from the UAV. Transmission of images can be performed via a wired or wireless connection (such as SD card, Ethernet, USB, Wifi, 3G, 4G, 5G and / or Bluetooth).

[0092] Additionally, cloud computing system 106 may determine a first score for structure 114 based on the first image (704). For example, cloud computing system 106 may determine the score for the first image to be equal to the highest temperature of structure 114. In some examples, cloud computing system 106 may be configured to determine a score for another element of the solar panel assembly, such as an electrical wiring connection. The cloud computing system may also be configured to determine scores for ground erosion and vegetation overgrowth. Additionally, in Figure 7In the example of FIG. 1 , cloud computing system 106 may determine a second score for structure 114 based on the second image ( 706 ). The score may be based on the temperature and / or color of structure 114 or any visible defects on structure 114. Additionally, cloud computing system 106 may determine the score based on the intensity of light emitted by structure 114 during the electroluminescence test.

[0093] The cloud computing system 106 may determine that the structure 114 includes a fault based on the first score and the second score (708). The cloud computing system 106 may also output an indication that the solar panel includes a fault (710). The indication may include a graphical icon on a graphical user interface, where the graphical icon may indicate the location and condition of the solar panel. The cloud computing system 106 may also output an indication of a fault in other elements of the solar panel assembly as a graphical icon. The graphical icon may include information about which element has the fault so that a worker can find the target element.

[0094] Figure 8 is a conceptual diagram showing a technique for comparing photos according to the technology of the present disclosure. Figure 8 In the example of , the UAV captures multiple images at various image capture locations. The image capture locations are identified by (x, y, and z) coordinates. For example, the UAV may capture images (i.e., take photos) at locations (1,1,1), (2,1,1), etc. in a single session, such as Figure 8 As shown in the example of . The captured images may include an x-direction image, a y-direction image, and a z-direction image. The x-direction image includes images with the same y and z coordinates but different x coordinates. The y-direction image includes images with the same x and z coordinates but different y coordinates. The z-direction image includes images with different x and y coordinates but different z coordinates. The captured images are then stored in a database, as shown by the circle marked with a plus sign. In some examples, the captured images are transmitted to a predetermined cloud server for evaluation using 3G, 4G, 5G, NBIOT, or another wireless technology. Using these two storage technologies can help ensure the secure storage of the captured images.

[0095] exist Figure 8 In the example of FIG. 1 , for each image, the cloud computing system 106 may retrieve a database (e.g., the image archive 312 ( Figure 3 ))Retrieves an image with the same coordinates as this image, but that was captured at the time corresponding to Figure 8 The images in the "Photos Taken" box were taken at the time before the session time. Figure 8In , retrieval is represented by a circle containing an X mark. For example, for an image taken from an image capture location with coordinates (2,1,1), the computing system 106 may retrieve a historical image taken from an image capture location with coordinates (2,1,1) from a database (which may be located on an SD card, a cloud server, or other location). The cloud computing system 106 may then compare the two corresponding images. If there are significant differences between the corresponding images, an inspection process for the structure may be performed. Otherwise, no further action is taken until the next flight of the UAV to test whether the structure has a fault. The inspection process for the structure may be performed in various ways. For example, the inspection process for the structure may be offline, online, or semi-online. When the inspection process is offline, the inspection of the infrared image and the inspection of the visible light image are performed manually. When the inspection process is online, the inspection of the infrared image and the inspection of the visible light image are automatically performed by the cloud computing system 106. When the inspection process is semi-online, the inspection of the infrared image and the inspection of the visible light image are automatically performed by the cloud computing system 106, but the inspection results are manually verified and the arrangement of tasks for fixing or maintaining the structure is provided. In Figure 8 In the example of , cloud computing system 106 may perform the process for each of the x-direction image, the y-direction image, and the z-direction image.

[0096] The cloud computing system 106 may perform an automated mapping integration of the solar farm area to determine whether a set of images may cover the entire solar farm. For larger solar farms, the UAV may not be able to capture images of the entire solar farm during a single flight mission. Therefore, the cloud computing system 106 may need to stitch together multiple sets of images to form a complete model of the solar farm.

[0097] Fig. 9 A score decision graph for detecting faults in a solar panel according to the techniques of this disclosure is shown. Cloud computing system 106 may use such a graph to determine whether a solar panel has a fault based on the percentage of visible light images and infrared images that have scores exceeding a threshold. Fig. 9 In the example of , the UAV may capture both a visible light photograph (ie, a standard photograph) and an infrared image of the structure.

[0098] In some examples, cloud computing system 106 may determine whether a solar panel has a fault based on only one type of image (e.g., a visible light image and an infrared image). Cloud computing system 106 may acquire a set of at least two images of the solar panel. Cloud computing system 106 may then determine a set of scores for the solar panel based on the set of images. Cloud computing system 106 may be configured to assign a score to each image by, for example, determining one or more temperatures and / or one or more colors of the solar panel in each infrared image or determining whether there are any visible differences on the solar panel in each visible light image. Cloud computing system 106 may then be configured to determine whether each score exceeds a threshold score, which may represent the temperature and / or color of the infrared image or the size of the difference in the visible light image (e.g., the size of debris on the solar panel). Cloud computing system 106 may be further configured to compare the percentage of scores that exceed the threshold score with the threshold score percentage. Fig. 9 As shown, there may be more than one threshold percentage, such as two percent, five percent, ten percent, twenty percent, etc. If the cloud computing system 106 determines that the fractional percentage exceeds the threshold percentage, the cloud computing system 106 may determine that the solar panel requires maintenance (e.g., cleaning, washing), requires repair (e.g., repairing a mechanical defect in a portion of the solar panel), or requires replacement.

[0099] In some examples, cloud computing system 106 may be configured to determine a single score for a visible light image and an infrared image captured simultaneously. Cloud computing system 106 may be configured to determine a first score based on a first visible light image and a first infrared image of a solar panel captured at a first time (e.g., on day one). Cloud computing system 106 may be configured to determine a second score based on a second visible light image and a second infrared image captured at a second time.

[0100] The cloud computing system 106 may compare images in a new batch of images of the solar panel to determine whether the solar panel has a fault, as described elsewhere in this disclosure. Analysis of the new batch of images may indicate that only some of the images in the new batch of images show a fault in the solar panel (i.e., the images do not meet specifications). If the percentage of images showing structural movement is small enough, it is possible that the solar panel does not actually have a fault, or the fault may have little effect on the solar panel's power generation. A fault in a solar panel may not always be significant enough to change the power generation of a solar farm. The system of the present disclosure may help detect small faults in a solar farm. Alternatively, when the percentage of images showing a fault in a solar panel is within a certain percentage range, the solar panel may likely need maintenance or repair.

[0101] exist Fig. 9In an example, an image is considered "out of specification" if it has a score outside of an acceptable range. For example, an "out of specification" image may show a sufficiently large difference in the characteristics of a solar panel from the expected characteristics. For example, if an infrared image shows that the temperature and / or color of a portion of a solar panel exceeds an acceptable temperature threshold and / or an acceptable color threshold, then the infrared image may be considered "out of specification." In another example, if a standard visible light image shows cracks, peeling, warping or bending, debris accumulation (e.g., dust, metal filings, iron filings, bird droppings), etc. of the solar panel, then the image may be considered "out of specification."

[0102] The cloud computing system 106 may determine whether a solar panel is in good condition, whether a solar panel needs maintenance, whether a solar panel needs repair, or whether the ground level of a solar panel is out of specification based on the percentage of standard images of the solar panel that are “out of specification” and the percentage of infrared photos of the solar panel that are “out of specification”. Fig. 9 In the example of , if the number of “out-of-spec” standard images of the solar panel is less than 2% and the number of “out-of-spec” infrared images of the solar panel is also less than 2%, then the cloud computing system 106 may determine that the solar panel is in good condition. Fig. 9 In the example of FIG. 1 , if the number of “out-of-spec” standard images of a solar panel is less than 2% and the number of “out-of-spec” infrared images of a solar panel is between 2% and 5%, the cloud computing system 106 may determine that the solar panel requires maintenance. Fig. 9 percentages other than those shown in the examples.

[0103] If the cloud computing system 106 determines that the solar panel is in good condition, the cloud computing system 106 may continue to monitor the status of the solar panel, which is referred to as "tracking" in this article. The cloud computing system 106 may be configured to track the status of the solar panel by saving images and scores associated with the solar panel to determine whether maintenance, repair, or replacement of the solar panel is indicated. In some examples, the cloud computing system 106 may be configured to monitor the amount of electrical power generated by a string of solar panels to determine whether any of the solar panels in the string has a fault. If the (daily, weekly, monthly, quarterly, semi-annual, or annual) power generation rate is lower than a predefined ratio such as sixty percent or fifty percent, the cloud computing system 106 may reduce one or more thresholds of the solar panels in the string. A normal power generation rate may be, for example, eighty percent, so a reduction in the power generation rate below a predefined ratio may indicate a fault somewhere in the string.

[0104] The one or more threshold scores and the one or more threshold percentages may be based on an estimate of solar panel performance. For example, a small amount of dust may not significantly affect the amount of electricity generated by a solar panel. Therefore, cloud computing system 106 may be configured to assign a relatively small weight to a speck of dust and a relatively large weight to bird droppings on a solar panel, which can cause damage to the solar panel if not removed quickly. The customer can change Fig. 9 percentages shown to achieve higher power generation efficiency or lower maintenance costs and shorter repair times.

[0105] Fig.10 1 is a flow chart illustrating a visible light image control algorithm for fault classification according to the techniques of the present disclosure. The cloud computing system 106 may first collect images and use template matching to create a model of a solar farm (1000, 1002). The cloud computing system 106 may use template matching to match images to solar panels. If the image shows that the solar panel is obscured by debris (1004), the cloud computing system 106 may immediately schedule cleaning.

[0106] The cloud computing system 106 can then use corner detection to determine if a large crack exists in the solar panel (1006, 1008). Large cracks may be more likely to occur at the corners of the solar panel. The corner detection algorithm may define a corner of the solar panel or another structure as a point with low self-similarity. Each pixel in the image is algorithmically tested to see if a crack corner exists by considering how similar a patch centered at the pixel is to nearby mostly overlapping patches. Thus, fine problems, small problems, and large problems on the solar panel may be realized using a template matching method from the image.

[0107] The tasks of the above four groups of inspection status (good, clean, maintain, replace) will be assigned to each solar panel worker by using manual or automatic processes and procedures. The solar panel station supervisor or manager can act as the first level of the manager system to provide task allocation for daily tasks of each worker. Senior managers can monitor the entire maintenance and repair tasks using web portal technology. The entire work process can be based on a web-based system. Users can access information about solar panels via software applications so that users can observe the entire solar panel maintenance and repair status online on mobile devices. The system can also include the ability to easily generate monthly, quarterly, semi-annual and annual status reports.

[0108] Fig.11is a flow chart showing an infrared image control algorithm for fault classification according to the technology of this disclosure. The system will make recommendations based on temperature differences and color differences identified in the infrared image. For example, if the entire solar panel module is warmer than the common module that may indicate an interconnect problem. If a single cell or a string of cells shows as a hot spot or a warmer "patch pattern", the cause may be a defective bypass diode, an internal short circuit, or a cell mismatch. Shadows and cracks in the cell can show up as hot spots or polygonal patches in the thermal image. A rise in the temperature of a cell or a portion of a cell indicates a defective cell or the presence of a shadow.

[0109] Fault identification based on temperature and / or color in infrared images may include threshold logic. For example, if the measured panel temperature exceeds a predefined value of twenty degrees Celsius, the cloud computing system 106 may determine that the panel is overheated and generate a message notifying the user to replace the panel. If the measured temperature differs from the predefined threshold by five degrees Celsius, the cloud computing system 106 may determine that the panel is in good or normal condition and does not require maintenance. If the measured temperature exceeds the predefined threshold by ten degrees Celsius, the cloud computing system 106 may assign the panel to a technician for further inspection. The technician may make maintenance decisions offline. There may be four states: repair, maintenance, estimated percentage of failure, and tracking. The cloud computing system 106 may assign small cracks to the maintenance and tracking status categories.

[0110] One method of testing solar panels includes electroluminescence testing for detailed diagnostic purposes. Solar panel electroluminescence testing relies on the same principles as testing light emitting diodes (LEDs). In order for a solar farm to perform electroluminescence testing, a low-power feedback switch design can first be installed in the solar panel station. The switch will be designed to allow a single solar panel area to be turned on or off for electroluminescence testing. During this electroluminescence test, the solar panel shifts from generating electricity to consuming electrical energy. Due to this reverse design, the solar panel acts as an LED and emits light that can be detected using visible light images.

[0111] Using the brightness and darkness on the solar panel through a special electroluminescence camera, the cloud computing system 106 can determine the quality status of the solar panel. In the electroluminescence test, the peak brightness of the solar panel can be a current of about one hundred nanoamps in the form of a Gaussian distribution graph, but each solar panel manufacturer can have different design methods and current levels. Therefore, the expected current for electroluminescence testing in a solar farm can be five hundred milliamps, one amp, two amps, or four amps. The voltage can be twelve, twenty-four, or forty-eight volts DC.

[0112] Electroluminescence testing may work best with a higher resolution camera of up to 6000×4000 pixels, but lower resolution infrared cameras can also be used for electroluminescence testing. Therefore, if the test uses a larger viewpoint of each solar panel with low power injection, a standard infrared camera with 640×480 pixels can be used for thermal electroluminescence inspection. For electroluminescence testing, nighttime inspections, especially in the absence of moonlight, are best. Electroluminescence testing is a good solar panel fault detection method to detect faults such as cracks and faulty sections of the solar panel. When solar cells have aged, electroluminescence testing may be a good way for photovoltaic modules to become more efficient in solar energy conversion. Electroluminescence testing can detect structural defects, lattice quality, and overall encapsulant quality.

[0113] When a feedback current is induced in a group of solar panels, the solar panels will produce electroluminescence (i.e., light). The electroluminescence of the solar panels will result in one of these results: standard emission for no faults; weak emission in the near infrared (NIR) to detect shunting; crystallization defects; and broken finger electrodes that may have occurred during the cell manufacturing process.

[0114] Ambient brightness will interfere with electroluminescence testing, so it may be desirable to conduct electroluminescence testing at night with little or no moonlight. It may also be desirable to conduct electroluminescence testing in rainless or low humidity conditions. A power storage system can provide current to drive the solar panel into electroluminescence mode. Solar cells are essentially large diodes, and the radiative recombination of carriers causes light emission. The solar panel status can be captured using an infrared camera. The cloud computing system 106 can use infrared images to determine individual solar panel failures and / or defects due to aging or manufacturing errors. The cloud computing system 106 can use software template matching techniques to automatically provide the user with correct repair or maintenance suggestions.

[0115] As an indirect bandgap semiconductor, most recombination in silicon occurs via defects or Auger recombination. The amount of interband recombination that produces radiative emission can be relatively low. However, even in silicon a small amount of radiative recombination can occur, and an external detector can be used to sense the signal. The technique requires electrical contacts and therefore can only be used after metallization has been applied and the cell is substantially complete. Electroluminescence testing can provide a wealth of data on the area-dependent uniformity of solar cells and modules.

[0116] If the solar panel plant has a power feedback switch device for electroluminescence testing, the cloud computing system 106 can use a standard UAV flight plan and a thermal camera trigger / scanning system. The cloud computing system 106 can also use the entire solar field and three-dimensional mapping technology to perform electroluminescence testing. In order to fully integrate each piece of data, a special button can be used to set the solar panel electroluminescence test in the radio controller and / or on the ground control station. Once the solar panel station electroluminescence button is turned on, the cloud computing system 106 can automatically perform the electroluminescence test using the UAV flight plan, and the camera trigger function can be activated. A gap list of individual test steps can be designed so that the cloud computing system 106 can automatically complete the entire test. Based on the electroluminescence test, the cloud computing system 106 can determine internal faults, structural faults, usage faults, and / or manufacturing defect faults.

[0117] The cloud computing system 106 may be configured to determine a score for the solar panel by at least determining the light intensity emitted by the solar panel in the first image during the first electroluminescence test. Low light intensity may indicate a fault such as a crack, a broken electrical connection, or another fault. The cloud computing system 106 may assign a higher score to indicate a possible fault based on the lower light intensity. The cloud computing system 106 may then be configured to determine a second score for the solar panel by at least determining the second light intensity emitted by the solar panel in the second image during the second electroluminescence test. The second electroluminescence test may be conducted on a different date than the first electroluminescence test. The electroluminescence test may occur at night during times with less moonlight (e.g., cloudy nights and / or a new moon phase).

[0118] Fig.12 1 is a flow chart illustrating mapping and combining visible light and infrared images for fault detection according to the techniques of the present disclosure. On the infrared images, two comparison algorithms can be used. The first algorithm is based on infrared image light enhancement and density. The second algorithm is based on infrared image temperature. In the first algorithm, the cloud computing system 106 determines the color range of each image of the solar panel (1200). The cloud computing system 106 can then be configured to apply a self-learning process or a machine learning algorithm to detect solar panel faults on two or more images of the solar panel (1202). The cloud computing system 106 can determine the fault based on the color indicating the temperature difference (1204). The cloud computing system 106 can analyze the temperature and / or color in the infrared image to determine the condition of the solar panel.

[0119] In the second algorithm, the infrared image camera will generate two kinds of data based on the infrared image: location data and individual distributed temperatures (1206). Since the infrared image temperature is distributed, the average temperature of each solar panel can be calculated (1208). The cloud computing system 106 can then automatically calculate the temperature of the entire solar panel farm, where the outside weather temperature is used as the normal temperature of a single solar panel group. The cloud computing system 106 can use the average temperature and the normal temperature of all solar panels in the solar panel farm as the basis for identifying irregular temperatures.

[0120] If parts of the solar panel are hotter than other parts, the warm areas will be clearly displayed in the infrared image (1210). Depending on the shape and location, hot spots and areas can indicate several different faults. The cloud computing system 106 can use algorithms to provide condition diagnostics to determine whether the temperature of the solar panel is normal or abnormal. The cloud computing system 106 can be configured to provide an indication to the user whether the identified "abnormal" solar panel requires further screening or maintenance. The cloud computing system 106 can sum the results of the two algorithms and publish the fault location to the cloud network for distribution to users. The cloud computing system 106 can cause the mobile device to present an indication of the fault location to the user by presenting a graphical user interface to the user (1212). The graphical user interface may include grid lines for the entire solar farm, with each image location positioned on the grid line (1214).

[0121] The cloud computing system 106 can make recommendations based on the identified temperature differences. For example, if the entire module is warmer than usual, this condition may indicate an interconnect problem with the module. If the cloud computing system 106 determines that individual cells or strings of cells have hot spots or warmer "patch patterns", the cause may be a defective bypass diode, an internal short circuit, or a cell mismatch. The cloud computing system 106 can determine that the solar panel has shadows and cracks in the cells based on hot spots or polygonal patches in the thermal image. The temperature increase of a cell or a portion of a cell may indicate that the cell is defective or shadowed.

[0122] Temperature-based fault identification may be based on threshold logic. For example, if the measured panel temperature exceeds a predefined value of twenty degrees Celsius, the cloud computing system 106 may determine that the panel is an overheated panel and generate a message notifying the user to replace the panel. If the measured temperature differs from the predefined threshold by five degrees Celsius, the cloud computing system 106 may classify the panel as a normal panel and recommend no maintenance. If the measured temperature exceeds the predefined threshold by ten degrees Celsius, the cloud computing system 106 may recommend that the solar panel be further inspected by a technician who can determine the appropriate type of offline maintenance.

[0123] Fig.13is a flow chart illustrating task management for repair, maintenance, and tracking according to the techniques of the present disclosure. The cloud computing system 106 can be configured to determine whether a solar panel should have repair (1300) or maintenance (1308) based on the score assigned to the image (see, e.g. Fig. 9 ). If the cloud computing system 106 determines that the solar panel needs repair, the cloud computing system 106 can be configured to double check by analyzing electroluminescence visual or electroluminescence testing (1302). If the cloud computing system 106 verifies that repair is needed (1304), the cloud computing system 106 will move the solar panel to the repair list (1306). Therefore, the solar panel will remain in the repair group due to damage, exceeding the threshold score, and / or internal disconnection.

[0124] If cloud computing system 106 determines that no repair is needed, cloud computing system 106 can be configured to move the solar panel to a maintenance list ( 1308 ) and calculate an average normal temperature of the solar panel based on the infrared image ( 1310 ). Figure 5B An example of a temperature difference across a portion of a solar panel is shown. If the cloud computing system 106 verifies the maintenance criteria (1312), the cloud computing system 106 may move the solar panel to a tracking list for further monitoring. Verifying the maintenance criteria may include checking the solar panel to determine if maintenance is required. For example, a user may review the image to determine if the visible defect is a blade or another object that may no longer be on the solar panel. If the user determines that the visible defect is an object adhered to the solar panel, such as bird droppings or mud spots, the user may mark the solar panel for cleaning.

[0125] Factors used to determine whether to place the solar panel on the tracking list include the percentage of portions of the solar panel that are damaged, the likelihood that the solar panel will reduce the performance of the entire area of ​​the solar panel, and the likelihood that the solar panel will experience a fire, smoke, emergency, or explosion. If the cloud computing system 106 does not verify the maintenance criteria, the cloud computing system 106 may keep the solar panel in the maintenance group (1316) and move to the next repair item (1318). The cloud computing system 106 may then continue to consider the next flight priority item (e.g., a subsequent solar panel).

[0126] Fig.14is a flow chart showing a validation control algorithm for two-dimensional / three-dimensional mapping techniques according to the technology of the present disclosure. The cloud computing system 106 may fly the UAV to a starting point in a solar farm based on the three-dimensional coordinates of the starting point (1400, 1402, 1404). The cloud computing system 106 may also cause the UAV to capture images at a predefined scanning speed and at a predefined flight position in the solar farm (1406). The UAV may store the images to an SD card and / or wirelessly transmit the image data to the cloud computing system 106 (1408). If the cloud computing system 106 determines that the captured image does not cover the desired scanning area, the cloud computing system 106 may cause the UAV to repeat the flight plan to capture the missing images (1410, 1412). Otherwise, the cloud computing system 106 may store the images and grid lines indicating the location of each image to the cloud network (1414).

[0127] Fig.15 is a flow chart illustrating a procedure for electronic repair according to the techniques of this disclosure. A solar farm may include a central control processing station that monitors the total power generation of the solar farm. The smallest piece of the solar farm that the control station can monitor and manage may be limited to a set of circuit breaker sections or a small transformer section. The control station may take into account environmental conditions such as time of year, time of day, cloud cover, precipitation, and other conditions that may affect the amount of power generated. In addition, individual solar panel conditions such as the percentage of solar panels affected may also be included in the analysis flow chart.

[0128] There may be four groups of solar panels: recommended repair, recommended maintenance, percentage of affected area, historical tracking data. In some examples, cloud computing system 106 may collapse the four groups into two groups: recommended repair and recommended maintenance. Cloud computing system 106 may move the group of recommended maintenance, the group of percentage of affected area, and the group of historical tracking data back into the maintenance review group.

[0129] The cloud computing system 106 can send the repair list to the station manager, who can assign tasks to field workers (1500). The repair workers can also use mobile devices such as smart phones or tablets to understand the location of each solar panel, identify which solar panel needs to be repaired, and ensure that the damaged solar panel has the condition determined by the cloud computing system 106 based on the images from the UAV inspection. Each field worker can receive the repair list via a mobile device (1502).

[0130] The cloud computing system 106 may be configured to generate and output an alert indicating that the solar panel has a fault by causing the computing device to display the location and condition of the solar panel to the station manager. The cloud computing system 106 may also be configured to cause the mobile device to display the location and condition to the field worker so that the field worker can find the solar panel. Depending on the condition of the solar panel, the field worker may obtain a replacement solar panel and / or repair tools from the warehouse (1504). The mobile device may also present the field worker with a repair procedure and a repair execution sheet or checklist.

[0131] To perform the repair, the mobile device may present the location of the solar panel to the field worker to help the field worker identify the solar panel (1506). The field worker may verify that the cloud computing system 106 accurately determined the condition of the solar panel. The mobile device may also be configured to present the inspection procedure and sign-off form to the field worker (1508). The mobile device may also present the inspection tool list to the field worker. After the repair and inspection are complete, the mobile device may direct the field worker to the next repair task (1510).

[0132] During the preparation, repair, and inspection processes, the cloud computing system 106 may be configured to present a graphical user interface to the user. The graphical user interface may include a map of the solar farm and a graphical icon indicating the location of the solar panels in the solar farm. The location of the graphical icon on the graphical user interface may help users (e.g., field workers) locate solar panels within the solar farm. The cloud computing system 106 may present a graphical icon of the solar panel in response to determining that the solar panel has a fault. In this way, the graphical user interface may only show the solar panels with faults to reduce clutter on the graphical user interface. In some examples, the graphical user interface may include functionality that allows the user to access details of the solar panels (including non-faulty solar panels).

[0133] The cloud computing system 106 can be configured to present a graphical user interface via a mobile device, which can be remote from the cloud computing system 106. The cloud computing system 106 and the mobile device can communicate via a cloud network, such as a virtual private network. The cloud network can connect mobile devices, desktop devices, and server computers. All parties in the solar farm, such as managers, workers, shareholders, etc., can access the cloud network. The cloud technology used to automatically save the image on the web server is not limited to a local or global Internet cloud. The cloud technology can be a private and / or public cloud protected by a user identification and password. The cloud network can generate a software application for a mobile device to present a map and indication of solar panel failures to the user.

[0134] The cloud computing system 106 may be configured to receive user input from a first user, such as a manager. The user input may indicate that the manager is assigning the solar panel to a second user, such as a field worker. In response to receiving the user input, the cloud computing system 106 may be configured to associate the solar panel with the second user. The cloud computing system 106 may then be configured to determine that the mobile device is associated with the second user, and present a graphical user interface via the mobile device in response to determining that the mobile device is associated with the second user. If the mobile device is a smartphone or tablet assigned by the second user's company, the mobile device may be associated with the second user.

[0135] The cloud computing system 106 may also be configured to determine a confidence metric for determining that the solar panel includes a fault. The confidence metric may indicate the severity of damage to the solar panel. In some examples, the color or size of the graphical icon may indicate the severity of the problem with the solar panel. For example, a red graphical icon may indicate a more severe or more urgent problem, and a green graphical icon may indicate a less severe or less urgent problem.

[0136] The cloud application may include the following functions. First, the cloud application may include an automatic flight plan for the UAV and a predefined flight location for triggering the camera to capture images. The cloud application may automatically receive images from the UAV via wireless transmission and generate a mapping model of the solar farm with the images. The cloud application may then automatically analyze the images to detect faults in the solar panels. The cloud application may classify each fault into a group or category (good, maintenance, or repair) and assign maintenance and repair tasks to users. The cloud application may also generate reports including relevant data for the user. The cloud computing system may be linked to a power generation monitoring computing system for power efficiency verification.

[0137] Fig.16 is a flow chart illustrating a software algorithm for forming a model of an entire solar farm according to the technology of the present disclosure. The model of the solar farm may be based on one or more sets of images captured on one or more UAV flights. The solar farm may include several zones, each of which may include solar panels, towers, transformers, inverters, circuit breaker boxes, and / or wires. For example, between zone 1 and zone 2, the solar farm may include towers and wires. When mapped zone 1 and mapped zone 2 are connected together, the cloud computing system 106 may include a map of each zone and a map of the towers and wires. The cloud computing system 106 may store a map of the solar farm and use the map to generate a two-dimensional or three-dimensional model of the solar farm. The model may allow a user to zoom in or out of a specific solar panel to view the entire solar farm or group of solar panels.

[0138] The cloud computing system 106 may be configured to build a model of a solar farm and determine, based on the model, that the first image shows a particular solar panel within the solar farm. The cloud computing system 106 may associate the first image with the solar panel in response to determining that the first image shows the solar panel. The association between the first image and the solar panel may be a software object that causes the first image to be displayed when a user searches for solar panels. The cloud computing system 106 may receive a second image and determine, based on the model, that the second image shows a solar panel. The cloud computing system 106 may associate the second image with the solar panel in response to determining that the second image shows the solar panel.

[0139] Fig.17 A fault detection and identification process according to the techniques of this disclosure is shown. Fig.17 Three groups of solar panels are shown: good condition, maintenance, and repair. For solar panels that require maintenance and repair, the cloud computing system 106 can be configured to arrange maintenance or repair by assigning the solar panel to a field worker and alerting the worker to the location and condition of the solar panel. The cloud computing system 106 can track the condition of the solar panel to assess the progress of the fault (e.g., whether maintenance has resolved the fault). Tracking can include evaluating the scores of future images of the solar panel to determine whether the condition has improved. For solar panels that have been repaired, the cloud computing system 106 can keep the solar panel on the repair list until the cloud computing system 106 determines that new images show that the fault has been cleared (e.g., the solar panel is in good condition).

[0140] Fig.18 A graphical user interface 1800 is shown that includes graphical icons 1820 and 1830 indicating solar panels with faults according to some examples of the present disclosure. Graphical user interface 1800 includes a map of a solar farm with multiple rows of solar panels 1810 to 1813. Cloud computing system 106 can present graphical user interface 1800 to a user, or can cause a mobile device to present graphical user interface 1800 to a user. Graphic icons 1820 and 1830 represent solar panels with faults.

[0141] A user may interact with graphical user interface 1800 by providing user input. For example, a user may select graphical icon 1820 or graphical icon 1830 to access information about solar panels and faults, including images of solar panels. A user may zoom in or out of a view of a solar farm, or a user may move the view laterally without changing the view. As the user zooms in, graphical user interface 1800 may present graphical icons for each solar panel, including solar panels that do not have faults. Graphical user interface 1800 may present information for a particular solar panel, even a non-faulty solar panel, based on user input indicating that the user has selected a graphical icon associated with a particular solar panel. The view of the solar farm shown in graphical user interface 1800 may be the entire solar farm or only a small portion of the solar farm.

[0142] In one or more examples, the described functions may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or communication media including any medium that facilitates the transmission of a computer program from one place to another, for example, according to a communication protocol. Thus, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0143] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical optical disks, digital versatile disks (DVDs), floppy disks and Blu-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to copy data optically. The above combination should also be included in the scope of computer-readable media.

[0144] Instructions may be executed by one or more processors such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, these techniques may be implemented entirely in one or more circuits or logic elements.

[0145] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in the present disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0146] Embodiment 1. A method for detecting a fault in a solar panel in a solar farm, the method comprising acquiring, by a computing system, a first image of the solar panel captured by a UAV during a first flight and a second image of the solar panel captured by the UAV during a second flight. The method further comprises determining, by the computing system, a first score for the solar panel based on the first image, and determining a second score for the solar panel based on the second image. The method further comprises determining, by the computing system, that the solar panel includes a fault based on the first score and the second score. The method comprises outputting, by the computing system, an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0147] Embodiment 2. The method of embodiment 1 further includes acquiring, by the computing system, a set of images of the solar panel, the set of images including the first image and the second image captured by the UAV during a set of flights including the first flight and the second flight. The method also includes determining, by the computing system, a set of scores for the solar panel based on the set of images by determining a corresponding score for the solar panel based at least on each corresponding image in the set of images. The method includes determining, by the computing system, that at least a threshold score percentage in the set of scores is less than a threshold score, wherein determining that the solar panel includes the fault is further based on determining that at least the threshold score percentage in the set of scores is less than the threshold score.

[0148] Embodiment 3. The method of embodiments 1 to 2 or any combination thereof, wherein the first image is a first visible light image of the solar panel, and the second image is a second visible light image of the solar panel. The method further includes acquiring, by the computing system, a first infrared image of the solar panel captured by the UAV during the first flight. The method further includes acquiring, by the computing system, a second infrared image of the solar panel captured by the UAV during the second flight. Determining the first score of the solar panel is further based on the first infrared image, and determining the second score of the solar panel is further based on the second infrared image.

[0149] Embodiment 4. The method of embodiments 1 to 3 or any combination thereof, wherein determining the first score of the solar panel based on the first infrared image comprises determining the first score of the solar panel based on a temperature of the solar panel in the first infrared image. Additionally, determining the second score of the solar panel based on the second infrared image comprises determining the second score of the solar panel based on a temperature of the solar panel in the second infrared image. The color of the solar panel may be used in addition to or in lieu of the temperature of the solar panel.

[0150] Embodiment 5. The method of embodiments 1 to 4 or any combination thereof, wherein determining the first score for the solar panel comprises determining a first light intensity emitted by the solar panel in the first image during a first electroluminescence test. Additionally, determining the second score for the solar panel comprises determining a second light intensity emitted by the solar panel in the second image during a second electroluminescence test.

[0151] Embodiment 6. The method of embodiments 1 to 5 or any combination thereof, further comprising presenting a graphical user interface including a map of the solar farm via a mobile device.

[0152] Embodiment 7. The method of embodiments 1 to 6 or any combination thereof, wherein outputting the indication that the solar panel includes the fault comprises presenting a graphical icon of a solar panel on the graphical user interface via the mobile device in response to determining that the solar panel includes the fault.

[0153] Embodiment 8. The method of embodiments 1 to 7 or any combination thereof, further comprising receiving, by the computing system, a user input from a first user, and associating, by the computing system, the solar panel with a second user in response to receiving the user input. The method further comprises determining, by the computing system, that the mobile device is associated with the second user. The method further comprises presenting, via the mobile device, the graphical user interface including the graphical icon in response to determining that the solar panel includes the fault and in response to determining that the mobile device is associated with the second user.

[0154] Embodiment 9. The method of embodiments 1 to 8 or any combination thereof, further comprising determining a confidence metric for determining that the solar panel includes the fault, wherein presenting the graphical icon comprises presenting the graphical icon of the solar panel on the graphical user interface via the mobile device based on the confidence metric.

[0155] Embodiment 10. The method of embodiments 1 to 9 or any combination thereof, wherein the fault is a first fault, the method further comprising determining, by the computing system and based on the first image, a third score for an element of a solar panel assembly that includes the solar panel, wherein the element is not a solar panel. The method further comprises determining, by the computing system and based on the second image, a fourth score for the element, and determining, by the computing system, that the element includes a second fault based on the third score and the fourth score. The method comprises outputting, by the computing system, an indication that the element includes the second fault in response to determining that the element includes the fault.

[0156] Embodiment 11. The method of embodiments 1 to 10 or any combination thereof, further comprising constructing a model of the solar farm and determining based on the model of the solar farm that the first image shows the solar panel. The method further comprises associating the first image with the solar panel in response to determining that the first image shows the solar panel. The method further comprises determining based on the model of the solar farm that the second image shows the solar panel. The method comprises associating the second image with the solar panel in response to determining that the second image shows the solar panel.

[0157] Embodiment 12. A computing system including a memory configured to store a first image of a solar panel captured by a UAV during a first flight, and to store a second image of the solar panel captured by the UAV during a second flight. The computing system also includes one or more processing circuits configured to determine a first score for the solar panel based on the first image, and to determine a second score for the solar panel based on the first image. The one or more processing circuits are further configured to determine whether the solar panel includes a fault based on the first score and the second score. The one or more processing circuits are further configured to output an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0158] Embodiment 13. The computing system of embodiment 12, wherein the memory device is configured to store a set of images of the solar panel, the set of images including the first image and the second image captured by the UAV during a set of flights including the first flight and the second flight. The processing circuit is further configured to determine a set of scores for the solar panel based on the set of images by determining a corresponding score for the solar panel based at least on each corresponding image in the set of images. The processing circuit is also configured to determine that at least a threshold percentage of the set of scores is less than a threshold score. The processing circuit is configured to determine that the solar panel includes the fault further based on determining that at least the threshold percentage of the set of scores is less than the threshold score.

[0159] Embodiment 14. The computing system of Embodiments 12 to 13 or any combination thereof, wherein the first image is a first visible light image of the solar panel and the second image is a second visible light image of the solar panel. The memory device is further configured to store a first infrared image of the solar panel captured by the UAV during the first flight, and to store a second infrared image of the solar panel captured by the UAV during the second flight. The processing circuit is configured to determine the first score of the solar panel further based on a temperature of the solar panel in the first infrared image. In addition, the processing circuit is configured to determine the second score of the solar panel further based on a temperature of the solar panel in the second infrared image.

[0160] Embodiment 15. The computing system of embodiments 12 to 14 or any combination thereof, wherein the processing circuit is configured to determine the first score for the solar panel by at least determining a first light intensity emitted by the solar panel in the first image during a first electroluminescence test. The processing circuit is configured to determine the second score for the solar panel by at least determining a second light intensity emitted by the solar panel in the second image during a second electroluminescence test.

[0161] Embodiment 16. The computing system of embodiments 12 to 15 or any combination thereof, wherein the processing circuit is further configured to present, via a mobile device, a graphical user interface including a map of the solar farm. The processing circuit is configured to output the indication that the solar panel includes the fault by presenting, at least via the mobile device, a graphical icon of the solar panel on the graphical user interface in response to determining that the solar panel includes the fault.

[0162] Embodiment 17. The computing system of embodiments 12 to 16 or any combination thereof, wherein the processing circuit is further configured to receive user input from a first user and, in response to receiving the user input, associate the solar panel with a second user. The processing circuit is further configured to determine that the mobile device is associated with the second user and, in response to determining that the solar panel includes the fault and in response to determining that the mobile device is associated with the second user, present, via the mobile device, the graphical user interface including the graphical icon.

[0163] Embodiment 18. The computing system of embodiments 12 to 17 or any combination thereof, further comprising determining a confidence metric for determining that the solar panel includes the fault, wherein presenting the graphical icon comprises presenting the graphical icon of the solar panel on the graphical user interface via the mobile device based on the confidence metric.

[0164] Embodiment 19. The computing system of embodiments 12 to 18 or any combination thereof, wherein the processing circuit is further configured to build a model of the solar farm and determine, based on the model of the solar farm, that the first image shows the solar panel. The processing circuit is further configured to associate the first image in response to determining, based on the model of the solar farm, that the first image shows the solar panel. The processing circuit is further configured to determine, based on the model of the solar farm, that the second image shows the solar panel, and associate the second image in response to determining, based on the model of the solar farm, that the second image shows the solar panel.

[0165] Embodiment 20. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a computing system to acquire a first image of the solar panel captured by a UAV during a first flight and a second image of the solar panel captured by the UAV during a second flight. The instructions, when executed, also cause the computing system to determine a first score for the solar panel based on the first image, and to determine a second score for the solar panel based on the first image. The instructions, when executed, also cause the computing system to determine whether the solar panel includes a fault based on the first score and the second score. The instructions, when executed, cause the computing system to output an indication that the solar panel includes the fault in response to determining that the solar panel includes the fault.

[0166] Various embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims

1. A method for detecting a fault in a solar panel in a solar farm, the method comprising: Acquiring, by a computing system, a set of images of the solar panel, including a first image of the solar panel captured by an unmanned aerial vehicle (UAV) during a first flight; acquiring, by the computing system, a second image of the solar panel captured by the UAV during a second flight; determining, by the computing system, a set of scores for the solar panel based on the set of images at least by determining a respective score for the solar panel based on each respective image in the set of images, the set of scores comprising a first score for the solar panel based on the first image and a second score for the solar panel based on the second image; determining, by the computing system, that at least a threshold percentage of the scores in the set of scores are less than a threshold score; determining, by the computing system, that the solar panel comprises a fault based on determining that at least a threshold percentage of the scores in the set of scores is less than a threshold score; In response to determining that the solar panel includes the fault, an indication is output by the computing system that the solar panel includes the fault.

2. The method according to claim 1, The first image is a first visible light image of the solar panel, The second image is a second visible light image of the solar panel, and the method further comprises: acquiring, by the computing system, a first infrared image of the solar panel captured by the UAV during the first flight; as well as acquiring, by the computing system, a second infrared image of the solar panel captured by the UAV during the second flight, wherein determining the first score for the solar panel is further based on a temperature of the solar panel in the first infrared image, and Wherein determining the second score for the solar panel is further based on the temperature of the solar panel in the second infrared image.

3. The method according to claim 1, wherein determining the first score for the solar panel comprises determining a first light intensity emitted by the solar panel in the first image during a first electroluminescence test, and Wherein determining the second score for the solar panel includes determining a second light intensity emitted by the solar panel in the second image during a second electroluminescence test.

4. The method according to claim 1, further comprising: receiving, by the computing system, user input from a first user; associating, by the computing system, the solar panel with a second user in response to receiving the user input; determining, by the computing system, that the mobile device is associated with the second user; as well as In response to determining that the solar panel includes the fault and in response to determining that the mobile device is associated with the second user, presenting, via the mobile device, a graphical user interface including a map of the solar farm and graphical icons.

5. The method according to claim 1, further comprising: presenting, via a mobile device, a graphical user interface including a map of the solar farm; as well as determining a confidence metric for determining that the solar panel includes the fault, Wherein outputting an indication that the solar panel includes the fault includes presenting, via the mobile device, a graphical icon of the solar panel on the graphical user interface based on the confidence metric and in response to determining that the solar panel includes the fault.

6. The method according to claim 1, wherein the fault is a first fault, the method further comprising: determining, by the computing system and based on the first image, a third score for an element of a solar panel assembly that includes the solar panel, wherein the element is not the solar panel; determining, by the computing system and based on the second image, a fourth score for the element; determining, by the computing system, whether the component includes a second fault based on the third score and the fourth score; as well as In response to determining that the element includes the second fault, an indication that the element includes the second fault is output by the computing system.

7. A computing system comprising: A memory device, the memory device being configured to: storing a set of images of a solar panel, including a first image captured by an unmanned aerial vehicle (UAV) during a first flight and a second image captured by the UAV during a second flight; and A processing circuit, the processing circuit being configured to: determining a set of scores for the solar panel based on the set of images at least by determining a respective score for the solar panel based on each respective image in the set of images, the set of scores comprising a first score for the solar panel based on the first image and a second score for the solar panel based on the second image; determining that at least a threshold percentage of the scores in the set of scores are less than a threshold score; determining whether the solar panel includes a fault based on determining that at least a threshold percentage of the scores in the set of scores is less than the threshold score; and In response to determining that the solar panel includes the fault, an indication that the solar panel includes the fault is output.

8. The computing system according to claim 7, The first image is a first visible light image of the solar panel, Wherein the second image is a second visible light image of the solar panel, wherein the memory device is further configured to: storing a first infrared image of the solar panel captured by the UAV during the first flight; and storing a second infrared image of the solar panel captured by the UAV during the second flight, wherein the processing circuit is configured to determine the first score for the solar panel further based on a temperature of the solar panel in the first infrared image, and Wherein the processing circuit is configured to determine the second score for the solar panel further based on a temperature of the solar panel in the second infrared image.

9. The computing system according to claim 7, wherein the processing circuit is configured to determine the first score for the solar panel at least by determining a first light intensity emitted by the solar panel in the first image during a first electroluminescence test, and Wherein the processing circuit is configured to determine the second score for the solar panel at least by determining a second light intensity emitted by the solar panel in the second image during a second electroluminescence test.

Images