Solar panel inspection system and inspection method

Abstract

This invention provides a remote, contactless, and labor-saving remote inspection system and method that can present inspection reports containing evaluation results much faster and automatically than conventional methods. [Solution] The system includes: an imaging device 20 that captures infrared and visible light videos of the light-receiving surface 14U of the solar panel array 14 from a position P in the sky, far above the light-receiving surface 14U; a video acquisition unit 22 that acquires the infrared and visible light videos captured by the imaging device 20, with information on the installation positions of each of the multiple solar modules 12 added to them; an operating state analysis unit 24 that analyzes the operating state of the photoelectric conversion equipment 18 based on the infrared and visible light videos; and a report creation unit 26 that creates an inspection report based on the operating state analyzed by the operating state analysis unit 24.

Description

[Technical Field]

[0001] The present invention relates to a system and method for remotely inspecting the state of an object based on thermal images, and more particularly to a system and method for remotely inspecting the internal operating state or surface state of an object, such as a solar panel, by collecting thermal images of the object's surface and using the heat generation information of the object as shown in the thermal images. [Background technology]

[0002] In recent years, remote inspection systems that use images of objects such as solar panels, taken from a distance, to inspect the surface condition and internal operating state of those objects have been attracting attention.

[0003] This situation stems from a need to efficiently inspect the deterioration of infrastructure, particularly high-rise buildings, tunnel walls, and bridges—areas that are difficult for people to access—as well as solar panels, which have been rapidly installed in recent years. Currently, a considerable number of infrastructure structures that are over 50 years old and showing signs of deterioration remain in place, requiring urgent attention. Furthermore, regular inspections and checks are essential not only for such aging infrastructure but also for relatively new infrastructure such as solar panels (including so-called mega solar power plants) to maintain their performance. However, there is a shortage of personnel and budget for managing such inspections and checks. As a result, infrastructure inspections are not keeping pace, and there are particular concerns about an increase in dangerous areas. At the very least, regular inspections at appropriate times are required to prevent accidents.

[0004] In this context, an example of a remote inspection system is the "solar panel inspection device" described in Patent Document 1. This inspection device is configured to mount an infrared camera on an unmanned aerial vehicle (drone), photograph the solar panel from above with the infrared camera, and inspect the solar panel for abnormal heat generation (hot spots) on a cell-by-cell basis from the images captured by the infrared camera.

[0005] Furthermore, Patent Document 2 also provides an example of a system that uses an infrared camera mounted on an unmanned aerial vehicle to remotely photograph solar panels. In this example as well, modules exhibiting abnormalities (hot spots) are identified from the temperature distribution of the thermal image of the panel collected by the infrared camera. In particular, Patent Document 2 is characterized by avoiding duplicate determination of abnormal areas by using the latitude and longitude information of the abnormal areas. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Utility Model Registration No. 3246475 Publication [Patent Document 2] Japanese Patent Publication No. 2020-112499 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, even if the inspection functions described in Patent Documents 1 and 2 above are used, they are far from solving the problems faced in actual inspection sites. That is, at the site, for example, the surface condition of each panel (dirt caused by wind and dust, bird droppings, leaves and other foreign matter, and junction boxes connecting the panels and wiring) can be observed visually while walking around the solar panel site, or they can be observed visually from a high vantage point using a telescope. When the scale of solar panels is large, such visual observation is not time-consuming and is prone to oversights, so the mechanized systems described in Patent Documents 1 and 2 above are certainly desirable. At the same time, a disadvantage of simple visual observation is that it is difficult to detect areas with abnormal heat generation (hot spots).

[0008] Therefore, even if an unmanned aerial vehicle (drone), an infrared camera, and an analysis device for the thermal images captured by the infrared camera are combined, as in the inspection systems described in Patent Documents 1 and 2, the labor-intensive post-processing that is the most common problem in the field remains unresolved. This post-processing involves speeding up the analysis of the captured thermal images and providing the analysis results to the client who requested the inspection in an easy-to-understand report.

[0009] In actual field work, even if abnormal or deteriorated areas are found on each module of a solar panel, images of those areas must be manually attached to a report, and a specialist or experienced person must provide commentary on the condition of the area, whether countermeasures are necessary, and what those countermeasures should entail. The report created in this way is then sent to the client who requested the inspection. Specifically, when manually interpreting thermal images one by one, it is said that even for experienced individuals, it often takes about 30 minutes to interpret 100 thermal images. In the case of mega solar power plants, where the number of modules that make up the solar panels is large, there is already a shortage of manpower, and there is a possibility of overlooking issues. Furthermore, the considerable time spent on image interpretation and report creation is undesirable from the perspective of inspection costs.

[0010] Although Patent Document 2 mentions automating the creation of inspection reports, it has difficulties in determining what kind of content should be included in the inspection reports, and is therefore not practical.

[0011] This invention has been made in view of the above-mentioned circumstances, and aims to provide an inspection system and inspection method that can collect thermal images of the surface of an object such as a solar panel, remotely inspect the internal operating state or surface state of the object from the heat generation information of the object shown in the thermal images, and automatically generate an inspection report more quickly and with more detailed content.

[0012] To illustrate this objective using solar panels as an example, the goal is to provide a remote, contactless, and labor-saving remote inspection system and method that uses an imaging device mounted on an unmanned aerial vehicle (drone) to photograph the surface of objects such as solar panels and infrastructure from a distance, evaluates the internal and surface conditions of the object based on the image data, and automatically generates an inspection report detailing the evaluation results much faster and more efficiently than conventional methods. [Means for solving the problem]

[0013] To achieve the above objective, according to one aspect of the present invention, A remote inspection system for remotely inspecting photoelectric conversion equipment, which includes a solar panel array in which multiple solar modules, which are the smallest unit of replacement during maintenance and inspection and are responsible for photoelectric conversion, are arranged in an array or string, and a junction box containing connection terminals for connecting the output terminals of the solar panel array to the DC wiring that connects to the power conditioner, A video acquisition means that acquires infrared light video and visible light video by adding the installation position of each of the multiple solar modules to infrared light video and visible light video taken of the light-receiving surface of the solar panel array from a position above and at a distance from the light-receiving surface of the solar panel array, An operating state analysis means for analyzing the operating state of the photoelectric conversion equipment based on the infrared light video and the visible light video, A report generation means for creating an inspection report based on the operating state analyzed by the operating state analysis means, A remote inspection system for photoelectric conversion equipment is provided, characterized by having the following features.

[0014] For example, the report (report) created by the report creation means can be provided to the client upon request. [Brief explanation of the drawing]

[0015] [Figure 1]FIG. 1 is a diagram conceptually explaining a solar module array of a solar power generation system according to the first embodiment. [Figure 2] FIG. 2 is a diagram conceptually explaining a solar module, a drone, and a processing system according to the embodiment. [Figure 3] FIG. 3 is a block diagram showing an overview of a remote inspection system according to the second embodiment. [Figure 4] FIG. 4 is a diagram explaining the arrangement relationship of a solar module array, a solar module, and cells according to the embodiment. [Figure 5] FIG. 5 is a diagram (FIG. 5(A)) explaining the vertical arrangement of solar modules, a diagram (FIG. 5(B)) explaining the flight route of a drone, and a diagram (FIG. 5(C)) explaining the positional relationship between a drone (unmanned aerial vehicle) equipped with a photographing device and a solar panel array. [Figure 6] FIG. 6 is a block diagram showing an outline of the electrical circuit configuration between an inspection management server and an image / report management server. [Figure 7] FIG. 7 is a schematic flowchart exemplifying the processing executed on a user terminal. [Figure 8] FIG. 8 is a schematic flowchart exemplifying the process of flight plan creation executed on an inspection management server. [Figure 9] FIG. 9 is a schematic flowchart exemplifying the flow of abnormality determination by AI and automatic generation of an inspection report executed on an inspection management server. [Figure 10] FIG. 10 is an explanatory diagram exemplifying classification patterns when classifying the operating states (driving states and environmental factors) of solar modules by category. [Figure 11] FIG. 11 is a schematic flowchart exemplifying the processing associated with the reception of an inspection report, etc., executed on an image / report management server. [Figure 12] FIG. 13 is a diagram explaining an example of the screen of a user terminal after the inspection is completed. [Figure 13] FIG. 13 is a diagram explaining an example of the screen of a user terminal after the inspection is completed. [Figure 14]Figure 14 is a schematic diagram of a simplified and modified classification pattern from the classification pattern shown in Figure 10. [Modes for carrying out the invention]

[0016] With reference to the attached drawings, embodiments of the remote inspection system and remote inspection method for photoelectric conversion equipment according to the present invention will be described.

[0017] <First Embodiment> The first embodiment shows an example of a basic configuration in which the remote inspection system for photoelectric conversion equipment according to the present invention is applied to the remote inspection of a solar power generation system.

[0018] Figures 1 and 2 conceptually illustrate the core structure of this remote inspection system.

[0019] As shown in Figures 1 and 2, the remote inspection system 10 is configured to remotely inspect a photoelectric conversion system 18 that includes a photoelectric conversion system 18, which comprises a photoelectric panel array 14 in which multiple photoelectric modules 12 (hereinafter simply referred to as modules), which are the smallest units for replacement during maintenance and inspection and are responsible for photoelectric conversion, are arranged in an array or string shape, and a junction box 16 that incorporates connection terminals (described later) that connect the output terminals (described later) of the photoelectric panel array 14 to a DC wiring that connects to a power conditioner (not shown).

[0020] As shown in Figure 2, this inspection system 10 includes a camera 20 that captures infrared and visible light video of the light-receiving surface 14U of the solar panel array 14 from a position P (at a certain time) in the sky, which is located some distance above the light-receiving surface 14U of the solar panel array 14, and This imaging device 20 captures infrared and visible light videos, to which information on the installation position of each of the multiple solar modules 12 is added. The video acquisition unit 22 acquires these infrared and visible light videos. Based on the infrared light video and the visible light video, the operating state analysis unit 24 analyzes the operating state of the photoelectric conversion equipment 18, The system includes a report generation unit 26 that creates an inspection report based on the operating status analyzed by the operating status analysis unit 24.

[0021] Of the conceptual framework described above, the imaging device 20 is specifically an image acquisition means mounted on an unmanned, autonomously flying drone (unmanned aerial vehicle). The video acquisition unit 22, the operational status analysis unit 24, and the report creation unit 26 are specifically provided as a functional processing system SY performed by a server built on the web through the execution of a predetermined program by a computer system (composed of servers 64 and 66, which will be described later). In particular, the operational status analysis unit 24 may be processed in whole or in part by AI (artificial intelligence). This will be described in the embodiments described later as a processing method suitably applicable to the present invention.

[0022] Unlike conventional inspection systems, this conceptual configuration is characterized by its ability to inspect not only multiple solar modules 12 arranged in an array or string configuration, but also the operating status of the connection boxes 16 installed on the back or side of each module 12 or a predetermined number of modules. Another feature is that the imaging device 20 captures the light-receiving surface 14U of the solar panel array 14 in infrared and visible light video. Furthermore, it is equipped with a report generation unit 26 that automatically creates an inspection report based on the operating status analyzed by the operating status analysis unit 24.

[0023] These characteristics will become clearer through the more detailed explanation provided later.

[0024] This remote inspection system 10 is a system that obtains inspection information indicating the operational status and physical surface condition of a facility by analyzing (including AI (artificial intelligence) analysis) image data (infrared image (thermal image), visible light image) taken by imaging devices 20 such as an infrared camera 32 and a visible light camera 34 attached to the drone 30 (unmanned aerial vehicle) while the drone flies along a pre-set flight path at a short distance above the photoelectric conversion equipment 18 (facility, object). The image data is captured by imaging devices 20 such as an infrared camera 32 and a visible light camera 34 attached to the drone. Since the operator (person) controlling the drone 30 is located in a remote location such as the vicinity of the facility, this image data collection creates an inspection system that collects information remotely.

[0025] While there are various types of facilities that can implement this remote inspection system 10, it is preferable that the imaging device 20 is equipped with at least an infrared camera 32, which can detect the thermal reaction when infrared light emitted from the light-receiving surface 14U of the photoelectric conversion equipment 18 is imaged, and that the information from this thermal reaction reflects the internal and external conditions of the facility.

[0026] As variations in the objects of photography, instead of the photoelectric conversion equipment 18 mentioned above, buildings with a certain degree of covering (houses, factories, commercial buildings), structures without coverings such as infrastructure, towers, and wind power generation equipment, and even places where the shape cannot be defined, such as forests, rivers, and snow-covered mountains, can be used.

[0027] Returning to the embodiment, an example of this photoelectric conversion equipment 18 is a solar panel facility, as will be described later. This solar panel array 14 ranges from relatively small-scale installations on the roofs of houses or buildings to large-scale solar panel facilities, also known as mega solar, installed far out in the field. Solar panels installed on the roofs of houses or buildings are not usually easily inspected visually, so they are subject to inspection by this embodiment. Furthermore, the usefulness of the remote inspection system 10 according to this embodiment is demonstrated even more in large-scale facilities such as mega solar, from the standpoint of addressing labor shortages, inspection accuracy, and inspection costs. This point will be described in detail later.

[0028] On the other hand, considering that thermal images are collected from the drone 30 by the infrared camera 32, it is sufficient that the thermal reaction information reflects some internal information and surface condition information of the object being inspected. Therefore, as exemplified in the modified examples above, it can be applied to systems for searching for missing persons or detecting avalanches in forests and winter mountains, systems for detecting the initial stages of forest fires through regular patrol flights, livestock management systems in pastures, and systems for preventing damage from wild animals.

[0029] Furthermore, its implementation is also effective for buildings and civil engineering facilities collectively referred to as infrastructure. For example, the exterior walls of aging buildings, as well as the exteriors of public buildings, high-rise buildings, road tunnels, road and railway bridges, dams, port facilities, airport facilities, and wind turbine blades, are usually not easily accessible, but require regular inspections. Typically, various inspections are carried out by hand, such as visual inspections, tapping tests, and ultrasonic scans, but all of these require scaffolding construction and manpower before inspection, and even skilled inspectors need a lot of time to prepare reports.

[0030] On the other hand, a drone capable of setting a highly accurate flight path, a recent development, can be used as the unmanned aerial vehicle (UAV) equipped with an infrared camera 32 and a visible light camera 34, which are imaging means. Therefore, the remote inspection system 10 according to this embodiment mounts the infrared camera 32 and the visible light camera 34 on such a drone 30 (UAV) capable of setting a highly accurate flight path to image the exterior of the facility. As a result, if any abnormality (operating state and / or surface state) that differs from the normal state occurs in the thermal image or visible light image, it can be found and reported, for example, by AI analysis. The learning algorithm for this AI analysis may be an algorithm that divides such a target into certain ranges and patterns the appearance of abnormal parts in each range for machine learning, or it may be a deep learning algorithm that does not require training with training data.

[0031] For example, if there is a cavity between a wall and its internal structure (i.e., inside the wall), the air temperature in that area will be different from the surrounding areas without cavities. Also, if a part of the wall is peeling off, the temperature in that area will also be different from the surrounding area. By detecting this temperature difference with an infrared camera, and if necessary, by visually inspecting or analyzing visible light images with AI, it is possible to detect or estimate any abnormalities or anomalies. In this way, even when targeting only infrastructure, the remote inspection system according to this embodiment can be used in a variety of ways.

[0032] While remote inspection systems are expected to have a variety of applications, a representative example of such a system, applied to photoelectric conversion equipment primarily using solar panels as photoelectric conversion elements, will be described in detail below.

[0033] <Second Embodiment> Referring to Figures 3 to 14, a remote inspection system for solar panels according to a second embodiment and its modified form will be described in detail.

[0034] Note that the reference numerals for the components here partially overlap with those in Figures 1 and 2 described above.

[0035] The remote inspection system 10, which inspects the solar panel array 14 (also called a solar power generation panel) shown in Figure 4, comprises an inspection management server 64 built on the internet 62 and an image / report management server 66, which acts as a central server, also built on the internet 62, as shown in Figure 3. Furthermore, this remote inspection system 10 is configured to work in conjunction with a drone 30, an unmanned aerial vehicle equipped with an imaging device 20 (infrared camera 32, visible light camera 34) that captures infrared and visible light images, to inspect the solar panel array 14 remotely and without contact. For this reason, the remote inspection system 10 includes the drone 30.

[0036] Furthermore, in this remote inspection system 10, the image / report management server 66 is configured to be able to communicate with the user's (customer's) PC terminal (user terminal) 68 (68A~68N) via the internet 62, which serves as a communication network.

[0037] From this PC terminal 68 (68A~68N), a request is sent to the image / report management server 66 for inspection of the operating state (thermal state) of the solar panel array 14 or the physical surface state of its light-receiving surface 14U. Upon receiving this inspection request, the image / report management server 66 sends inspection information such as the inspection site and inspection area to the inspection management server 64, and instructs the inspection management server 64 to inspect the operating state and surface state of the solar panel array 14 using the drone 30, under the planned drone flight route, with infrared and visible light. The method of this inspection will be described later, but the image data (inspection data) analyzed by the inspection management server 64 is sent to the image / report management server 66. This image / report management server 66 manages the acquired image data and is configured to create a report showing the inspection results from that image data. This report (inspection report) is sent back to the PC terminal 68A (~68N) in response to the PC terminal 68A (~68N) accessing the image / report management server 66.

[0038] The following describes the detailed connections of the individual components of this remote inspection system 10.

[0039] In this embodiment, the solar panel array 14 to be imaged is the main part of the photoelectric power generation equipment 18 of a solar power generation system 11, also known as a mega solar power plant, and is usually installed in mountainous or plain areas. Of course, the solar panel array to be imaged, i.e., to be inspected, is not necessarily a solar panel array of the scale that can be called a mega solar power plant, but may be a relatively small-scale solar panel array installed on the roof of a private residence, for example.

[0040] The solar power generation system 11 consists of a photoelectric power generation facility 18 installed at a site such as a mountainous area, and the DC power generated by this photoelectric power generation facility 18 is sent via line LN (see Figure 1) to a power conditioner (not shown). This power conditioner converts the DC power to AC power, and this AC power is connected to the commercial power grid via a distribution board, substation equipment, and electricity meters for selling electricity and electricity meters for buying electricity (none of which are shown).

[0041] As shown in a schematic representation in Figure 4, this photoelectric power generation equipment 18 consists of a solar panel array 14 in which multiple solar modules 12 are arranged in an array by a frame 12A, and lines LN are drawn out from the multiple solar modules 12 via junction boxes 16. For example, a junction box 16 is installed in a corner or in the center of the back surface 12R (corresponding to the light-receiving surface 14U) of one solar module 12, for every few solar modules 12. In other words, when viewed from the light-receiving surface 14U, the junction box 16 is usually hidden by the module's appearance and located on the back side.

[0042] The junction box 16 includes a case to prevent wind and rain from entering, and a connection structure inside the case that allows the output terminals on the module side and the input terminals on the line side to be detachably connected.

[0043] In this embodiment, the solar panel array 14 is a facility in which multiple solar modules 12, which are the smallest units for maintenance and replacement, are arranged in an array or string shape, as shown in Figure 4. In other words, in this embodiment, the solar panel array 14 is an assembly of solar modules 12, and refers to a flat plate-shaped portion supported by pillars or frames to receive sunlight.

[0044] Each of the multiple solar modules 12 is typically arranged in a matrix along a certain direction D and in a direction P perpendicular to that direction D, at the most advantageous direction and angle (angle θ with respect to the horizontal plane) for receiving sunlight through the frame 12A and support columns 12P. Each solar module 12 further has multiple cells 12S (the smallest unit of a solar cell) arranged in a matrix within the module, and is typically formed in a rectangular shape as shown in Figure 4. Each cell 12S is a roughly rectangular flat substrate, and a semiconductor layer that generates an electromotive force when it receives sunlight is formed within the substrate, and is configured to perform solar power generation. During maintenance, modules are replaced or repaired, so in this embodiment as well, it is configured to be inspected on a module-by-module basis.

[0045] This solar panel array 14 is subjected to anomaly analysis from various perspectives, based on infrared and visible light images of the top surface (light-receiving surface) 14U of the solar panel, captured from above by an imaging camera mounted on a drone 30. This anomaly analysis includes: • Whether or not each of the cells 12S of the multiple solar modules has a part that indicates a condition that may affect power generation. • Whether or not an abnormal condition (hot spot) affecting the power generation has occurred, and / or, • Is there any damage to the connection structure of the junction box 16, or is it likely to be damaged? This perspective is included. In this embodiment, the term "abnormal state" collectively refers to any state that may progress from a state in which an abnormal state (thermal abnormal state) has occurred to the abnormal state that existed before the occurrence of the abnormal state.

[0046] In addition, in the solar panel array 14 according to this embodiment, if an abnormality is detected in one of the cells 12S, the entire solar module 12 including that cell will be replaced. Depending on the extent of the damage to the connection structure of the junction box 16, the entire junction box 16 may be replaced.

[0047] The flight plan for the aforementioned drone 30 is formulated by the inspector interactively inputting information into the inspection management server 64. Therefore, the drone 30 performs the aforementioned imaging while flying autonomously, for example, according to the flight plan data and GPS data.

[0048] Specifically, the drone 30 is, for example, a multi-rotor helicopter as a small unmanned aerial vehicle, with four rotors (rotating wings) arranged around the body at 90-degree intervals in the circumferential direction, and is configured to be capable of autonomous flight through automatic control according to a pre-set flight plan. Of course, it is also possible to give flight commands via remote control through real-time manual operation. Furthermore, a system may be adopted in which multiple drones 30 are flown in parallel and each is responsible for a different area of ​​coverage.

[0049] The drone 30 itself may be a drone of known configuration, and in addition to a multi-rotor and motor drive mechanism for autonomous flight or remote control, it may also have an electrical system, such as a control unit, positioning unit, imaging unit, data storage unit, and data transmission unit (not shown). In this embodiment, the type of drone 30 is not limited as long as it is configured to fly autonomously according to a pre-set flight plan.

[0050] The control unit controls the flight based on signals from GPS (Global Positioning System) satellites and information from the positioning unit, based on a flight plan input by the operator or calculated based on flight conditions (latitude, longitude, altitude, direction, etc.), and also controls the acquisition of infrared and thermal images by the imaging unit.

[0051] Although not shown in the diagram, this control unit is mainly composed of a computer including a CPU as the central processing unit, and controls the drive unit of the drone 30, calculates the flight path, flight conditions, imaging conditions, etc. based on control programs stored in ROM, RAM, etc. as recording media, and controls the drive of the drone 30 and the operation of the imaging unit.

[0052] The positioning unit receives signals from GPS satellites to measure the current position of the drone 30, so the current position of the drone 30 can be determined by latitude and longitude, and autonomous flight is performed based on this positioning information and flight path information. In addition to GPS, other satellite positioning systems such as GLONASS, Galileo, and Quasi-Zenith Satellite System (QZSS) may also be used.

[0053] Furthermore, as shown in Figures 1 and 2, the imaging unit is equipped with an infrared camera (infrared imaging means) 32 that captures infrared images (thermal images) of the object to be inspected as a video, and a visible light camera 34 that captures visible light images (surface images) of the same object to be inspected as a video. These cameras 32 and 34 are installed on the lower body of the drone 30 so as to have the same imaging field of view. Therefore, the infrared camera 32 and the visible light camera 34 are configured to simultaneously capture thermal and visible light images of the same object, i.e., the same solar panel array 14, as video in response to commands from the control unit while the drone 30 is in flight.

[0054] The video images captured by the infrared camera 32 and the visible light camera 34 are associated with date and time information indicating the date and time of capture, positioning information indicating the location (latitude and longitude) at which they were captured, and height information indicating the height at which they were captured. Therefore, the video images with this associated information are stored on the SD card M, which is detachably inserted into the infrared camera 32 and the visible light camera 34, respectively. SD The images are saved on the MSD (recording medium) according to their respective image types.

[0055] Therefore, as shown in Figure 6, the card reader 64RD reads the video data and acquires the video data with the aforementioned supplementary information attached, categorized by type (infrared video, visible video).

[0056] In other words, after the inspection flight of the drone 30 is completed, the SD card M SD、 M SDBy inserting the card into the card reader 64RD of the inspection management server 64, the system is configured to directly transmit the video data to the storage device (memory) of the inspection management server 64. Of course, this data transmission can also be done via wired or wireless connections through the data transmission unit.

[0057] The infrared camera 32 detects the infrared thermal energy emitted from each solar module 12, that is, from each cell 12S that makes up the module 12, and converts this into temperature to capture an infrared image (thermal image) showing the temperature distribution. In parallel with capturing this infrared image, the visible light camera 34 captures a visible light image by having visible light reflected from the top surface 14U of each solar cell module 12 incident on it. Both of these images are captured as moving images of a predetermined number of frames.

[0058] As shown in one example later, the infrared camera 32 and the visible light camera 34 are typically positioned diagonally above the solar panel array 14, which is installed at an angle to the horizontal plane, and the flight plan is designed so that their imaging range (including field of view) is directed towards the surface of the solar panel array 14, i.e., the light-receiving surface 14U of each solar module 12. Orienting the infrared camera 32 and the visible light camera 34 in the vertical direction of the drone 30, that is, setting the flight path in that manner, is not advisable considering the effects of reflected sunlight and other factors.

[0059] The latitude and longitude information, altitude information, and imaging direction information collected by the positioning unit are added as geotags to the captured infrared and visible light images. These geotags are used for image analysis, including web mapping, by AI (artificial intelligence) running on the inspection management server 64.

[0060] The data transmission unit, if necessary, is equipped with a communication device and transmits the motion image data captured by cameras 32 and 34 to the thermal image memory 64M configured in RAM2:122 of the inspection management server 64.

[0061] <Example of a flight plan> Now, as an example, suppose that multiple solar modules 12 are arranged in a vertical column along a certain direction D on a mountain slope, as shown in Figure 5(A), and that these vertically arranged panel bodies JR1 (~JR4) form a panel array with four columns arranged along an orthogonal direction P. When viewed in plan, as shown in Figure 5(B), an inspection starting point O is set at a predetermined distance from the left end of the first panel set JR12 in a certain direction D (i.e., the vertical direction of the panels) for the first two columns of vertically arranged panel bodies JR1 and JR2. In a plan view, this inspection starting point O is also positioned shifted by a predetermined distance from the midpoint of the two columns of vertically arranged panel bodies JR1 and JR2 in the orthogonal direction P. From there, two columns of vertically arranged panel bodies JR1 and JR2 are photographed at once. Of course, the number of vertically arranged panel bodies may be fractional.

[0062] The position P diagonally above the inspection starting point O (for example, an aerial position a certain distance DS away from one end of the vertical panel units JR1 to JR4 and a certain height HT above it) is set as the shooting start position IS. This shooting start position IS can be calculated by the inspection management server 14 through interactive dialogue with the inspector using the geographically given latitude and longitude position data when the vertical panel units JR1 to JR4 are positioned, along with the aforementioned certain distance DS and certain height HT (see Figure 5(C)). This calculation is performed by the inspection management server 64, which will be described later, along with the planning of the flight path FR. In other words, by setting this certain distance DS and certain height HT, the infrared camera 32 and visible light camera 34 mounted on the drone 30 can be positioned from an even more diagonally above the solar panel array 14, which is positioned diagonally to the horizontal plane.

[0063] These fixed distance DS and fixed height HT can be adjusted as needed depending on the geographical conditions of the panel installation and the weather conditions at the time. For example, the fixed distance DS can be adjusted within a range of 1 to 2 meters, and the fixed height HT within a range of 20 to 30 meters. Therefore, even though it is described as shooting from an oblique angle above, depending on the setting of the fixed height HT, it can be said that it is shooting almost directly from above, but these can be changed according to various conditions (geographical conditions, weather conditions, inspection time, etc.).

[0064] Of course, the infrared camera 32 and the visible light camera 34 are configured to be tiltable within a certain angular range. Therefore, under normal circumstances, the shooting direction of the cameras 32 and 34 is set to be perpendicular to the upper surface (light-receiving surface) 14U of the solar panel array 14. However, since the above tilt adjustment is possible, it does not have to be perfectly perpendicular; it is sufficient to have an angular range that is close to perpendicular by using the tilt function.

[0065] The flight path FR is set to move linearly from the shooting start position IS in the inspection flight direction D, while both cameras 32 and 34 capture moving images of the first two rows of vertical panel bodies JR1 and JR2. Once the other end of the first two rows of vertical panel bodies JR1 and JR2 is reached and the predetermined turning point RT is reached, the flight path FR is set to shift in the orthogonal direction P, and return to the shooting start side while similarly capturing the next two rows of vertical panel bodies JR3 and JR4. Thereafter, it follows a meandering path in a zigzag pattern in the lateral direction of the vertical panel bodies. Of course, if the vertical panel bodies are shifted in the orthogonal direction P along the way, the flight path FR is also shifted according to that shift. Furthermore, when there is only one vertical panel body remaining, the flight path FR is set to move along the inspection flight direction D at a position diagonally above that single panel body. The flight speed is set to approximately 2 m / s, depending on weather conditions as well as the shooting capabilities of cameras 32 and 34.

[0066] As described above, the drone 30 flies diagonally above the solar panel array 14 along the inspection flight direction D, while the infrared camera 32 and visible light camera 34 mounted on the drone 30 capture video of two rows of vertical panel units JR1, JR2 (JR3, JR4). This reduces the shadow of the drone 30 from being cast on the light-receiving surface 14U of the solar panel array 14, i.e., the light-receiving surface 12U of each solar module 12, and reduces the influence of sunlight reflection from the light-receiving surface 14U, allowing for the capture of less noisy and more accurate video images. Furthermore, by flying horizontally, i.e., along the longitudinal direction (inspection flight direction) D of the vertical panel units JR1, JR2, a stable flight path FR can be set that does not require much consideration of the height difference of the installation surface of the solar panel array 14 (i.e., the height difference between the solar modules 12). This contributes to high-precision shooting.

[0067] While the drone 30 is flying along the flight path FR, the video images captured by the infrared camera 32 and the visible light camera 34 are stored on the SD card M SD It is saved to the SD card M. Therefore, once the inspection flight of drone 30 is complete, the SD card M SD The card is inserted into the card reader 64RD of the inspection management server 64, and the video data is read into the image memory 64M of the inspection management server 64, which will be described later.

[0068] (Inspection management server and image / report management server) Next, the configurations of the inspection management server 64 and the image / report management server 66 will be described with reference to Figure 6.

[0069] In this remote inspection system 10, the inspection management server 64 and the image / report management server 66 described above are connected to each other via a wired or wireless communication network 62, such as the Internet. Furthermore, one or more user terminals 68A (~68N) can connect to the image / report management server 66 via the communication network 62. User terminals 68A (~68N) are terminal devices of users (our company) participating in the remote inspection system 10, and are computer devices such as PCs.

[0070] As shown in Figure 6, the inspection management server 64 includes a processor 110 that functions as a computer device, and under the control and management of this processor 110 are an AI analysis device 112, a database (DB) 114, the aforementioned card reader 64RD, and a transceiver unit 106.

[0071] The processor 110 has an input / output interface 112, and a CPU 116, a ROM 118, a first RAM 1 120, and a second RAM 2 122 are connected to this input / output interface 112 via a bus 114 so as to be able to communicate with each other. Of these, the second RAM 2 122 is configured with a memory area that constitutes the image memory 64M mentioned above.

[0072] This processor 110 is the central hub for managing the entire inspection process, which involves processing infrared and visible light video images taken from above and at an oblique angle by a drone 30 to determine whether or not there are any abnormal parts in the solar modules 12 due to malfunctions or other reasons, using AI. A predetermined processing computer program for this management is pre-stored in ROM 118. Therefore, when the CPU 116 starts up, it is configured to call the processing program from ROM 118 to its work area and sequentially execute the processing steps described in the processing program. Temporary data associated with this processing is written to the first RAM 1:120 in a readable format for temporary data storage.

[0073] Also, as described above, the second RAM 2:122 is a recording medium that provides a memory area including an image memory of 64M. The moving image data of the infrared moving image and the visible light moving image captured by the infrared camera 32 and the visible light camera 34 mounted on the drone 30 are inserted into the SD card M SD recorded in. Therefore, when the drone 30 finishes the inspection flight and lands, the SD card M SD is manually removed from the image recorder, and the SD card M SD is inserted into the card reader 64RD for image transfer. Instead of this manual operation, it may be configured to transfer images via wireless or wired communication. Thus, under the management of the CPU 116, the moving image data of the infrared moving image and the visible light moving image captured by the cameras 32 and 34 mounted on the drone 30 are read into the image memory 64M (second RAM 2:122) of the processor 110 via the SD card M SD .

[0074] Also, the processor 110 is a means for executing an AI analysis application separately from the separate AI analyzer 112, and its CPU 116 as an arithmetic unit executes a program including a given AI analysis process. The outline of this processing program is shown in FIG. 7. As will be described later, this processing program includes reading of moving images, preprocessing of moving images, analysis and determination of abnormal parts of the solar panel array 14 by AI, creation of inspection reports, and transmission of inspection reports and image data to the image-report management server 66. These processes will be detailed again in the overall flow.

[0075] Returning to the overall system shown in Figure 7, the image / report management server 66 receives inspection reports and various types of image data (including still images and moving images before preprocessing) used in the inspection from the inspection management server 64 via the communication network 62. This image / report management server 66 stores and manages inspection reports based on the AI ​​analysis results of the solar panel array 14 and the image data that formed the basis of the inspection, and also builds a platform (PF) on the cloud for users to view them with user terminals 68A (~68N) (see Figure 4). Therefore, users can request inspections (hereinafter simply referred to as inspections) of the operating state (thermal state) of the solar panel array 14 in the solar power generation system 11 or the physical surface state of its light-receiving surface 14U from their terminals 68A (~68N).

[0076] Specifically, the image and report management server 66, as shown in Figure 7, includes a processor 210 that functions as a computer device, and a database (DB) 212 and image storage 214 under the control and management of this processor 210. The database 212 mainly stores inspection requests, inspection reports, etc., sent from user terminals 68A (~68N) in a read / write format under the control of the processor 66.

[0077] Furthermore, the database 212 contains a memory area for a correspondence table 212T, which associates flight position information such as latitude, longitude, and altitude of the drone's flight path with the map location information where the solar panel array 14 is actually installed, for the purpose of setting the flight path of the drone 30 (described later) and web mapping. This correspondence table 212T is created and stored in advance for each installation location of the solar panel array 14. Therefore, when an inspection request is received from the user terminal 68, the processor 210 sends the correspondence data stored in the correspondence table 212T related to that inspection request, along with the request information, from the image / report management server 66 to the inspection management server 64.

[0078] The image storage 214 stores image data (including raw still images and moving images before preprocessing) sent from the inspection management server 64 via the communication network 62 in a read / write format. By displaying and observing this raw image data on a monitor on the user's side, useful information about the panel and surrounding environment can sometimes be obtained.

[0079] The processor 210 includes an input / output interface 222, and a CPU 226, a ROM 228, and a RAM 230 connected to this input / output interface 222 via a bus 224 for mutual communication. Of these, the ROM 228 pre-stores a program for image and report management performed by the processor 210 in a callable format. Therefore, the CPU 226 reads the program for user management, inspection request reception and registration, and image and report management from the ROM 228 into its work area and sequentially executes the steps described in the program. The RAM 230 has the function of temporarily storing data from the calculation process.

[0080] As an example, this remote inspection system 10 is operated on a membership basis, so users, i.e., users of this remote inspection system 10, pre-register necessary information with the image / report management server 66 via the user terminal 68A (~68N). In other words, users are assigned a login code and password in advance, and various information about the solar power generation system 11 specified by the user (system address, environmental conditions, number of modules making up the solar panel array 14, the arrangement and location of each module (latitude and longitude information), and surrounding map information, etc.) is registered in advance at the time of membership contract or before an inspection request is made. This information is stored in the database 212 in an updatable manner.

[0081] <Processing flow> I) Receipt of inspection requests and inspection reports As described above, when a user accesses the image / report management server 66 from a user terminal 68 (68A~68N) to request an inspection, the communication shown in Figure 7 takes place via the communication network 62.

[0082] When a user logs into the remote inspection system 10 from the user terminal 68 (Figure 7: Step S2), the processor 210 (i.e., CPU 226) displays a user-specific inspection list screen on the display screen of the user terminal 68 (Step S4). Then, the processor 210 interactively determines whether or not the user has made a new inspection request (Step S6), and if there is a new request, it accepts the information, namely the registration ID that specifies the solar power generation system 11 to be inspected, which has been registered in advance, via screen input on the user terminal 68 (Step S8). The processor 210 then transmits this registration ID and the corresponding information stored in the aforementioned correspondence table 212T to the inspection management server 64 via the input / output interface 222 and the communication network 102 (Step S10).

[0083] As a result, as described later, the inspection management server 64 reads various information about the solar power generation system 11 linked to its registration ID (including the system's address, environmental conditions, the number of modules making up the solar panel array 14, the location of each module (latitude and longitude information), and the aforementioned correspondence information), creates a flight plan including the date and time of flight, flight path, and work time, taking weather forecasts into consideration, and transmits it to the worker or to the drone via communication as flight control information. At this time, the flight path plan refers to the aforementioned correspondence information (i.e., information that correlates the flight position information such as the latitude, longitude, and altitude of the drone's flight path with the location information on the map where the solar panel array 14 is actually installed), so that more accurate positional information of the flight path can be reflected in the flight control information, specifically the absolute values ​​of latitude, longitude, and altitude. This leads to more accurate setting of the flight path, and consequently, more accurate imaging of thermal and visible light images, enabling more reliable inspections. On the other hand, when the decision to reject the request is NO in step S6, the processor 210 determines that it is not a request for a new inspection, and further determines in the next step whether it has already received, i.e., whether it has received the inspection report after the inspection is completed (step S12). If, as a result of this determination, an inspection report after the inspection is completed is available, the user terminal 68 is instructed to download the inspection report and raw image data (step S14: constitutes the report request means). This allows the user to have the inspection results, i.e., the inspection report showing the operating status of the solar panel array 14 (solar power generation panel) that was requested to be inspected, displayed on the user terminal 68 (step S16: analysis result display means), and to have it recorded and stored on the terminal's recording device (step S18). Steps S12 and S14 functionally constitute the transmission and reception means. In this case, if necessary, infrared image data and visible light image data of the inspected solar panel array 14 can also be displayed and observed on the screen visually.

[0084] II) Flight planning and inspection When the inspection management server 64 receives an inspection ID related to an inspection request as a request from the image / report management server 66 (Figure 8, step S32), the processor 110, i.e., the CPU 116, reads the inspection information of the solar panel array 14 of the solar power generation system 11 associated with that inspection ID from the database 114 (step S34). This inspection information includes the number of one or more solar modules 12 (simply modules) arranged in an array or string shape, which are the photoelectric conversion elements of the solar panel array 14 installed at the actual site of the solar power generation system 11, the location of each module (latitude and longitude information), the address of the solar panel array 14, and environmental conditions.

[0085] The processor 110 then creates a flight plan for the drone 30 based on the inspection information (step S36). The worker provides the drone 30 with flight control information in accordance with the flight plan, and the drone 30 autonomously flies over the solar panel array 14 site. As a result, the infrared camera 32 and visible light camera 34 mounted on the drone 30 capture video of the solar panel array 14 from an oblique angle above. This video data is stored on SD cards inserted into image recorders 31 mounted on cameras 32 and 34, respectively. SD It will be recorded.

[0086] III) Image analysis and preparation of examination reports Once the on-site photography by the drone 30, specifically infrared and visible light photography (both in color) of the solar panel array 14 being inspected, is complete, the images are transferred from the image recorders of the infrared camera 32 and the visible light camera 34 to an SD card M SD Each of these will be collected. These SD cards M SD The infrared video data and visible light video data stored in the system are read by the card reader 64RD, respectively.

[0087] The read video data is stored in a designated storage area of ​​the database 114, according to its type.

[0088] Therefore, once the image reading and storage described above is completed in the inspection management server 64, the processor 110 performs image analysis, that is, analysis of the operating status of each solar module 12 (the driving status of the cell itself (thermal status including thermal abnormalities) and the physical surface status of the cell light-receiving surface 14U), mainly using infrared moving images (thermal images). This will be described in detail below.

[0089] In the inspection management server 64, the processor 110 inputs infrared video data already read and stored from the database 114 into its work area (Figure 9, step S62). Next, the processor 110 divides the infrared video data (data format is, for example, mp4) into N (N≧1 positive integer) infrared still images (data format is Jpeg or TIFF, etc.) in which one or more solar modules 12 are captured (step S64: forming an image division means). Next, the processor 110 applies various known correction processes, such as angle correction, to the divided still images (step S66). Next, the processor 110 converts the divided color images into grayscale images (step S67). Converting to grayscale distribution images reduces identification noise that originates from color, unlike the distribution method for color images.

[0090] Furthermore, the processor 110 applies AI processing to the corrected infrared image to individually isolate the solar modules 12, which are the units to be replaced during maintenance (step S68). For this AI processing, for example, Mask R-CNN, a deep learning method for object detection and instance segmentation, is used. Alternatively, the YOLO (You Only Look Once) method, especially YOLOv11 which offers a good balance between real-time performance and accuracy, or ViTDet (Vision Transformer Detector), which enables highly accurate detection, may be used.

[0091] The infrared images of each solar module 12 detected in this manner thermally reflect the operating state (driving state (thermal state including thermal abnormalities) and the physical representation of the cell light-receiving surface 14U) of the entire module, individual cells 12S, and junction box 16. In other words, by imaging each solar module 12 with infrared images and comparing them with surrounding modules, thermally relative differences can be detected. These differences are expressed at the level of the multiple cells 12S that make up each solar module 12. These differences are caused by the normal or abnormal driving state (thermal state) of each cell 12S and junction box 16, as well as by environmental factors such as the accumulation of leaves and dust on the cells or the presence of bird droppings. Therefore, when classifying these operating states (driving state and environmental factors) into categories, the following classification patterns can be given as an example.

[0092] • N: Normal operating condition (no thermal or environmental abnormalities) • H: Anomalies in parts of a module's cells (including junction boxes), including hotspots (thermal anomalies), shadows caused by vegetation or trees, and soil accumulation. • M: High-temperature abnormalities throughout the module, including cracks on the module surface. • High temperature abnormalities in the cells that make up column C ·S: Influence of obstacles, etc. on surface condition

[0093] These categories N, H, M, C, and S can be schematically represented using a single solar module 12 as shown in Figure 10. This photographic diagram in Figure 10 is taken from a Kaggle dataset as an example of a classification pattern. Of course, this classification pattern is not the only one that can be used; a simplified classification pattern, such as the modified version described later (Figure 14), may also be used.

[0094] In this embodiment, although not visible in the photographic image of Figure 10, it has been confirmed that the shadow of the junction box 16, which is located on the back side of the solar panel array 14, is also captured in the infrared image. Therefore, abnormalities in the junction box 16 (such as disconnection or detachment) are also reflected in the categorization of the operating status described above. For example, if the connection to the junction box 16 is disconnected, the effect of that disconnection will be reflected even if the entire module is operating normally.

[0095] The processor 110 then sequentially inputs the grayscale infrared images extracted from the aforementioned solar module 12 into the AI ​​analysis device 112, and performs deep learning using an AI model that constitutes a multi-layer neural network to analyze the thermal operating state of the solar module 12 based on the classification patterns described above. From the results of this analysis, the processor obtains an inference value indicating whether the solar module 12 is abnormal and should be replaced or repaired (if there is no abnormality, it is judged to be normal and does not require replacement or repair), and which of the aforementioned categories is the cause of the abnormality (Figure 10, step S70: constitutes the abnormality determination means). The algorithm used for this AI model (machine learning model) is classification. Specific architectures for classification include AlexNet, VGG, GoogleNet, ResNet, DenseNet, MobileNet, EfficientNet, and ViT. These architectures employ a process called a "convolutional layer" to extract features from an image and a process called a "pooling layer" to reduce the image size while retaining features. An AI model is used that has been trained using classification patterns of modules that mimic the aforementioned anomaly categories as training data. In this embodiment, a dedicated AI analysis device 112, separate from the processor 110, is provided for the detection and determination of anomalies by category using this AI analysis, from the viewpoint of load balancing. Of course, the processor 110 may also be configured to perform such AI analysis.

[0096] Furthermore, the AI ​​analysis processing of the AI ​​analysis device 112 may employ a processing configuration in which AI models are connected in a column, first estimating the presence or absence of a module abnormality, and based on that result, estimating what classification pattern the module abnormality is due to, the content of the abnormality, and / or, further estimating the need for maintenance and replacement for each module based on the estimation results.

[0097] Next, the processor 110, based on the determination results made by the AI ​​analysis device 112, performs a process to overlay a mark indicating the module's location, its location data (coordinate data), relative temperature, and a symbol (by category) indicating the abnormality determination result onto the visible light color image data (infrared light color image or black and white image data may also be used) of the solar modules 12 that have been determined to have an abnormality corresponding to one of the abnormality categories described above (step S72: forming a marking means). This overlaid information is conceptually represented as a marking M in Figure 12. Of course, if the user clicks on the marking M, the module's location (latitude and longitude information, as well as location information at the installation site), relative temperature, and abnormality determination result may be overlaid sequentially or in a single pop-up. At this time, the estimated temperature information may be displayed in a different color from the other marking information.

[0098] Next, the processor 110 refers to the aforementioned correspondence information (i.e., information that associates flight position information such as the latitude, longitude, and altitude of the drone's flight path with the map location information (latitude and longitude information) where the solar panel array 14 is actually installed) to identify the module location information (latitude and longitude information) attached to each module data, and pastes the aforementioned superimposed module image data (preferably a visible light color image) onto the identified actual location on the map (so-called web mapping) (step S74: forming the superposition means). Steps S70 to S76 form the abnormal / normal display means.

[0099] This allows the solar panel array 14 to be represented as if it were located on a realistic map of its surroundings. Visualizing this representation enhances the sense of realism in the inspection results, and it is expected to improve the understanding of abnormalities by allowing for consideration of the surrounding environment, such as the presence of many shaded areas due to rapid tree growth. Furthermore, since the aforementioned correspondence information is used in the web mapping, it is possible to obtain more accurate and reliable report generation materials.

[0100] An example of this web mapping is schematically shown in Figure 12. As shown in the figure, visible light color images of multiple solar panel arrays 14 are mapped onto an actual map, with abnormal areas marked for each module (see symbol MG). In Figure 12, symbol HL schematically represents a hilly area on the map, DR schematically represents a road, and FD schematically represents the field (installation site) where multiple solar panel arrays 14 are installed. Therefore, it is easy to understand, for example, the location of modules with abnormal heat generation (hot spots) or modules with accumulated dust on the surface that are excessively blocking sunlight, within the overall field, and it is possible to determine the order of access during maintenance and replacement efficiently.

[0101] The marking M can, of course, be superimposed with a black and white symbol. In response, the processor 110, which has AI analysis application functionality, creates an inspection port (step S76). This inspection report summarizes the following as tabular data: user name, inspection ID, inspection date and time, inspection location, number of inspections, weather conditions, abnormality judgment for each module (modules other than those judged as abnormal are judged as normal), cause of abnormality (category notation indicating the type of abnormality), and notes from the inspector. Of course, statistical and aggregated data on the cause of abnormality and the abnormality occurrence rate may also be added.

[0102] This inspection report is transmitted from the inspection management server 64 to the aforementioned image and report management server 66 via the communication network 62, along with the web-mapped module image data and the raw infrared and visible light video captured by the drone 30 (step S78).

[0103] Furthermore, by having the processor 110 execute the process shown in Figure 9, step S62 functionally constitutes a reading means, steps S64 to S68 functionally constitute a preprocessing means, step S70 functionally constitutes an AI analysis and judgment means, and steps S72 to S76 functionally constitute a report generation means. In addition, each step in Figure 9 functionally corresponds to the aforementioned video acquisition unit (means) 22 for step S62, to the operating state analysis unit (means) 24 for steps S64 to S74, and to the report creation unit (means) 26 for steps S72 to S76. For this reason, the processor 212 of the image / report management server 66 receives the inspection reports, web-mapped module image data, and raw infrared and visible light videos (Figure 11, step S82), adds them to the inspection list (step S84), and saves them to the database 212 (step S86). Furthermore, the processor 212 notifies the user who requested the inspection (user terminal 68) that the inspection has been completed (step S88).

[0104] As a result, the user can access the image / report management server 66 from their user terminal 68, download the inspection report, web-mapped module image data, and raw infrared and visible light video (Figure 8, step S14), view and confirm them (step S16), and record and save them to their own recording device (step S18).

[0105] Figure 13 shows an example screen of user terminal 68. For example, when a user (Ikebukuro Co., Ltd.) logs in after receiving a notification that the inspection report has been completed, they can see a thumbnail of the inspection location, the date and time of inspection, the date and time of report registration upon completion of the inspection, the inspection results (report), a download icon, and inspection comments. Clicking the download icon allows the user to download the inspection report as an inspection result from the image / report management server 66. In addition to information such as the date and time of inspection and the inspection location, the inspection information for all modules (modules with thermal abnormalities, module marking images, classification of abnormalities) and web mapping images are displayed. In addition, the user can also view the raw visible light images.

[0106] In this way, users can obtain not only inspection reports but also web-mapped module image data and raw infrared and visible light videos as supplementary information to the inspection results. Therefore, they can understand the operating status of the solar panel array 14 (driving status (thermal status including thermal abnormalities) and the physical representation of the cell light-receiving surface 14U) from the contents of the inspection report and appropriately determine the timing of partial maintenance, repairs, or module replacements in order to improve operating efficiency. At the same time, they can also understand the conditions of the surrounding environment of the solar panel array 14 from the web-mapped images and videos. In particular, the actual abnormal state of modules reported as having abnormalities in the inspection report can be complementarily understood from the visible light videos.

[0107] Based on the above, the drone can be made to fly autonomously, and an imaging device mounted on the drone can capture video of the solar panel array and the connection box located on the back of it, using infrared and visible light from a position away from the array. Based on this video data, the internal and surface conditions of these panels can be subjected to AI analysis based on image patterns (anomaly classification patterns) that classify the cause of the anomaly according to category. Since these image patterns are mostly reflected as combinations of rectangular cell units in response to infrared light, even if there is an accumulation of dust on part of the cell surface, it is easy to consolidate them into multiple image patterns based on the cell shape.

[0108] Therefore, by training the AI ​​model with multiple image patterns, it can classify the diverse thermal distribution patterns given as inspection images into a predetermined set of multiple image patterns (anomalous classification patterns). Modules that do not have a thermal distribution pattern that does not belong to this image pattern are evaluated as normal. For this reason, the multiple image patterns can be set according to the predetermined number and distribution state of cells necessary for maintenance and repair such as module replacement.

[0109] For example, if a thermal anomaly (relatively high temperature area) is detected in only one cell within a solar module 12, and it is determined that the module as a whole does not yet need to be replaced, it does not need to be included in the multiple image patterns used for training described above. Similarly, if a thermal anomaly is detected in fewer than a predetermined number of cells distributed within a single solar module 12, it does not need to be included in the multiple image patterns used for training described above. However, if the number of cells estimated to be exhibiting the thermal anomaly exceeds that predetermined number, the system can be configured to conclude that the module's power generation efficiency falls below the acceptable value and therefore needs to be replaced. This predetermined number can be freely determined, for example, 4 or 5 cells.

[0110] On the other hand, in the case of an image pattern in which all cells in a vertical or horizontal row within a single solar module 12 are exhibiting thermal abnormalities, even if the number of affected cells is less than a predetermined value, the AI ​​may determine that it is only a matter of time before the abnormality in the vertical or horizontal cells spreads horizontally or vertically. In this case, an image pattern (thermal image pattern) in which all cells in a vertical or horizontal row mimic thermal abnormalities should be included in the multiple image patterns used for training.

[0111] From this perspective, the classification patterns for anomaly categories shown in Figure 10 may be simplified as shown in Figure 14. In other words, schematically represented as shown in Figure 14, AI learning may be performed using four classification patterns: overall anomaly (Figure (A)), anomaly affecting the entire column (or row) (Figure (B)), localized anomaly (Figure (C): one or more localized anomalies), and no anomaly (Figure (D)). This reduces the number of classification patterns and thus the computational load on the AI. At the same time, it is possible to provide simpler information regarding maintenance management methods. For example, in the case of Figure (A) and (B), information such as replacing the module can be provided, while in the case of Figure (C), information such as "requires further observation" can be provided.

[0112] Of course, the classification patterns for abnormality categories are not limited to the photographs and schematic diagrams shown in Figures 10 and 14, and the number of classification patterns can be increased. For example, the classification patterns for cell 12S and junction box 16 can be separated and fed to an AI learning model to train the model. This allows for a more accurate estimation of thermal abnormalities associated with damage or disconnection of the junction box 16. In particular, since the junction box 16 is often located on the back side of the cell 12S group, even if a thermal abnormality occurs in the junction box 16, it is often difficult to detect visually or even with AI analysis in infrared images. However, by separating the classification patterns, it becomes possible to improve the accuracy of interpretation.

[0113] Thus, because this is an AI analysis of thermal image patterns derived from cell-level combinations, it offers higher analysis speed and accuracy compared to conventional methods that simply perform AI analysis on a pixel-by-pixel basis.

[0114] Of course, even when considering thermal image patterns derived from cell-level combinations, there are various factors involved, including those purely due to electrical failures in cells, those due to environmental factors such as dust, fallen leaves, and tree shadows, combinations of these factors, weather conditions during inspection, reflection and scattering of sunlight, and heat transfer to adjacent cells. In addition, differences in ambient temperature also contribute to differences in the degree of heat generation during normal operation.

[0115] Therefore, as can be seen from Figure 10, thermal image patterns are not visualized as combinations of cells with clear rectangular outlines. Rather, they are often visualized as thermal patterns with diverse shapes of high-temperature areas, such as blurred outlines where the spaces between surrounding cells are not clearly visible, or ambiguous spots indicating high temperatures in neighboring cells. For this reason, while it is not absolutely impossible for an examiner to visually interpret these thermal images, even a skilled interpreter would find it extremely time-consuming and the estimation accuracy extremely low. For this reason, there is great value in using AI analysis to evaluate abnormal distributions by training an AI model with module images that show a diverse distribution pattern based on cell shape, which represents a relative temperature difference. This AI analysis improves estimation accuracy and significantly reduces estimation time.

[0116] As a result, inspection reports containing the evaluation results can be generated and presented much faster and automatically than before. Therefore, a remote, contactless, and labor-saving remote inspection system and method can be provided.

[0117] In particular, in AI analysis, solar modules are automatically extracted from video footage using an AI-based object detection method. Furthermore, thermal abnormalities corresponding to the cause of the abnormality are pre-classified into categories, and an AI learning model trained on these abnormality patterns is used to automatically determine the presence or absence of an abnormality and the type of abnormality.

[0118] Because this module extraction and anomaly detection can be provided as a series of automated, coordinated processes based on AI learning, it differs from conventional methods where images are manually inspected, pasted, and observation results are recorded. This allows for significantly faster and more accurate anomaly detection with fewer oversights. As a result, the effort and time required to create inspection reports for users who have requested inspections can be greatly reduced, thereby improving the throughput of inspection report creation.

[0119] According to the applicant's assessment, the time required to review 100 images for a conventional manual inspection report, even for a skilled radiologist, was approximately 30 minutes. However, with the inspection report creation method according to the present invention, this time can be reduced to approximately 1 to 1 few minutes.

[0120] Furthermore, compared to visual analysis, it reduces the likelihood of oversights and errors, and allows specialized personnel to be freed up for other tasks, thus reducing the overall cost of the inspection.

[0121] In addition, as mentioned above, web mapping images and infrared and visible light video are also provided. Users can use these image data to complement the inspection report, easily obtain information about the surrounding environment that is difficult to understand from the inspection report alone, and use this information to help set maintenance management and replacement timings for the faulty modules indicated in the inspection report.

[0122] In this way, the AI ​​image analysis system extracts images that are highly likely to contain abnormalities, enabling a dramatic reduction in working time and costs. This inspection system is therefore not limited to solar panels, but can also be applied to non-contact inspection systems for other buildings and infrastructure. [Explanation of symbols]

[0123] 10 Remote inspection systems for solar panels (also known as photovoltaic panels) 11. Solar power generation system 12. Solar modules (or simply modules) 12S Cell 14 Solar panel arrays 14U light receiving surface 16 junction box 18. Photoelectric conversion equipment 20 Imaging device 22. Video acquisition unit (forms the means for acquiring video) 24. Operating state analysis unit (forming an operating state analysis means) 26. Report Creation Section (Constitutes the means of creating reports) 30. Drones (Unmanned Aerial Vehicles) 30 Image Recorder 31 Image Recorder 32. Infrared camera (infrared imaging means) 34 Visible light camera M SD A removable SD card (which forms the memory) is part of the recording medium. 62. Communication Network (Internet) 64 Inspection Management Server 64RD Card Reader (acts as a reader) 66 Image / Report Management Server 68 (68A~68N) User (Customer) Terminal 110 processors 112 AI analysis device 114 Databases 210 processors 212 Databases (Test Result Databases) 214 Image Storage

Claims

1. A solar panel inspection system for remotely inspecting photoelectric conversion equipment, which includes a solar panel array in which multiple modules, which are the smallest units for maintenance and replacement and are responsible for photoelectric conversion, are arranged in an array or string, and a junction box that incorporates connection terminals for connecting the output terminals of the solar panel array to the DC wiring that connects to the power conditioner, A video acquisition means that acquires infrared light video and visible light video by assigning the installation positions of each of the multiple modules to infrared light video and visible light video taken of the light-receiving surface of the solar panel array from a position in the sky at a distance from the light-receiving surface of the solar panel array, An operating state analysis means that performs AI analysis of the operating state of the photoelectric conversion equipment based on the infrared light video and the visible light video, A report generation means for creating an inspection report based on the operating state analyzed by the operating state analysis means, A solar panel inspection system for photoelectric conversion equipment, characterized by having the following features.

2. The solar panel inspection system according to claim 1, characterized in that the video acquisition means comprises an infrared camera and a visible light camera mounted on an unmanned aerial vehicle and capable of capturing infrared video and visible light video, respectively, and an image division means for dividing the captured infrared video and visible light video into a plurality of still images, respectively.

3. The infrared camera and the visible light camera have a memory that records the infrared video and the visible light video data in a predetermined file format during the flight of the unmanned aircraft. The system includes an image transmission means for transmitting the captured infrared video and the visible video to the image division means, The video transmission means includes a reader that reads the infrared video and visible video data recorded in the memory. The solar panel inspection system according to feature 2.

4. The solar panel inspection system according to claim 3, wherein the image division means is configured to divide the infrared light video and the visible light video data recorded in the predetermined file format into a plurality of still images of a predetermined pixel size.

5. The solar panel inspection system according to claim 1, wherein the operating state is at least one of the following: an abnormal state in which there is an abnormality in the driving of the plurality of modules; the environmental state of the light-receiving surface which may affect the output value due to the photoelectric conversion of the plurality of modules; and a damaged state of the junction box.

6. The solar panel inspection system according to claim 5, wherein the operating state is an abnormal state in which an abnormality occurs in the driving of the plurality of modules, and the environmental state of the light-receiving surface which may affect the output value due to the photoelectric conversion of the plurality of modules.

7. The aforementioned operating state analysis means is A database storing reference data indicating the heat distribution state for each module, which is predetermined for the abnormal and normal conditions, in each of the multiple still images of infrared light divided by the image division means, An anomaly determination means that, by referring to the heat distribution state stored in the database, determines for each module whether or not there is a module in each of the multiple infrared light still images that contains at least some of the cells exhibiting the anomaly, The marking means includes a marking mechanism for identifying and marking the location of each module in the overall solar panel for each module in which the abnormality determination means has determined that at least a part of the abnormality is occurring, A solar panel inspection system according to any one of claims 1 to 6, characterized by comprising the above.

8. The solar panel inspection system according to claim 7, wherein the marking means is configured to mark the entire module at the identified location with estimated temperature information in a different color from other normal modules.

9. The database stores multiple patterns of the heat distribution state for each factor, including a predetermined pattern indicating the aforementioned anomaly. The abnormality determination means is configured to identify the module having the abnormality by referring the heat distribution state of each module to be inspected to the plurality of patterns. The solar panel inspection system according to feature 7.

10. The solar panel inspection system according to claim 9, characterized in that the plurality of patterns include patterns indicating that a part of the module is at a high temperature or generating heat at a high temperature, patterns indicating an anomaly in the module that the entire module is at a high temperature, and patterns indicating that the entire array of solar panels arranged in a string is at a high temperature.

11. The solar panel inspection system according to any one of claims 1 to 10, characterized in that the panel condition analysis means is configured to perform the analysis using AI (artificial intelligence).

12. The aforementioned report generation means is A superimposing means superimposes a top view of the solar panel array, which is modeled after the solar panel array, onto a realistic map where the solar panel array exists, according to the shooting position. On the superimposed upper surface, an abnormality / normality display means is provided to display the analysis results, enabling the abnormality and normality of the plurality of modules on a module-by-module basis. A solar panel inspection system according to any one of claims 1 to 11, characterized by comprising the above.

13. The solar panel inspection system according to any one of claims 1 to 12 is configured as a system including a server built on the Internet by the operator who operates the solar panel inspection system, This system has a terminal that is used by a user and is connected to the server via the internet, The aforementioned terminal is A reporting request means by which the user specifies the user ID and requests the server to report the analysis results indicated by the reporting means via the Internet, When the analysis results are transmitted from the server via the Internet, the analysis results are displayed on the screen of the display device. Equipped with, The aforementioned server, An inspection result database that stores information including at least a user ID assigned to each user, the inspection date of the solar panel array and the connection box for cable connections connected to the solar panel array, performed in response to the use of the solar panel inspection system requested by each user, and the analysis results indicated by the analysis means, The inspection result database includes a transmission / reception means for sending and receiving the information to and from the terminal via the Internet, Equipped with, The aforementioned transmitting and receiving means is The terminal receives the request from the reporting request means via the Internet. When the aforementioned request is received, the system searches the inspection result database according to the user ID and sends back information via the internet to the terminal indicating a list screen that displays a complete list of all the analysis results in response to all the aforementioned requests for that user ID. In response to the aforementioned return, the terminal accepts the specification of a specific analysis result via the Internet. This designation causes the terminal to download the specified analysis result from the inspection result database when that specific analysis result is specified. A solar panel inspection system characterized by being configured in such a way.

14. In a solar panel inspection method that inspects for abnormalities in each of multiple modules, which are the smallest units for maintenance and replacement and are responsible for photoelectric conversion, by arranging multiple modules in an array or string shape, the solar panel array is photographed from the top side of the array. A video acquisition step in which the upper surface of the solar panel array is captured from above using infrared and visible light, and infrared and visible light videos are acquired as images, with the installation position of each of the multiple modules added. Image division step of dividing the infrared light video and the visible light video into a plurality of still images along the installation direction of the solar panel array, A panel condition analysis step, based on the divided still images, analyzes whether each of the multiple modules is in a state that may affect its power generation, or a state in which an abnormality is occurring in said power generation. A report creation step, based on the conditions that may affect the power generation or conditions that are abnormal in the power generation, as analyzed by the panel condition analysis step, creates a report indicating the operating status of the multiple modules. A solar panel inspection method characterized by comprising the following features.

15. The solar panel inspection method according to claim 14, characterized in that the video acquisition step is configured to include an infrared camera and a visible light camera mounted on an unmanned aerial vehicle, which capture the infrared video and the visible light video, respectively, and to pass the captured infrared video and the visible light video to the image splitting step.

Images