Photovoltaic power generation panel inspection movie processing method, inspection movie processing program, and inspection movie processing system

Abstract

To provide a photovoltaic power generation panel inspection movie processing method which allows a user to efficiently inspect a lower surface of a photovoltaic power generation panel.SOLUTION: In order to solve the above problems, a photovoltaic power generation panel inspection movie processing method according to an embodiment of the present invention has a step of estimating a position posture upon shooting of an inspection apparatus from a captured data obtained by the inspection apparatus, a step of comparing the position posture with position posture information of a photovoltaic power generation panel to specify the photovoltaic power generation panel form the captured data for extracting it, a step of comparing the photovoltaic power generation panel on the captured data with the position posture information of the photovoltaic power generation panel to estimate positional relationship between the inspection apparatus and the photovoltaic power generation panel, a step of converting the photovoltaic power generation panel captured from the positional relationship in an included direction relative to a captured surface into a captured data obtained by capture angular transformation, a step of outputting a result detection of abnormality, and a step of storing an inspection result.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] An embodiment of the present invention relates to an inspection video processing method, an inspection video processing program, and an inspection video processing system for a photovoltaic power generation panel. [Background technology]

[0002] Due to the recent increase in the severity of natural disasters and concerns about a future shortage of electrical safety personnel, there is a demand for labor-saving and unmanned operation and maintenance of solar power plants through the use of automatic inspection devices.

[0003] In particular, at large-scale solar power plants known as mega solar power plants with an output of over 1 MW, it is necessary to efficiently inspect the solar panels installed on vast sites with different environments.

[0004] Conventionally, there have been proposed inspection devices and methods for photovoltaic panels using flying mobile aircraft such as drones. According to these inspection devices and methods, when photographing the upper surface of a photovoltaic panel, i.e., the surface irradiated with sunlight, the photograph is taken from a sufficiently high altitude from an overhead perspective, so that inspection can be performed efficiently in a short time. On the other hand, when photographing the lower surface of a photovoltaic panel, i.e., the surface facing the ground, restrictions arise such as the inability to take an overhead photograph from a distance due to the presence of the ground or another photovoltaic panel, so that it has been difficult to inspect efficiently in a short time, as with the upper surface of a photovoltaic panel.

[0005] Inspecting the underside of solar panels is important to find abnormalities such as deterioration or burnout of the back panel, deterioration or burnout of the junction box, rust on the mounting frame, loose fixing bolts, broken cables, disconnected earth wires, insect nests, etc. If these abnormalities are left unchecked, it could lead to a decrease in power generation, a shortened lifespan of the entire system, damage from natural disasters, etc. In addition, as bifacial panels are expected to become more widespread in the future, it is expected that the need to inspect the underside of solar panels will increase even further. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2020-112499 [Summary of the Invention] [Problems to be Solved by the Invention]

[0007] As described above, when photographing the lower surface of a solar power generation panel with a flying mobile body such as a drone, restrictions such as being unable to take an aerial photograph at a distance due to the presence of the ground or another solar power generation panel occur, and the flying mobile body such as a drone is easily affected by positional deviation and shaking during photographing. The user needs to additionally perform an operation of extracting an image suitable for inspection from the photographed image, and there is a problem that workability is poor.

[0008] In addition, the photographed image is an image of the lower surface of an inclined solar power generation panel, and there is a problem that visibility is poor if the image as it is is used for viewing by the user.

[0009] The present invention has been made in view of such conventional circumstances, and an object thereof is to provide a method for processing inspection images of a solar power generation panel, an inspection image processing program, and an inspection image processing system that improve the workability and visibility of a user and enable the user to efficiently inspect the lower surface of a solar power generation panel. [Means for Solving the Problems]

[0010] In order to achieve the above object, the inspection video processing method for a solar power generation panel according to the present embodiment is an inspection video processing method for a solar power generation panel executed by a control device, and includes steps of: estimating the position and orientation of the inspection device at the time of shooting from shooting data captured by the inspection device; comparing the position and orientation with the position and orientation information of the solar power generation panel to identify and extract the solar power generation panel captured by the inspection device from the shooting data; comparing the solar power generation panel on the shooting data with the position and orientation information to estimate the positional relationship between the inspection device and the solar power generation panel; converting, based on the shooting data and the positional relationship, the shooting data into shooting data after shooting angle conversion corresponding to shooting data captured at a shooting angle smaller than the shooting angle of the solar power generation panel by the inspection device; performing abnormality detection on the shooting data or the shooting data after shooting angle conversion and outputting an abnormality detection result; and saving the position and orientation of the inspection device at the time of shooting, the shooting data, the shooting data after shooting angle conversion, and the abnormality detection result as inspection results.

[0011] In addition, in order to achieve the above object, the inspection video processing program for a photovoltaic panel according to the present embodiment is an inspection video processing program for a photovoltaic panel executed by a control device, and includes instructions for estimating the position and orientation of the inspection device at the time of shooting from shooting data captured by the inspection device, instructions for comparing the position and orientation with the position and orientation information of the photovoltaic panel to identify and extract the photovoltaic panel captured by the inspection device from the shooting data, instructions for comparing the photovoltaic panel on the shooting data with the position and orientation information to estimate the positional relationship between the inspection device and the photovoltaic panel, instructions for converting the shooting data into shooting data after shooting angle conversion corresponding to shooting data captured at a shooting angle smaller than the shooting angle of the photovoltaic panel by the inspection device based on the shooting data and the positional relationship, instructions for performing abnormality detection on the shooting data or the shooting data after shooting angle conversion and outputting an abnormality detection result, and instructions for saving the position and orientation of the inspection device at the time of shooting, the shooting data, the shooting data after shooting angle conversion, and the abnormality detection result as inspection results.

[0012] Furthermore, in order to achieve the above object, a solar power generation panel inspection video processing system according to the present embodiment is a solar power generation panel inspection video processing system including a control device that executes video processing of shooting data captured by an inspection device, wherein the control device includes: a position and orientation estimation unit that estimates the position and orientation of the inspection device at the time of shooting from the shooting data; a panel video extraction unit that compares the position and orientation with the position and orientation information of the solar power generation panel held by the panel arrangement database unit of the control device to identify and extract the solar power generation panel photographed by the inspection device from the shooting data; a position high-precision estimation processing unit that compares the solar power generation panel on the shooting data with the position and orientation information to estimate the positional relationship between the inspection device and the solar power generation panel; a video conversion unit that converts the shooting data into shooting data after shooting angle conversion corresponding to shooting data captured at a shooting angle smaller than the shooting angle of the solar power generation panel photographed by the inspection device based on the shooting data and the positional relationship; an abnormality processing unit that performs abnormality detection on the shooting data or the shooting data after shooting angle conversion and outputs an abnormality detection result; and an inspection video database unit that stores the position and orientation of the inspection device at the time of shooting, the shooting data, the shooting data after shooting angle conversion, and the abnormality detection result as inspection results.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0014] Hereinafter, an inspection video processing system, an inspection video processing method, and an inspection video processing program for a photovoltaic panel according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the embodiments shown below are examples of embodiments of the present invention and are not intended to limit the scope of the invention. Also, in the drawings referred to in the embodiments, the same parts or parts having the same function may be denoted by the same reference numerals or similar reference numerals, and the description thereof may be omitted. Further, the dimensional ratios in the drawings may be different from the actual ratios, and a part of the configuration may be omitted from the drawings.

[0015] The inspection video processing system 1 for a photovoltaic panel according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a configuration diagram of the inspection video processing system 1 for a photovoltaic panel according to this embodiment. The inspection video processing system 1 for a photovoltaic panel in FIG. 1 includes an inspection device 2, a control device 3, and a terminal device 4.

[0016] The inspection device 2 is a moving body such as a flying moving body like a drone or a traveling robot, and autonomously moves by automatic control according to a preset movement command or a movement command by remote operation. In addition to mechanisms (not shown) necessary for autonomous movement and remote operation, the inspection device 2 includes a photographing unit (not shown) for photographing a photovoltaic panel, and while autonomously moving by automatic control according to a movement command, the photographing unit photographs the lower surface of the photovoltaic panel. Here, the photographing unit refers to one that can photograph at least the lower surface of the photovoltaic panel. Examples thereof include a general camera, an omnidirectional camera, a smartphone built-in camera, a stereo camera, a thermal camera, a multispectral camera, etc. The data photographed by this photographing unit may be a photograph or a moving image.

[0017] When the communication unit (not shown) of the inspection device 2 transmits the photographed data of the photovoltaic panel, the photographing time, etc., and the communication unit of the control device 3 receives them, the control device 3 executes video processing based on the information held in the database of the control device 3. The control device 3 includes a processing unit 5 and a database unit 6 in addition to a control mechanism (not shown) that controls the inspection device 2. The processing unit 5 is a mechanism that performs video processing on the photographed data received from the inspection device 2, and includes a position and orientation estimation unit 7, a panel video extraction unit 8, a video preprocessing unit 9, a high-precision position estimation processing unit 10, a video conversion unit 11, and an abnormality processing unit 12. The database unit 6 is a database that holds information related to the photovoltaic panel, etc., and includes a panel layout database unit 13 and an inspection video database unit 14.

[0018] The position and orientation estimation unit 7 estimates the position and orientation of the inspection device 2 at the time of photographing using the photographed data received from the inspection device 2. Specifically, for example, the position and orientation estimation unit 7 creates an environmental map from the photographed data of the surrounding environment using VSLAM (Visual Simulated Localization and Mapping: self-position estimation), and estimates the position and orientation of the inspection device 2 at the time of photographing by integrating the environmental map and the photographed data. In addition to the position and orientation estimation unit 7, for example, the position and orientation of the inspection device 2 at the time of photographing may be estimated by incorporating or externally attaching a position information acquisition unit such as a transmitter of GPS (Global Positioning System: global positioning system) that measures the current position on the earth using artificial satellites, or by incorporating or externally attaching a position information acquisition unit such as an IMU (Inertial Measurement Unit: inertial measurement unit) that includes an acceleration sensor that detects translational motion for measuring three-dimensional inertial motion and a gyro sensor that detects rotational motion.

[0019] The panel image extraction unit 8 identifies and extracts the photovoltaic panel photographed by the inspection device 2. Specifically, for example, the panel image extraction unit 8 compares the position and orientation of the inspection device 2 estimated by the position and orientation estimation unit 7 with the position and orientation information of the photovoltaic panel held by the panel layout database unit 13, and reprojects the range of the photovoltaic panel held by the panel layout database unit 13 selected thereby onto the photographed data received from the inspection device 2, and identifies and extracts the range within the reprojection as the image of the photovoltaic panel.

[0020] When image processing is performed by the panel image extraction unit 8, by setting in advance the upper limit value of the relative position and orientation between the inspection device 2 and the photovoltaic panel as the shooting condition, when calculating the difference by comparing the position and orientation of the inspection device 2 estimated by the position and orientation estimation unit 7 with the position and orientation information of the photovoltaic panel held by the panel layout database unit 13, if the difference exceeds the pre-set upper limit value of the relative position and orientation, the control device 3 may output a feedback as a movement command to the inspection device 2.

[0021] The pre-image processing unit 9 corrects the image of the photovoltaic panel extracted by the panel image extraction unit 8. Specifically, for example, the pre-image processing unit 9 calculates the change amount per unit time of the position and orientation at the times before and after shooting, and if the change amount is a certain value or more, excludes the image as being affected by position shift or shaking. Also, for example, the pre-image processing unit 9 checks the luminance distribution of the extracted image of the photovoltaic panel, and if it does not fall within a specified distribution in advance, excludes the image. On the other hand, if it falls within the specified distribution in advance, the luminance distribution is corrected by linear or histogram equalization so as to be the specified distribution.

[0022] When image processing is performed by the image preprocessing unit 9, if the captured data is not suitable as the captured data of the solar power generation panel due to darkness caused by backlighting, weather conditions, etc., the position and orientation of the inspection device 2 estimated by the position and orientation estimation unit 7 are compared with the position and orientation information of the solar power generation panel stored in the panel layout database unit 13, and the area of the solar power generation panel in the captured data is estimated. Then, the imaging conditions are estimated such that the luminance distribution within the area becomes the luminance distribution within a previously specified range, and the control device 3 may output a feedback as a correction command to the inspection device 2 based on the estimated imaging conditions.

[0023] The high-precision position estimation processing unit 10 compares the position and orientation of the solar power generation panel on the captured data corrected by the image preprocessing unit 9 with the position and orientation information of the solar power generation panel held by the panel layout database unit 13, and estimates the positional relationship between the inspection device 2 and the solar power generation panel captured by the inspection device 2. Specifically, for example, the high-precision position estimation processing unit 10 extracts parts such as the rectangular area of the solar power generation panel or the intersection of the pedestals, which are characteristic parts of the solar power generation panel, from the captured data corrected by the image preprocessing unit 9, extracts the three-dimensional coordinates of the part from the panel layout database unit 13, and solves the PnP (Perspective-n-point) problem based on the correspondence between the extracted three-dimensional coordinates of the part and the two-dimensional coordinates of the part on the captured data, thereby estimating the positional relationship between the inspection device 2 and the solar power generation panel.

[0024] The image conversion unit 11 converts the captured data captured by the inspection device 2 into captured data after image angle conversion corresponding to the captured data captured at an image angle smaller than the imaging angle of the solar power generation panel by the inspection device 2 based on the positional relationship between the inspection device 2 estimated by the position high-precision estimation processing unit 10 and the solar power generation panel captured by the inspection device 2. The imaging angle referred to here means the angle formed by the imaging direction of the inspection device 2 with respect to the normal direction of the solar power generation panel. Specifically, for example, the image conversion unit 11 extracts the three-dimensional coordinates corresponding to the four corners of the solar power generation panel on the captured data from the panel layout database unit 13 based on the captured data captured by the inspection device 2 and the positional relationship between the inspection device 2 and the solar power generation panel estimated by the position high-precision estimation processing unit 10, and performs conversion so that the four corners of the solar power generation panel on the captured data become a rectangle with the designed aspect ratio. It is preferable that the captured data after image angle conversion is converted into a state where the normal direction of the solar power generation panel and the imaging direction of the inspection device 2 overlap, that is, the imaging angle is 0 degrees and the solar power generation panel is facing directly.

[0025] The abnormality processing unit 12 detects abnormalities in the solar power generation panel for the captured data or the captured data after image angle conversion. Specifically, for example, the abnormality processing unit 12 performs abnormality detection by difference from past images or machine learning using a model that has learned abnormal images in advance for the captured data or the captured data after image angle conversion, and outputs an abnormality detection result in which the abnormalities existing in the solar power generation panel are visualized by coordinate values, contour diagrams, etc.

[0026] The panel layout database unit 13 stores information on the layout of the solar power generation panel, such as the coordinates, dimensions, frame dimensions, and orientation of the installed solar power generation panel.

[0027] The inspection video database unit 14 stores by associating the shooting data captured by the inspection device 2 of the solar power generation panel, the shooting time, the position and orientation at the time of shooting of the inspection device 2 estimated by the position and orientation estimation unit 7, the shooting data after shooting angle conversion converted by the video conversion unit 11, the shooting data output by the abnormality processing unit 14, or the abnormality detection result related to the shooting data after shooting angle conversion, etc.

[0028] The terminal device 4 is a terminal device such as a PC terminal or a mobile terminal used by the user. The terminal device 4 includes a display operation unit 15 in addition to mechanisms (not shown) provided in the terminal device 4 itself.

[0029] The display operation unit 15 is a display operation unit for the user to view the inspection results stored in the inspection video database unit 14 via a web browser or an application. The inspection results referred to here refer to the shooting data, shooting time, position and orientation at the time of shooting of the inspection device 2, shooting data after shooting angle conversion, and abnormality detection results related to the shooting data or shooting data after shooting angle conversion stored in the inspection video database unit 14. This display operation unit 15 enables the control device 3 to detect operations by the user and to display data output from the control device 3.

[0030] The details of the display operation unit 15 will be described with reference to FIGS. 2 to 4. FIGS. 2 to 4 are explanatory diagrams of the display operation unit 15 of the terminal device 4 in the inspection video processing system 1 for the solar power generation panel according to the present embodiment. The display operation unit 15 in FIG. 2 includes a display screen 20, a layer switching screen 21, and a date selection screen 22. The display operation unit 15 in FIG. 3 further includes a shooting data display screen 23, an abnormality detection result display screen 24, a date selection screen 25, a date selection screen 26, and a report creation selection screen 27 in addition to FIG. 2. The display operation unit 15 in FIG. 4 further includes a shooting data comparison screen 28 and an abnormality detection result comparison screen 29 in addition to FIGS. 2 and 3.

[0031] The display screen 20 displays the drawings previously saved by the user. For example, as shown in FIG. 2, this drawing is an overhead view of the photovoltaic panel 30, and the overhead view is saved in association with the movement path at the time of shooting by the inspection device 2 in advance.

[0032] The layer switching screen 21 is for the user to select when various data stored in the inspection video database unit 14 is to be superimposed and displayed on the drawing displayed on the display screen 20. The various data mentioned here are, for example, the movement path, the photographed data after conversion of the photographing angle, the abnormality detection result, etc. As shown in FIG. 2(a), when the user selects the movement path, the movement path 31 of the inspection device 2 determined based on the position and posture of the inspection device 2 at the time of shooting stored in the inspection video database unit 14 is superimposed and displayed on the drawing displayed on the display screen 20. Also, as shown in FIG. 2(b), when the user selects the photographed data after conversion of the photographing angle, the photographed data 32 after conversion of the photographing angle stored in the inspection video database unit 14 is superimposed and displayed on the drawing displayed on the display screen 20. Furthermore, as shown in FIG. 2(c), when the user selects the abnormality detection result, the abnormality detection result 33 of the photovoltaic panel stored in the inspection video database unit 14 is superimposed and displayed on the drawing displayed on the display screen 20. At this time, the abnormality detection position 34 existing on the photovoltaic panel is visualized as, for example, coordinate values or a contour map.

[0033] The date selection screen 22 is for the user to select the date of the drawing data displayed on the display screen 20.

[0034] The photographed data display screen 23 is displayed, for example, on the upper right side of the display screen 20 when the user selects the position 35 of the movement path 31 of the inspection device 2 superimposed on the display screen 20 as shown in FIG. 3. On this photographed data display screen 23, the photographed data after conversion of the photographing angle stored in the inspection video database unit 14 corresponding to the position 35 selected by the movement path 31 is displayed.

[0035] When the user selects the position 35 on the movement path 31 of the inspection device 2 superimposed on the display screen 20, for example, as shown in FIG. 3, the abnormal detection result display screen 24 is displayed at the lower right of the display screen 20. On this abnormal detection result display screen 24, the abnormal detection results of the solar power generation panel stored in the inspection video database unit 14 corresponding to the position 35 selected on the movement path 31 are displayed.

[0036] In FIG. 3, the case where the user selects the position 35 on the movement path 31 of the inspection device 2 superimposed on the display screen 20 is taken as an example to explain the shooting data display screen 23 and the abnormal detection result display screen 24. However, the same applies when the user selects other positions on the movement path 31 of the inspection device 2 superimposed on the display screen 20 that the user wants to view. Needless to say, the same also applies when the user selects the panel that the user wants to view from the shooting data 32 after shooting angle conversion superimposed on the display screen 20 shown in FIG. 2(b), or when the user selects the panel that the user wants to view from the abnormal detection results 33 superimposed on the display screen 20 shown in FIG. 2(c).

[0037] The date selection screen 25 is for the user to select the date of the data to be compared with the shooting data after shooting angle conversion displayed on the shooting data display screen 23. When the user selects the date of the data to be compared on this date selection screen 25, as shown in FIG. 4, the shooting data comparison screen 28 is displayed side by side with the shooting data display screen 23. On this shooting data comparison screen 28, the shooting data after shooting angle conversion corresponding to the date of the data selected by the user and corresponding to the solar power generation panel displayed on the shooting data display screen 23 is displayed.

[0038] The date selection screen 26 is for the user to select the date of the data to be compared with the abnormality detection results displayed on the abnormality detection result display screen 24. When the user selects the date of the data to be compared on this date selection screen 26, as shown in FIG. 4, the abnormality detection result comparison screen 29 is displayed side by side with the abnormality detection result display screen 24. On this abnormality detection result comparison screen 29, the abnormality detection results corresponding to the date of the data selected by the user and corresponding to the solar power generation panel displayed on the abnormality detection result display screen 24 are displayed.

[0039] The report creation selection screen 27 is for the user to select when they want to save the data displayed at that time. When the user selects this report creation selection screen 27, the data displayed at that time is output as a file such as PDF or JPG, or as a URL.

[0040] Here, with reference to FIG. 5, the inspection video processing method and inspection video processing program until the data that the user views using the terminal device 4 will be described. FIG. 5 is a flowchart showing processing 100 in the inspection video processing method and inspection video processing program for the solar power generation panel according to the present embodiment.

[0041] Processing 100 is performed when the control device 3 receives shooting data from the inspection device 2, and one or more processors (not shown) provided in the control device 3 execute a software program and / or instruction set stored in a memory (not shown).

[0042] First, when the communication unit of the inspection device 2 transmits the shooting data taken by the inspection device 2 to the control device 3 and the communication unit of the control device 3 receives this, the position and orientation estimation unit 7 of the control device 3 estimates the position and orientation of the inspection device 2 at the time of shooting from the shooting data taken by the inspection device 2 (S101).

[0043] Next, the panel video extraction unit 8 of the control device 3 compares the position and orientation of the inspection device 2 during shooting estimated by the position and orientation estimation unit 7 of the control device 3 with the position and orientation information of the solar power generation panels held by the panel layout database unit 13 of the control device 3, identifies and extracts the solar power generation panels from the shooting data captured by the inspection device 2 (S102).

[0044] Next, the video preprocessing unit 9 of the control device 3 corrects the shooting data of the solar power generation panels extracted by the panel video extraction unit 8 of the control device 3 (S103).

[0045] Subsequently, the position high-precision estimation processing unit 10 of the control device 3 compares the solar power generation panels on the shooting data corrected by the video preprocessing unit 9 with the position and orientation information of the solar power generation panels held by the panel layout database unit 13, and estimates the positional relationship between the inspection device 2 and the solar power generation panels captured by the inspection device 2 (S104).

[0046] Subsequently, based on the shooting data captured by the inspection device 2 and the positional relationship between the inspection device 2 and the solar power generation panels captured by the inspection device 2 estimated by the position high-precision estimation processing unit 10 of the control device 3, the video conversion unit 11 of the control device 3 converts the shooting data into shooting angle-converted shooting data corresponding to the shooting data captured at a shooting angle smaller than the shooting angle of the solar power generation panels by the inspection device 2 (S105).

[0047] Subsequently, the abnormality processing unit 12 of the control device 3 performs abnormality detection on the shooting data or the shooting angle-converted shooting data, and outputs the abnormality detection result of the solar power generation panels (S106).

[0048] Finally, the inspection video database unit 14 of the control device 3 associates and stores the shooting data, shooting time, position and orientation of the inspection device 2 during shooting, shooting angle-converted shooting data, and the abnormality detection result related to the shooting data or the shooting angle-converted shooting data as inspection results (S107).

[0049] Next, with reference to FIG. 6, a method for inspecting video processing and an inspection video processing program when a user views inspection results using the terminal device 4 will be described. FIG. 6 is a flowchart showing a process 200 in the inspection video processing method and the inspection video processing program for the photovoltaic panel according to the present embodiment.

[0050] When the control device 3 detects each operation of the terminal device 4 performed by the user to view the inspection results, the process 200 is performed by one or more processors provided in the control device 3 and the terminal device 4 executing a software program and / or an instruction set stored in the memory.

[0051] First, when the user accesses the control device 3 via a web browser or an application on the terminal device 4, the control device 3 detects this access by the user and outputs data corresponding to this access from the inspection video database unit 14, so that a drawing in which the movement path at the time of shooting of the inspection device 2 previously saved by the user is linked is displayed on the display screen 20 of the terminal device 4 (from S201 to S203).

[0052] Next, when the user selects the date of the inspection results to be viewed on the date selection screen 22 of the terminal device 4, the control device 3 detects the date selected by the user on the date selection screen 22 and outputs data corresponding to this date from the inspection video database unit 14, so that a drawing in which the movement path at the time of shooting of the inspection device 2 corresponding to the date selected by the user is linked is displayed on the display screen 20 of the terminal device 4 (from S204 to S206).

[0053] Next, when the user selects an item to be superimposed on the drawing on the display screen 20 on the layer switching screen 21 of the terminal device 4, the control device 3 detects the item selected by the user on the layer switching screen 21 and outputs data corresponding to this item from the inspection device database unit 14, so that the inspection results of the item selected by the user are superimposed on the drawing displayed on the display screen 20 of the terminal device 4 (from S207 to S209).

[0054] Subsequently, when the user selects a position to view among the inspection results superimposed on the drawing displayed on the display screen 20 of the terminal device 4, the control device 3 detects the position selected by the user on the display screen 20 and outputs data corresponding to this position from the inspection video database unit 14, so that the photographed data after the photographing angle conversion and the abnormality detection result at the position selected by the user are respectively displayed on the photographed data display screen 23 and the abnormality detection result display screen 24 displayed on the right side of the display screen 20 of the terminal device 4 (from S210 to S212).

[0055] Subsequently, when the user selects the date of the inspection result to be compared on the date selection screen 25 or the date selection screen 26 of the terminal device 4, the control device 3 detects the date selected by the user on the date selection screen 25 or the date selection screen 26 and outputs data corresponding to this date from the inspection video database unit 14, so that the photographed data after the photographing angle conversion and the abnormality detection result on the date selected by the user are arranged and displayed on the photographed data comparison screen 28 or the abnormality detection result comparison screen 29 displayed on the right side of the photographed data display screen 23 or the abnormality detection result display screen 24 of the terminal device 4 (from S213 to S215).

[0056] Finally, when the user selects the report creation selection screen 27 of the terminal device 4, the inspection results displayed on the display operation unit 15 of the terminal device 4 at the time of this selection are output as files such as PDF and JPG or URLs (S216 and S217).

[0057] As described above, according to the inspection video processing system 1 of the photovoltaic panel of the present embodiment, the inspection device 2 moves autonomously by automatic control according to a preset movement command or a movement command by remote operation, captures an image of the lower surface of the photovoltaic panel, and transmits the captured data to the control device 3. When the control device 3 receives this captured data, it estimates the position and orientation of the inspection device 2 at the time of imaging, compares the estimated position and orientation of the inspection device 2 at the time of imaging with the position and orientation information of the photovoltaic panel held by the panel arrangement database unit 14, identifies and extracts the photovoltaic panel from the captured data, corrects the image of the extracted photovoltaic panel, compares the photovoltaic panel on the corrected captured data with the photovoltaic panel on the panel arrangement database unit 14, estimates the positional relationship between the inspection device 2 and the photovoltaic panel, based on the captured data and the estimated positional relationship between the inspection device 2 and the photovoltaic panel, converts the captured data corresponding to the captured data captured at an imaging angle smaller than the imaging angle of the photovoltaic panel by the inspection device 2 into post-imaging angle conversion captured data, performs anomaly detection on the captured data or the post-imaging angle conversion captured data, and outputs the anomaly detection result of the photovoltaic panel, and associates and stores the captured data, the position and orientation of the inspection device 2 at the time of imaging, the post-imaging angle conversion captured data, the anomaly detection result regarding the captured data or the post-imaging angle conversion captured data, etc. as inspection results in sequence.After that, when the user operates the terminal device 4 to view the inspection results, if the control device 3 detects that the user has accessed the control device 3 with the terminal device 4, the step of causing the terminal device 4 to display, from the inspection video database unit 14, a drawing associated with the movement path during shooting of the inspection device 2 corresponding to this access; if the control device 3 detects that the user has selected the date of the inspection results that the user wants to view with the terminal device 4, the step of causing the terminal device 4 to display, from the inspection video database unit 14, a drawing associated with the movement path during shooting of the inspection device 2 corresponding to this date; if the control device 3 detects that the user has selected an item for which the user wants to perform an overlay display on the drawing, the step of causing the inspection results corresponding to this item to be overlaid and displayed on the drawing displayed on the terminal device 4 from the inspection video database unit 14; if the control device 3 detects that the user has selected a position for which the user wants to view among the overlaid inspection results, the step of causing the inspection results corresponding to this position to be displayed on the terminal device 4 from the inspection video database unit 14; if the control device 3 detects that the user has selected the date of the inspection results that the user wants to compare, the step of causing the inspection results corresponding to this date to be displayed on the terminal device 4 from the inspection video database unit 14; if it is detected that the user has selected to save the data displayed on the terminal device 4, the step of causing the data displayed on the terminal device 4 at this point of selection to be output as a file is appropriately performed.

[0058] In the inspection of the lower surface of a solar power generation panel using a flying mobile body such as a conventional drone, limitations such as being unable to take an overhead shot at a distance due to the presence of the ground or another solar power generation panel, etc. occur, and it is easily affected by position deviation and shaking of the flying mobile body such as a drone during shooting. The user has to additionally perform the work of extracting a video suitable for inspection from the captured video, and there is a problem of poor workability. In addition, the captured video is a video of the lower surface of an inclined solar power generation panel, and there is a problem that the visibility is poor if the video as it is is used for viewing by the user.

[0059] On the other hand, in the inspection video processing system for a photovoltaic panel according to the present embodiment, by adopting the above-described configuration and operation, both the workability and visibility of the user can be improved, enabling the user to easily and efficiently inspect the lower surface of the photovoltaic panel.

[0060] In the inspection video processing system 1 for a photovoltaic panel according to the present embodiment, although the control device 3 and the terminal device 4 are configured as separate devices for the purpose of explanation, the present invention is not limited thereto. For example, the control device 3 and the terminal device 4 may be configured as an integrated device.

[0061] As described above, some embodiments of the present invention have been explained. However, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0062] 1... Inspection video processing system for a photovoltaic panel, 2... Inspection device, 3... Control device, 4... Terminal device, 5... Processing unit, 6... Database unit, 7... Position and orientation estimation unit, 8... Panel video extraction unit, 9... Video preprocessing unit, 10... High-precision position estimation processing unit, 11... Video conversion unit, 12... Abnormality processing unit, 13... Panel layout database unit, 14... Inspection video database unit, 15... Display operation unit, 20... Display screen, 21... Layer switching screen, 22... Date selection screen, 23... Shooting data display screen, 24... Abnormality detection result display screen, 25... Date selection screen, 26... Date selection screen, 27... Report creation selection screen, 28... Shooting data comparison screen, 29... Abnormality detection result comparison screen, 30... Photovoltaic panel, 31... Movement path, 32... Shooting data after conversion of shooting angle, 33... Abnormality detection result, 34... Abnormality detection position, 35... Position, 100... Processing, 200... Processing.

Claims

1. A method for inspecting video processing of a solar power generation panel executed by a control device, comprising: estimating the position and orientation of the inspection device at the time of shooting from the shooting data captured by the inspection device; comparing the position and orientation with the position and orientation information of the solar power generation panel, identifying and extracting the solar power generation panel photographed by the inspection device from the shooting data; comparing the solar power generation panel on the shooting data with the position and orientation information to estimate the positional relationship between the inspection device and the solar power generation panel; converting the shooting data into shooting data after shooting angle conversion corresponding to the shooting data captured at a shooting angle smaller than the shooting angle of the solar power generation panel by the inspection device based on the shooting data and the positional relationship; performing anomaly detection on the shooting data or the shooting data after shooting angle conversion and outputting an anomaly detection result; saving the position and orientation of the inspection device at the time of shooting, the shooting data, the shooting data after shooting angle conversion, and the anomaly detection result as inspection results; A method for inspecting video processing of a solar power generation panel, characterized by including the above steps.

2. When the control device detects that the user has accessed the control device with a terminal device, outputting data corresponding to this access to the terminal device; When the control device detects that the user has selected the date of the inspection result to be viewed with the terminal device, outputting data corresponding to this date to the terminal device; When the control device detects that the user has selected an item to be superimposed and displayed on the terminal device, outputting data corresponding to this item to the terminal device; When the control device detects that the user has selected a position to be viewed among the data superimposed and displayed on the terminal device, outputting data corresponding to this position to the terminal device; When the control device detects that the user has selected the date of the inspection result to be compared with the terminal device, outputting data corresponding to this date to the terminal device; When the control device detects that the user has selected to save the data displayed on the terminal device, causing the terminal device to output the data displayed at this time of selection as a file; The method for inspecting video processing of a solar power generation panel according to Claim 1, further characterized by including the above steps.

3. When the control device detects that the user has accessed the control device using the terminal device, causing the terminal device to display a drawing associated with the movement path during imaging of the inspection device corresponding to this access. When the control device detects that the user has selected the date of the inspection result that the user wishes to view using the terminal device, causing the terminal device to display a drawing associated with the movement path during imaging of the inspection device corresponding to this date. When the control device detects that the user has selected an item that the user wishes to superimpose and display on the terminal device, causing the inspection result corresponding to this item to be superimposed and displayed on the drawing displayed on the terminal device. When the control device detects that the user has selected a position that the user wishes to view among the inspection results superimposed and displayed on the terminal device, causing the inspection result corresponding to this position to be displayed on the terminal device. When the control device detects that the user has selected the date of the inspection result that the user wishes to compare using the terminal device, causing the inspection result corresponding to this date to be displayed on the terminal device. When the control device detects that the user has selected to save the data displayed on the terminal device, causing the data displayed on the terminal device at this point of selection to be output as a file. The method for processing inspection images of a photovoltaic panel according to claim 1, further comprising the above steps.

4. An inspection image processing program for a photovoltaic panel executed by a control device, Instructions for estimating the position and orientation of the inspection device during imaging from the imaging data captured by the inspection device. Instructions for comparing the position and orientation with the position and orientation information of the photovoltaic panel, and specifying and extracting the photovoltaic panel imaged by the inspection device from the imaging data. Instructions for comparing the photovoltaic panel on the imaging data with the position and orientation information, and estimating the positional relationship between the inspection device and the photovoltaic panel. Instructions for converting the imaging data to imaging data after imaging angle conversion corresponding to imaging data captured at an imaging angle smaller than the imaging angle of the photovoltaic panel by the inspection device based on the imaging data and the positional relationship. Instructions for performing abnormality detection on the imaging data or the imaging data after imaging angle conversion and outputting the abnormality detection result. Instructions for storing the position and orientation of the inspection device during imaging, the imaging data, the imaging data after imaging angle conversion, and the abnormality detection result as inspection results, A program for inspecting video processing of a solar power generation panel, characterized by including the above.

5. When the control device detects that the user has accessed the control device using a terminal device, instructions for outputting data corresponding to this access to the terminal device, When the control device detects that the user has selected the date of the inspection result that the user wants to view using the terminal device, instructions for outputting data corresponding to this date to the terminal device, When the control device detects that the user has selected an item that the user wants to superimpose and display on the terminal device, instructions for outputting data corresponding to this item to the terminal device, When the control device detects that the user has selected a position that the user wants to view among the data superimposed and displayed on the terminal device, instructions for outputting data corresponding to this position to the terminal device, When the control device detects that the user has selected the date of the inspection result that the user wants to compare using the terminal device, instructions for outputting data corresponding to this date to the terminal device, The program for inspecting video processing of a solar power generation panel according to claim 4, further characterized by including the above.

6. When the control device detects that the user has accessed the control device using a terminal device, a step of displaying on the terminal device a drawing associated with the movement path of the inspection device during imaging corresponding to this access, When the control device detects that the user has selected the date of the inspection result that the user wants to view using the terminal device, instructions for displaying on the terminal device a drawing associated with the movement path of the inspection device during imaging corresponding to this date, When the control device detects that the user has selected an item that the user wants to superimpose and display on the terminal device, instructions for superimposing and displaying the inspection result corresponding to this item on the drawing displayed on the terminal device, When the control device detects that the user has selected a position that the user wants to view among the inspection results superimposed and displayed on the terminal device, instructions for displaying the inspection result corresponding to this position on the terminal device, When the control device detects that the user has selected the date of the inspection result to be compared on the terminal device, an instruction for causing the terminal device to display the inspection result corresponding to this date, When the control device detects that the user has selected to save the data displayed on the terminal device, an instruction for causing the terminal device to output the data displayed at this point in time as a file, The inspection video processing program for a solar power generation panel according to claim 4, further comprising:

7. An inspection video processing system for a solar power generation panel including a control device that executes video processing of shooting data captured by an inspection device, The control device is A position and orientation estimation unit that estimates the position and orientation of the inspection device at the time of shooting from the shooting data, A panel video extraction unit that compares the position and orientation with the position and orientation information of the solar power generation panel held in the panel arrangement database unit of the control device, and identifies and extracts the solar power generation panel photographed by the inspection device from the shooting data, A position high-precision estimation processing unit that estimates the positional relationship between the inspection device and the solar power generation panel by comparing the solar power generation panel on the shooting data with the position and orientation information, A video conversion unit that converts the shooting data into shooting angle-converted shooting data corresponding to shooting data captured at a shooting angle smaller than the shooting angle of the solar power generation panel by the inspection device based on the shooting data and the positional relationship, An abnormality processing unit that performs abnormality detection on the shooting data or the shooting angle-converted shooting data and outputs an abnormality detection result, An inspection video database unit that stores the position and orientation of the inspection device at the time of shooting, the shooting data, the shooting angle-converted shooting data, and the abnormality detection result as inspection results, An inspection video processing system for a solar power generation panel, characterized in that it comprises:

8. An inspection video processing system for a solar power generation panel according to claim 7, further comprising a terminal device used by the user to view the inspection results, When the user operates to view the inspection results on the terminal device, the control device When detecting that the user has accessed the control device on the terminal device, outputs the data corresponding to this access to the terminal device, When detecting that the user has selected the date of the inspection result to be viewed on the terminal device, outputs the data corresponding to this date to the terminal device, When it is detected that the user has selected an item for which superimposed display on the terminal device is desired, data corresponding to this item is output to the terminal device. When it is detected that the user has selected a position for which browsing of the data superimposed on the terminal device is desired, data corresponding to this position is output to the terminal device. When it is detected that the user has selected a date of the inspection result to be compared by the terminal device, data corresponding to this date is output to the terminal device. The photovoltaic panel inspection video processing system according to claim 7, characterized in that when it is detected that the user has selected to save the data displayed on the terminal device, the data displayed on the terminal device at this point of selection is output as a file.

9. The photovoltaic panel inspection video processing system according to claim 7, further including a terminal device used by the user to view the inspection result, When the user operates to view the inspection result with the terminal device, the control device, When it is detected that the user has accessed the control device with the terminal device, a drawing linked to the movement path during shooting of the inspection device corresponding to this access is displayed on the terminal device. When it is detected that the user has selected a date of the inspection result to be browsed with the terminal device, a drawing linked to the movement path during shooting of the inspection device corresponding to this date is displayed on the terminal device. When it is detected that the user has selected an item for which superimposed display on the terminal device is desired, the inspection result corresponding to this item is superimposed and displayed on the drawing displayed on the terminal device. When it is detected that the user has selected a position for which browsing of the data superimposed on the terminal device is desired, the inspection result corresponding to this position is displayed on the terminal device. When it is detected that the user has selected a date of the inspection result to be compared by the terminal device, the inspection result corresponding to this date is displayed on the terminal device. The photovoltaic panel inspection video processing system according to claim 7, characterized in that when it is detected that the user has selected to save the data displayed on the terminal device, the data displayed on the terminal device at this point of selection is output as a file.

10. The terminal device according to claim 8 or 9, characterized by having a display operation unit that enables the control device to detect an operation by the user and to display data output from the control device, for a solar power generation panel inspection video processing system.

Images