Apparatus for inspecting defects of solar panels and method for operating same

Abstract

The present invention relates to an apparatus for detecting a defect of a solar panel and a method for operating the same, the apparatus for detecting a defect of a solar panel comprising: a model providing unit for providing a pre-trained model; a communication module receiving a captured image of the solar panel from the image sensor; and a processing module functionally connected to the model providing unit and the communication module, where the processing module is configured to collect captured images of the solar panel, apply the captured images to the pre-trained model to create an attention map, and determine whether there is a defect in the solar panel according to whether the attention map is abnormal.

Description

Technical Field

[0001] The present invention relates to a solution for supporting a fast and accurate inspection of solar panels for defects. Background Art

[0002] The use of fossil fuels such as oil and coal has caused various environmental problems such as air pollution. Recently, a lot of development has been made on environmentally friendly energy sources that replace fossil fuels, and as a part of it, interest in solar power generation has rapidly increased.

[0003] Solar power generation generates energy using solar cells that convert received sunlight into electrical energy. Solar panels with multiple solar cells arranged in an array are installed in locations where sunlight is well absorbed to generate energy.

[0004] Typically, solar panels are installed in large quantities in a very wide area to generate a large amount of energy, or are installed on the roof or top plate of a building suitable for collecting sunlight. However, since solar panels are installed and operated outdoors, they are directly exposed to the influence of the external environment. Therefore, solar panels often have defects, which may lead to a sharp drop in solar power generation or an increased risk of fire.

[0005] Therefore, it is important to determine the status of solar panels and quickly replace abnormal solar panels. In addition, many defects may occur during the manufacturing process of solar panels, so a method is needed to quickly check solar panels with these defects and replace or repair them. Summary of the invention

[0006] Technical issues

[0007] The present invention is proposed in view of the above situation, and an object of the present invention is to provide an apparatus and method capable of quickly and accurately determining whether there is an abnormality in a solar panel during the process of manufacturing the solar panel and the process of using the solar panel after installation.

[0008] Various other objects of the present invention will be mentioned together with the following description of the embodiments.

[0009] Technical Solution

[0010] In order to achieve the above-mentioned object, an apparatus for inspecting defects in a solar panel according to an embodiment of the present invention includes: a model providing unit, the model providing unit providing a pre-trained model; a communication module, the communication module receiving a captured image of a solar panel from an image sensor; and a processing module, the processing module being functionally connected to the model providing unit and the communication module. The processing module is configured to collect the captured image of the solar panel, generate an attention map by applying the captured image to the pre-trained model, and determine whether there is a defect in the solar panel according to whether the attention map is abnormal.

[0011] Specifically, the processing module applies an attention mechanism including convolutional neural network training to a large-scale public dataset.

[0012] Specifically, the processing module is configured to extract at least one statistical feature from the attention map, detect a correlation between the extracted statistical feature and a label value indicating a defective solar panel, and determine whether the solar panel is defective based on the degree of the correlation.

[0013] Specifically, the processing module is configured to extract a standard deviation of surface flatness of the solar panel from the attention map, and detect a correlation between the standard deviation of surface flatness of the defective solar panel and the standard deviation of surface flatness extracted from the attention map.

[0014] Specifically, the processing module is configured to extract at least one of a mean value of the surface flatness, a kurtosis of the surface flatness, and a skewness of the surface flatness, and detect a correlation between the extracted at least one of the mean value, the kurtosis, and the skewness and at least one of the mean value, the kurtosis, and the skewness in a graph generated from an image of the defective solar panel.

[0015] Specifically, the defect inspection device further includes an output module, which outputs a guidance message indicating whether the solar cell panel has defects.

[0016] In order to achieve the above-mentioned object, a method for inspecting defects in a solar panel according to an embodiment of the present invention comprises the following steps: preparing a pre-trained model; collecting captured images of the solar panel; generating an attention map by applying the captured images to the pre-trained model; and determining whether there is a defect in the solar panel according to whether the attention map is abnormal.

[0017] Specifically, the step of preparing the pre-trained model includes the following steps: applying an attention mechanism including convolutional neural network training to a large-scale public dataset.

[0018] Specifically, the step of determining whether there is a defect includes the following steps: extracting at least one statistical feature from the attention map; detecting a correlation between the extracted statistical feature and a label value indicating a defective solar panel; and determining whether the solar panel is defective based on the degree of the correlation.

[0019] Specifically, the step of extracting the statistical features includes the step of extracting the surface flatness of the solar panel from the attention map, and the step of detecting the correlation includes the step of detecting the correlation between the surface flatness of the defective solar panel and the surface flatness extracted from the attention map.

[0020] Specifically, the step of extracting the statistical feature includes the step of extracting at least one of a mean value of the surface flatness, a kurtosis of the surface flatness, and a skewness of the surface flatness, and the step of detecting the correlation includes the step of detecting a correlation between the extracted at least one of the mean value, the kurtosis, and the skewness and at least one of the mean value, the kurtosis, and the skewness in a graph generated from the image of the defective solar panel.

[0021] Specifically, the method further includes the step of outputting a guidance message indicating whether the solar cell panel is defective.

[0022] Beneficial effects

[0023] According to an embodiment of the present invention, an apparatus for inspecting defects in a solar panel and a method for operating the apparatus can quickly and accurately determine whether there is an abnormality in a solar panel during the manufacturing process of the solar panel, thereby reducing the use of defective solar panels and providing high reliability.

[0024] Furthermore, according to the present invention, by determining whether there is an abnormality based on the flatness of a solar panel in an environment in which the solar panel is used, it is relatively less affected by the surrounding environment than defect detection using thermal imaging, thereby providing more accurate results.

[0025] Other effects of the present invention will be described through the embodiments described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 are diagrams showing various states of a solar cell panel.

[0027] Figure 2 : is a graph showing the standard deviation value of the surface flatness of a non-defective solar cell panel and the standard deviation value of the surface flatness of a defective solar cell panel.

[0028] Figure 3 is a diagram showing examples of various characteristic values ​​applicable to the present invention.

[0029] Figure 4 is a diagram showing an example of an attention map of the MNIST dataset in a large-scale public dataset for a pre-trained model.

[0030] Figure 5 is a diagram showing an original solar panel image and an attention map.

[0031] Figure 6 is a graph showing the correlation coefficient between the extracted features and a label indicating whether a solar panel is defective.

[0032] Figure 7 is a graph showing defect detection performance using various learning models.

[0033] Figure 8 is a graph showing AUROC values ​​measured in the above-mentioned comparative experiment of defect detection performance.

[0034] Fig. 9 is a diagram showing an example of an environment in which a defect detection technology for a solar cell panel according to an embodiment of the present invention is applied.

[0035] Fig.10 : is a diagram showing an example of a configuration of a defect detection device for a solar cell panel according to an embodiment of the present invention.

[0036] Fig.11 is an exemplary diagram of a hardware system for implementing a defect detection device according to an embodiment of the present invention.

[0037] Fig.12 is a diagram showing an example of a signal flow between devices for defect detection of a solar cell panel according to an embodiment of the present invention.

[0038] Fig.13 is a diagram illustrating an example of a method of operating a defect detection apparatus in defect detection of a solar cell panel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention.

[0040] Furthermore, the technical terms used in the present specification should be interpreted as having the meanings generally understood by a person of ordinary skill in the art to which the present invention belongs, unless otherwise specifically defined in the present specification, and should not be interpreted as being too comprehensive or too narrow.

[0041] In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately express the ideas of the present invention, they should be replaced by technical terms that can be correctly understood by those skilled in the art, and the general terms used in the present invention should be interpreted according to their dictionary definitions or contexts, and should not be interpreted too narrowly.

[0042] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The same or similar components will be given the same reference numerals, and redundant descriptions thereof will be omitted.

[0043] Furthermore, in explaining the present invention, if it is judged that the detailed description of the related known art may make the subject matter of the present invention unclear, the detailed description thereof will be omitted.

[0044] In addition, it should be noted that the drawings are only intended to facilitate easy understanding of the present invention and should not be interpreted as limiting the scope of the present invention. The scope of the present invention should be interpreted as extending to all modifications, equivalents and substitutes other than the drawings.

[0045] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0046] In the case of solar power generation, the efficiency varies depending on the type of panel. For example, between a monocrystalline solar panel and a polycrystalline solar panel, which have different surface flatness (or smoothness) from each other, a monocrystalline solar panel with a relatively high surface flatness shows a higher efficiency. For example, the daily average efficiency of a monocrystalline solar panel is 6.65%, and the daily average efficiency of a polycrystalline solar panel is 5.38%. In addition, even if the surface of the original solar panel is smooth, the efficiency will decrease due to factors such as external damage, foreign matter, and aging. Therefore, it is desirable to identify signs of abnormality in solar panels and repair them quickly in order to maintain appropriate power generation efficiency.

[0047] Figure 1 is a diagram showing various states of a solar panel. For example, Figure 1 , (a) shows an image of an area of ​​the solar panel in good condition, and (b) and (c) show images of an area of ​​the solar panel that is partially scratched. Figure 1 In the figure, (d) and (e) show images of partially broken areas of the solar panel, and (f) shows an image of a partially contaminated area of ​​the solar panel. As described above, in order to maintain the proper efficiency of the solar panel, it is necessary to capture images of the solar panel showing signs of abnormality or having abnormality, analyze the captured images, and process them.

[0048] Previously, solar panel defect detection was performed based on machine learning or deep learning. However, the training process of the machine learning or deep learning method has a high cost (or high computational load) problem because the original high-resolution image is provided to the model for output calculation. In other words, even if the solar panel has a fixed pattern that is easy to understand, these features cannot be utilized, and cost loss occurs due to the use of an expensive learning model (such as deep learning). Therefore, the present invention considers a more concise model than before in order to reduce the computational cost such as the training process to an appropriate level.

[0049] In addition, from the perspective of renewable energy efficiency, the present invention supports performance improvement by additionally reusing a pre-trained model. For example, in the present invention, even if the pre-trained model does not perform learning on the solar panel, a pre-trained attention mechanism is used to detect key information of an image of a solar panel, thereby supporting improvement in defect detection performance. In this process, an attention map extracted by the attention mechanism is used to extract feature information for determining whether a defect exists in each area of ​​the solar panel. The present invention can apply several concise classification methods by using a simple statistical function of the original image or the attention map instead of using the entire pixel value.

[0050] The method of segmenting images to mask defective areas of solar panels is a useful method, but since masks representing defective areas must be constructed for all images in order to learn the segmentation model, it may be expensive in terms of time and manpower. In addition, since it is more economical to replace a defective solar panel than to repair it, it can be appropriately applied to solar panel management by only checking whether a defect has occurred rather than checking the location of the defect in the solar panel.

[0051] In the present invention, an attention mechanism is applied, which is a method to find out where the feature information is located in the image. During the learning process, predefined classes determine the criteria for focusing in the image (or video), but the attention mechanism can provide more comprehensive results with less class dependence than Class Activation Mapping (CAM).

[0052] In order to verify the defect detection technology of solar panels of the present invention, an electroluminescent photovoltaic (ELPV) data set is used, and a high-resolution ELPV image of 300×300 size is used for each sample. In order to simplify the problem in the entire data set, single crystal data is used. At the same time, based on the defect-free solar panel has Figure 1 The defect detection panel has a uniform smooth surface as shown in (a), while the defect detection panel has a non-uniform surface characteristic, and the present invention calculates the standard deviation of the entire image. Considering the influence of the basic grid pattern common to all panels, the standard deviation is measured for each height and width axis, and the measurement results are shown as follows: Figure 2 shown.

[0053] Figure 2 is a graph showing the standard deviation values ​​of the surface flatness of a non-defective solar panel and the standard deviation values ​​of the surface flatness of a defective solar panel. A lower standard deviation indicates a smoother surface, while a higher standard deviation indicates a rougher surface. As shown in the figure, the defective solar panel exhibits a higher standard deviation than the non-defective solar panel. Therefore, in the present invention, by using Figure 2 The standard deviation of each axis of the image in the image is used to distinguish whether there is a defect in the solar panel. In addition, the defect detection performance can be improved by extending the above features (providing defect detection performance through high standard deviation and low standard deviation) to adopt more statistical values ​​that can be extracted from the image.

[0054] Figure 3 is a diagram showing examples of various feature values ​​that can be applied to the present invention. For example, in the present invention, in addition to the standard deviation, the feature values ​​such as Figure 3 The 13 eigenvalues ​​(or characteristic values) shown can improve defect recognition performance.

[0055] Meanwhile, as described above, the attention mechanism associated with defect presence / absence classification can be utilized in the present invention, and for this purpose, neural network training can be applied. In this way, since the present invention does not perform a labeling process for defining defect types for all images, unnecessary costs are not incurred, and the attention mechanism of a model trained on a large-scale public dataset such as the modified National Institute of Standards and Technology database (MNIST) can be used. In the present invention, a simple convolutional neural network (CNN) including an inherent attention mechanism is trained using the MNIST dataset and is assumed to be a pre-trained model, but the present invention is not limited thereto. For example, the dataset used to train the neural network in the present invention may be a large-scale public dataset such as the MNIST dataset or the ImageNet dataset.

[0056] Figure 4 is a diagram showing an example of attention maps for a large-scale public dataset of a pre-trained model.

[0057] Regardless of whether the CNN model is trained on a large-scale public dataset (e.g., the MNIST dataset or the ImageNet dataset), the attention mechanism focuses on the end point, convergence point, or bending point of the journey, so appropriately applying the attention of the focused point to the defective area will obtain an increased standard deviation value of the defective panel area of ​​the solar panel. In addition, the present invention can improve the defect detection performance by applying feature extraction to the attention map. To this end, the present invention extracts an attention map of a captured image of a solar panel as a guide for defects by reusing a pre-trained CNN model. Regarding the attention mechanism of the solar panel, an image of the solar panel is provided to a pre-trained CNN model that is trained to classify handwritten digits to extract an attention map, and a qualitative evaluation is performed.

[0058] Figure 5 is a diagram showing the original solar panel image and the attention map. Figure 5 As shown in Figure 2, the pre-trained CNN model in the attention figure is mainly interested in areas containing non-smoothness and areas where the surface texture is changed due to contamination or mesh, and the defective areas are highlighted as shown in the figure. Figure 5 In FIG. 5 , row 501 shows an image of a non-defective solar panel, and row 502 shows an image of a defective solar panel. Figure 5 In the figure, column (a) shows the original captured image of a part of the solar panel, column (b) shows the attention map of the original captured image, and column (c) shows the image where the original captured image and the attention map are sequentially overlapped. Figure 3 The 13 statistical features described in .

[0059] Figure 6 is a graph showing the correlation coefficient between the extracted features and the label indicating whether the solar panel is defective. As shown in the figure, it can be seen that the correlation between the original image and the attention map is very high.

[0060] Figure 7 is a graph showing the defect detection performance using various learning models. For example, Figure 7 The machine learning models shown are decision trees (DT), random forests (RF), extreme gradient boosting (XGB), and light gradient boosting machines (LGBM). Even when a learning method is employed, these machine learning models have lower computational costs than the case where deep learning is applied. When training and validating the machine learning models, the entire data set is divided and applied at a ratio of 8:2, and performance measurements are performed for each hyperparameter assuming that each model can exhibit various performances depending on the hyperparameter. In the present invention, the area under the receiver operating characteristic curve is used as a defect detection performance indicator for solar panels, and a rule-based model comparing the defect detection performance of 13 eigenvalues ​​is applied without a separate learning method.

[0061] Figure 8 is a graph showing the AUROC values ​​measured in the above-mentioned comparative experiment of defect detection performance. Figure 8 As shown, the maximum AUROC measured from the original data is 0.831, and the AUROC of the attention map is 0.845. From these results, it can be seen that the feature extraction method of the present invention operates at an appropriate level and improves the performance through the attention mechanism of the pre-learning model.

[0062] Fig. 9 is a diagram showing an example of an environment in which a defect detection technology for a solar cell panel according to an embodiment of the present invention is applied.

[0063] Reference Fig. 9 , a defect detection environment 10 for a solar panel according to an embodiment includes an inspection station 300 , a solar panel 11 , an image sensor 200 , and a defect detection device 100 .

[0064] The inspection station 300 includes a mounting station on which the solar panel 11 can be placed. The inspection station 300 also includes a transport device capable of transporting a plurality of solar panels 11. The inspection station 300 including the transport device can be configured to transport the solar panel 11 at a specified speed for capturing an image of the solar panel 11, or can be temporarily stopped during image capture. At least a portion of the inspection station 300 can be aligned vertically and horizontally with the image sensor 200. The inspection station 300 is placed in a flat state for capturing a uniform image of the surface of the solar panel 11. In the case where the inspection station 300 includes the transport device, the inspection station 300 changes the moving path of the solar panel 11 according to whether the solar panel 11 is defective. At this point, the transport device of the inspection station 300 is communicatively connected to the defect detection device 100, and in response to the control of the defect detection device 100, it can load the solar panel where the defect is detected at a different position from the solar panel where the defect is not detected.

[0065] The solar panel 11 has a plurality of solar cells arranged in a specified pattern (e.g., a grid pattern). The solar cell can receive sunlight and generate electricity. After manufacturing, the solar panel 11 is placed on the inspection table 300 and captured by the image sensor 200. The image obtained by capturing the solar panel 11 is provided to the defect detection device 100. Meanwhile, in the above-mentioned function of detecting defects in the solar panel 11, the case of using a monocrystalline solar panel is mentioned, but the present invention is not limited thereto. Alternatively, the defect detection technology of the solar panel of the present invention can also be applied identically or similarly to a polycrystalline solar panel and any other solar panel including a repeated pattern.

[0066] The image sensor 200 includes a capture unit that captures the solar panel 11 placed on the inspection table 300 and a transmission unit that provides the captured image to the defect detection device 100. In addition, the image sensor 200 according to the embodiment of the present invention also includes a moving unit, which can adjust the capture direction of the image sensor 200 under the control of the defect detection device 100. All or at least some components of the image sensor 200 including the capture unit, the transmission unit and the moving unit can be implemented in the form of a software module or a hardware module executed by a processor, or can be implemented in the form of a combination of a software module and a hardware module. The capture unit captures an image in a specified direction. At this time, when the solar panel 11 is positioned at a specified position on the inspection table 300, the capture unit automatically acquires an image of the solar panel 11. Alternatively, when a certain period of time has passed after the solar panel 11 is placed on the inspection table 300, or when the solar panel 11 does not move within a specified period of time, the capture unit acquires the image of the solar panel 11. Alternatively, the capture unit acquires the image of the solar panel 11 according to a predetermined capture cycle. The capture unit includes at least one type of sensor (e.g., at least one of a black and white image sensor, an RGB sensor, a light sensor, a thermal imaging sensor, and a depth sensor) having a resolution capable of detecting at least the surface flatness of the solar panel 11. At the same time, the capture unit can also obtain an image of the solar panel 11 in response to the control of the defect detection device 100. In this case, the transmission unit of the image sensor 200 receives a capture signal from the defect detection device 100 and sends it to the capture unit. To this end, the transmission unit is responsible for interfacing with the defect detection device 100 (e.g., signal transmission and reception). The transmission unit sends a control signal of the defect detection device 100 to the capture unit, and sends the image of the solar panel 11 captured by the capture unit to the defect detection device 100. The moving unit of the image sensor 200 adjusts at least one of the position or the capturing direction of the image sensor 200. The moving unit may include, for example, a moving member capable of moving the image sensor in at least one direction in a horizontal plane, or a flying member capable of moving the image sensor in at least one direction in a vertical plane and a horizontal plane.

[0067] If the mobile unit includes a flying member, the image sensor 200 may be at least a part of the unmanned device. If the mobile unit includes a flying member, the image sensor 200 may obtain a captured image of the solar panel 11 placed at a designated position while moving along a pre-stored moving route, and provide the obtained image to the defect detection device 100. Alternatively, based on the control of an administrator or pre-stored information, the defect detection device 100 transmits information about the moving position or moving route of the image sensor 200 to the mobile unit of the image sensor 200 through a transmission unit. The timing when the image sensor 200 including the mobile unit captures an image of the solar panel 11 by using the capture unit may be after the solar panel 11 is installed at a designated position for photovoltaic power generation. If the target-type image sensor 200 is configured to utilize the captured image of the solar panel 11, the inspection table 300 may be a mounting table provided for photovoltaic power generation of the solar panel 11. The mounting table supports the solar panel 11 and is configured to adjust the mounting direction of the solar panel 11 according to the designed shape.

[0068] The defect detection apparatus 100 acquires a captured image of the solar panel 11 by controlling the image sensor 200, and generates an attention map by applying the acquired captured image to a pre-stored model. The defect detection apparatus 100 compares at least one feature point highlighted in the attention map with a data table (or an abnormal data table) corresponding to a defective panel, thereby determining whether the solar panel 11 corresponding to the attention map has a defect. The defect detection apparatus 100 includes: Fig.10 Configuration shown.

[0069] Fig.10 : is a diagram showing an example of a configuration of a defect detection device for a solar cell panel according to an embodiment of the present invention.

[0070] Reference Fig.10 , the defect detection device 100 may include a communication module 110 , an output module 120 , a model providing unit 130 and a processing module 140 .

[0071] The communication module 110 supports the communication function of the defect detection device 100. For example, the communication module 110 forms a communication channel with the image sensor 200. The communication module 110 sends control information related to controlling the image sensor 200 to the image sensor 200. In addition, the communication module 110 receives an image acquired by the image sensor 200. In another example, the communication module 110 may form a communication channel with a specific server device regarding information collection of the defect detection device 100, and receive information (or data) corresponding to a pre-trained model (e.g., a pre-trained CNN model) from the specific server device. Alternatively, the communication module 110 may receive a large-scale public data set (e.g., an MNIST data set or an ImageNet data set) for generating a pre-trained model from a specific server device. The communication module 110 forms a communication channel with a terminal of an administration or a terminal of a manager (hereinafter, a management terminal) that manages the presence or absence of defects in the solar panel 11. For example, if a defect is detected in a specific solar panel 11, in response to the control of the processing module 140, the communication module 110 sends management information indicating that a defect is detected in the solar panel 11 to the management terminal. At this time, the transmitted management information includes identification information (or serial information) of the solar cell panel where the defect is detected. In addition, the management information may further include at least one of an original captured image and an attention map used in defect detection.

[0072] The output module 120 outputs various information related to the operation of the defect detection apparatus 100. In this regard, the output module 120 includes at least one of an audio device capable of outputting an audio signal and a display device capable of outputting a text or an image. Alternatively, the output module 120 may include a vibration module or a buzzer signal output device capable of notifying that a defect in the solar panel 11 is detected. If the defect detection result in the solar panel 11 is set to be sent to the management terminal, the output module 120 may be omitted. The output module 120 may output at least one of information related to the pre-trained model, information about at least one feature point applied to the pre-trained model, a captured image provided by the image sensor 200, an attention map generated by applying the captured image to the pre-trained model, feature point information extracted from the attention map, and information about the presence or absence of a defect determined by comparing with a comparison value (or reference value) corresponding to a predetermined defective panel. In addition, the output module 120 may output state information of the image sensor 200. For example, the output module 120 may output whether the image sensor 200 is activated, direction information of capturing the image sensor 200 , a preview image collected by the image sensor 200 , location information of the image sensor 200 , and the like.

[0073] The model providing unit 130 stores and manages the pre-trained model, and provides the pre-trained model to the processing module 140 at the request of the processing module 140. Alternatively, the model providing unit 130 generates and stores the pre-trained model from a large-scale public data set (e.g., an MNIST data set or ImageNet) received through the communication module 110. The pre-trained model is a model related to attention map generation, and may include a model generated by applying an attention mechanism to a large-scale public data set. In the model generation process, information about at least one feature point (e.g., at least one of the flatness, mean, kurtosis, and skewness of the panel) may be applied to reduce the amount of calculation in the training process. The feature points applied to generate the pre-trained model may also be used in a subsequent process of determining whether the panel is defective through the attention map.

[0074] The processing module 140 performs transmission of signals, reception and processing of data or signals required for the operation of the defect detection device 100. For example, the processing module 140 controls the collection and storage of pre-trained models. In this regard, the processing module 140 collects a large-scale public data set (e.g., an MNIST data set or an ImageNet data set) or obtains a pre-stored data set from a storage unit, and applies an attention mechanism (e.g., a simple CNN) to the large-scale public data set to prepare a pre-trained model. Alternatively, the processing module 140 receives a pre-trained model from a server device for providing the pre-trained model through the communication module 110, and stores it in the model providing unit 130. With respect to receiving an image about the solar panel 11, the processing module 140 controls the operation of the image sensor 200. For example, the processing module 140 sends control information about the timing of capturing an image of the solar panel 11 to the image sensor 200. When the image sensor 200 is included in at least a portion of the target device, the processing module 140 controls the position of the image sensor 200 to move the image sensor 200 to the position of the solar panel 11 to be inspected for defects. The processing module 140 receives an image of the solar panel 11 acquired by the image sensor 200, and applies the received image to the pre-trained model to extract an attention map. The processing module 140 emphasizes the defective portion of the solar panel 11 through the attention map. In the process, the processing module 140 extracts at least one statistical feature (e.g., the standard deviation value of the surface flatness of the panel) from the attention map. In addition, the processing module 140 extracts at least one of the kurtosis, skewness, and mean value of the surface flatness from the attention map. The processing module 140 determines whether there is a defect based on the extracted statistical feature. In this regard, when a defect occurs in the solar panel 11, the processing module 140 may store and manage the statistical feature value as a comparison value or a reference value. Alternatively, the processing module 140 stores and manages a predetermined standard value or a reference value that can be used to determine the occurrence of a defect in the solar panel 11. In one example, the processing module 140 compares the reference value of the surface flatness of the defective solar panel 11 with the label value of the surface flatness and the defect state of the solar panel extracted from the attention map generated from the currently acquired capture image, and determines the presence or absence of the defect based on the correlation. In this process, if the correlation coefficient between the comparison value and the current value has a high similarity, the processing module 140 determines that the solar panel 11 is defective. In addition, the processing module 140 can improve the defect detection performance by comparing other statistical features (e.g., at least one of the mean value, kurtosis, and skewness of the captured image, or kurtosis and skewness of surface flatness) with the attention map. If the processing module 140 determines that the solar panel 11 is defective, it outputs a message requesting replacement of the solar panel 11.Alternatively, the processing module 140 transmits the result as to whether the solar cell panel 11 is defective to the management terminal through the communication module 110 .

[0075] Fig.11 is an exemplary diagram of a hardware system for implementing a defect detection device according to an embodiment of the present invention.

[0076] refer to Fig.11 The defect detection device 100 may include, for example, a mobile terminal such as a smart phone, a notebook, a desktop, a laptop, a wired or wireless mobile or fixed controller or a server with a user interface. Not limited thereto, any type of device equipped with a computing device and a communication device may be included.

[0077] For reference, in the case where the defect detection apparatus 100 is implemented in the form of a server, it can be implemented in the form of, for example, a web server, a database server, a proxy server, etc. In addition, one or more various software can be installed, which enables a network load balancing mechanism or a service device to operate on the Internet or any other network, and thus can be implemented as a computerized system.

[0078] Here, the network can be an http network, a dedicated line, an intranet or any other network. In addition, each component in the system according to an embodiment of the present invention can be connected to a secure network so that data is not attacked by any hacker or other third party.

[0079] Each component in the above-mentioned defect detection device 100 can be implemented in the form of a software module or a hardware module executed by a processor, or can be implemented in the form of a combination of a software module and a hardware module.

[0080] In this way, the software modules, hardware modules, or a combination thereof executed by a processor may be implemented as an actual hardware system (eg, a computer system).

[0081] In the following, reference will be made to Fig.11 A hardware system 2000 for implementing the defect detection apparatus 100 according to the embodiment of the present invention in the form of hardware is described.

[0082] like Fig.11 As shown, the hardware system 2000 may include a processor 2100 , a memory interface 2200 , and a peripheral device interface 2300 .

[0083] These respective elements in hardware system 2000 may be separate components or integrated into one or more integrated circuits and may be connected by a bus system (not shown).

[0084] Here, a bus system is an abstraction representing any one or more separate physical buses, communication links / interfaces and / or multi-point or point-to-point connections connected by appropriate bridges, adapters and / or controllers.

[0085] The processor 2100 is used to execute various software modules stored in the memory 2210 by communicating with the memory 2210 via the memory interface 2200 , so as to perform various functions in the hardware system.

[0086] In the memory 2210, the above-mentioned Fig.10 The communication module 110, the output module 120, the model providing unit 130 and the processing module 140 in the defect detection apparatus 100 described in the specification and the like may also store an operating system (OS).

[0087] An operating system (e.g., an embedded operating system such as I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) includes various processes, command sets, software components, and / or drivers that control and manage general system tasks (e.g., memory management, storage device control, power management, etc.) and play a role in facilitating communication between various hardware modules and software modules.

[0088] Memory 2210 may include a memory hierarchy including, but not limited to, cache, main memory, and auxiliary memory. The memory hierarchy may be implemented via any combination of, for example, RAM (e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and / or optical storage devices (e.g., disk drives, tapes, compact disks (CDs), digital video disks (DVDs)).

[0089] The peripheral device interface 2300 is used to implement communication between the processor 2100 and peripheral devices.

[0090] The peripheral devices will provide different functions to the hardware system 2000 and may include, for example, a communicator 2310 .

[0091] The communicator 2310 is used to provide a communication function with other devices. To this end, the communicator 2310 may include, for example but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, and a memory, and may also include a known circuit that performs the function.

[0092] The communicator 2310 may support communication protocols, such as WLAN (wireless LAN), DLNA (Digital Living Network Alliance), Wibro (wireless broadband), Wimax (Worldwide Interoperability for Microwave Access), GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), CDMA2000 (Code Division Multiple Access 2000), EV-DO (Enhanced Voice Data Optimized or Enhanced Voice Data Dedicated), WCDMA (Wideband CDMA), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), IEEE802.16, LTE (Long Term Evolution), LTE-A (Long Term Evolution - Advanced), 5G communication system, WMBS (Wireless Mobile Broadband Service), Bluetooth, RFID (Radio Frequency Identification), IrDA (Infrared Data Association), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), USC (Ultrasonic Communication), VLC (Visible Light Communication), Wi-Fi, Wi-Fi Direct, etc. In addition, as a wired communication network, it can include a wired LAN (local area network), a wired WAN (wide area network), PLC (power line communication), USB communication, Ethernet, serial communication, optical / coaxial cable, etc. This is not a limitation, and can include any protocol that can provide a communication environment with other devices.

[0093] In the hardware system 2000 according to the embodiment of the present invention, each component of the defect detection apparatus 100 stored in the memory 2210 in the form of a software module performs an interface with the communicator 2310 via the memory interface 2200 and the peripheral device interface 2300 in the form of a command executed by the processor 2100. Therefore, it is possible to effectively support the adjustment of the mounting structure of the image sensor 200 according to the result of the image consistency determination based on the previously learned alignment model.

[0094] As described above, the various elements in the image sensor operating environment according to the embodiments of the present invention can intuitively support the adjustment of the image sensor installation structure based on the result of the image-to-image consistency determination with respect to the image sensor installed for the purpose of predicting / detecting defects in the focus point of the manufacturing process. Therefore, it can be seen that the image sensor can be installed and operated without a sensor installation expert, thereby maximizing cost efficiency.

[0095] In the following, reference will be made to Fig.12 and Fig.13 Operation methods of components related to a defect detection technique for a solar panel according to an embodiment of the present invention are described.

[0096] Fig.12 is a diagram showing an example of a signal flow between devices for defect detection of a solar cell panel according to an embodiment of the present invention.

[0097] Reference Fig.12 , in the signal flow between components in the system environment related to solar panel defect detection according to an embodiment of the present invention, in step S1201, the defect detection device 100 generates control information. The control information may include various signals related to controlling the image sensor 200 and acquiring a captured image. For example, the control information includes at least one of an on / off control signal of the image sensor 200, an initialization signal of the image sensor 200, a capture direction control signal of the image sensor 200, and a capture timing control signal of the image sensor 200. In the case where the image sensor 200 constitutes at least a part of the target, the control information also includes a position control signal of the image sensor 200 (for example, a signal including movement route information for moving the image sensor 200 to a capture position of at least one solar panel 11). When the solar panel 11 is placed on the inspection table 300 and then captured, the control information includes a transportation control signal of the solar panel 11 and a movement path control signal of a defective panel and a non-defective panel.

[0098] In step S1203, the defect detection apparatus 100 transmits control information to the image sensor 200. In this regard, the defect detection apparatus 100 may form a communication channel with the image sensor 200. The communication channel is formed through at least one of a wired or wireless scheme.

[0099] In step S1205, the image sensor 200 performs image capture based on the control information. For example, the image sensor 200 performs image capture of the solar panel 11 according to the capture cycle included in the control information. In this process, the image sensor 200 may provide the defect detection device 100 with a preview image of the solar panel 11 placed on the inspection table 300, and upon receiving a capture signal for acquiring a captured image from the defect detection device 100, the image sensor 200 may capture an image in response to the received capture signal. Alternatively, when the solar panel 11 is placed at a designated position on the inspection table 300, the image sensor 200 may capture an image of the solar panel 11. Alternatively, the image sensor 200 may check whether the solar panel 11 is moving, and if it is not moving within a designated time, the image sensor 200 may capture an image of the solar panel 11.

[0100] In another example, in the case where the image sensor 200 is provided on at least a portion of the target device, the control information contains the position information of the image sensor 200. Therefore, based on the position information contained in the control information, the image sensor 200 moves to the designated position using the moving member provided in the target device. If the solar cell panel 11 installed at the corresponding position is located within the designated viewing angle, the image sensor 200 captures the image of the solar cell panel 11. Alternatively, when acquiring the preview image, if it is detected that the size of the solar cell panel 11 is larger than the designated size, the image sensor 200 captures the image of the solar cell panel 11. In the case where the control information contains information about a plurality of positions, the image sensor 200 moves and captures the image of the solar cell panel 11 installed at the corresponding position according to the information about the plurality of positions. If the solar cell panel 11 is not placed at the designated position, the image sensor 200 may capture the image at the corresponding position and also provide information indicating that the solar cell panel 11 does not exist to the defect detection device 100.

[0101] In step S1207, the image sensor 200 transmits the captured image to the defect detection device 100. As described above, the captured image includes an image obtained by capturing the solar cell panel 11 including a plurality of solar cells. At this time, the captured image includes at least one of various types of images, such as a black and white image, a color image, and a depth image. In another example, the image sensor 200 may divide the captured image into units of a specified size and sequentially provide the divided images to the defect detection device 100. At this time, the image sensor 200 may divide the captured image into a plurality of solar cell groups and transmit each divided solar cell group to the defect detection device 100.

[0102] In step S1209, the defect detection apparatus 100 applies the captured image to the pre-trained model. The pre-trained model includes a training model that provides a large-scale public data set (e.g., an MNIST data set or an ImageNet data set) to a convolutional neural network to which an attention mechanism is applied. At this point, the defect detection apparatus 100 collects a large-scale public data set and generates a pre-trained model by using a CNN. Alternatively, the defect detection apparatus 100 may receive a pre-trained model from an external device (or a server device that provides a model). In the process of generating a training model, the defect detection apparatus 100 may generate a training model based on information about at least one feature point (e.g., at least one of the standard deviation, mean, kurtosis, and skewness of surface flatness), thereby reducing the amount of computation required for model training.

[0103] In step S1211, the defect detection apparatus 100 determines and outputs whether a defect exists. To this end, the defect detection apparatus 100 applies the captured image provided by the image sensor 200 to the pre-trained model, thereby extracting an attention map. By analyzing the area highlighted by the attention map, the defect detection apparatus 100 can determine whether a defect exists. For example, the defect detection apparatus 100 extracts statistical features of the area highlighted by the attention map, and identifies the correlation between the extracted features and the defect correlation coefficient corresponding to the statistical features (for example, the correlation between the label of the feature of the defective solar panel or the abnormal panel and the statistical feature value of the currently acquired image). The defect detection apparatus 100 determines whether a defect exists based on the similarity of the correlation. If it is determined that the solar panel 11 is defective, the defect detection apparatus 100 outputs a message indicating that a defect has occurred in the corresponding solar panel 11. At this time, using the output module 120, the defect detection apparatus 100 outputs at least one of a text, an image, and an audio signal corresponding to the message. Alternatively, the defect detection apparatus 100 sends the message to an external device (for example, an external speaker or a management terminal) connected through the communication module 110.

[0104] In step S1213, the defect detection apparatus 100 sends a termination confirmation request message to the image sensor 200 to terminate the operation of detecting whether there is a defect in the solar cell panel 11. For example, the defect detection apparatus 100 sends a message to the image sensor to check whether there is an additional solar cell panel to be determined as defective. In step S1215, if there is no additional solar cell panel 11 to be determined as defective, the image sensor 200 notifies the defect detection apparatus 100 of the termination in step S1217. Meanwhile, if there is a solar cell panel to be determined as defective in step S1215, the process returns to step S1205 and re-executes the subsequent operation.

[0105] In step S1213, if the information about the number of solar panels 11 to be inspected or the positions of the solar panels 11 is stored in the defect detection apparatus 100, the defect detection apparatus 100 does not send a termination confirmation request message to the image sensor 200, and sends a message indicating termination to the image sensor 200. In response, the image sensor 200 switches to an inactive state.

[0106] Fig.13 is a diagram illustrating an example of a method of operating a defect detection apparatus in defect detection of a solar cell panel according to an embodiment of the present invention.

[0107] Reference Fig.13In the method for operating a defect detection apparatus related to defect detection of a solar panel according to an embodiment of the present invention, in step S1301, the defect detection apparatus 100 prepares a pre-trained model. The pre-trained model may include a model that applies an attention mechanism (e.g., a convolutional neural network) to a large-scale public dataset (e.g., an MNIST dataset or an ImageNet dataset). When preparing the pre-trained model, the defect detection apparatus 100 may apply at least one feature point (e.g., at least one of a standard deviation, a mean, a kurtosis, and a skewness of surface flatness) to the solar panel 11. The feature point applied when preparing the pre-trained model may be used when determining whether the solar panel 11 is defective.

[0108] In step S1303, the defect detection apparatus 100 checks whether an image is acquired. At this point, the defect detection apparatus 100 forms a communication channel with the image sensor 200 and prepares to receive a captured image of the solar panel 11 from the image sensor 200. In this process, if initialization of the image sensor 200 is required, the defect detection apparatus 100 sends control information related to initialization of the image sensor 200 to the image sensor 200. If no image is acquired, the process returns to step S1301 and waits or re-executes subsequent operations. Alternatively, if no image is acquired within a specified time, the defect detection apparatus 100 may switch to a sleep state or an inactive state.

[0109] In step S1303, if an image is acquired, the defect detection apparatus 100 generates an attention map for the acquired image. To generate the attention map, the defect detection apparatus 100 applies (or matches) the received image to the pre-trained model.

[0110] In step S1305, the defect detection device 100 checks whether the attention map is abnormal. If the attention map is normal, the defect detection device 100 returns to step S1301 and re-executes the subsequent operation. If the attention map is abnormal, the defect detection device 100 outputs a guidance message in step S1309. To this end, the defect detection device 100 pre-stores abnormal data (e.g., a comparison value table corresponding to the characteristics of the attention map in an abnormal state or a label value corresponding to the characteristics of a defective battery panel), and compares the pre-stored abnormal data with the currently acquired attention map. Alternatively, the defect detection device 100 extracts at least one statistical feature from the attention map (e.g., at least one of the standard deviation, mean, kurtosis, or skewness of the surface flatness of the battery panel), and compares the extracted statistical feature with the pre-stored abnormal data (e.g., the statistical feature of the defective battery panel) (e.g., detects correlation).

[0111] Meanwhile, although a structure has been described in which the defect detection device 100 and the image sensor 200 for the solar panel are separated and operated through the communication module 110, the present invention is not limited thereto. For example, at least a portion of the defect detection device 100 that detects defects in the solar panel 11 may be included in the image sensor 200. In this case, an input device for user input related to the operation control of the defect detection device 100 is provided separately, or the management terminal plays this role. In another example, in order to perform defect detection of the solar panel 11 more quickly and comprehensively, a plurality of image sensors 200 are configured to capture images of a plurality of solar panels, and when the defect detection device 100 receives a plurality of images, the defect detection device 100 may perform comparative analysis of the plurality of images simultaneously (in parallel) or in chronological order to quickly process a determination as to whether a defect exists.

[0112] As described above, the defect detection device for solar panels of the present invention and the method for operating the same not only reduce the computational cost in the learning process by using a statistical feature extraction method for determining whether a panel is defective, but also improve the functional information by recycling the pre-trained attention mechanism, so that when applied to defect detection in solar panels, low energy is consumed during the training and inference process of the defect detection model to maintain the renewable power generation efficiency at an appropriate level. For example, the AUROC is improved from a maximum value of 0.631 to 0.823 by recycling the attention mechanism of the model. The present invention defines statistical features in solar panels to reduce the computational load in the training process and uses them to detect defects in solar panels. Here, the present invention can improve the feature value by applying an attention mechanism that guides key information and reusing a pre-trained convolutional neural network that has learned a large-scale public data set (e.g., an MNIST data set or an ImageNet data set).

[0113] At the same time, the functional operations described herein can be implemented as digital electronic circuits, or as computer software, firmware or hardware including the structures disclosed herein and their structural equivalents, or by a combination thereof. The subject matter described herein can be implemented with one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible program storage medium for execution by a processing system or for controlling the operation of a processing system.

[0114] The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination thereof.

[0115] In this disclosure, the term "system" or "device" includes all mechanisms, devices and machines for processing data, including, for example, a programmable processor, a multiprocessor or a computer. In addition to hardware, a processing system may also include code that forms an execution environment for a computer program when requested, such as code constituting processor firmware, a protocol stack, a database management system, an operating system or a combination thereof.

[0116] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, a priori or procedural languages, and may be deployed in any form, including stand-alone programs or modules, components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a single file provided to a requested program, or in multiple interactive files (e.g., one or more modules, subroutines, or files storing code portions), or in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document). A computer program may be deployed to execute on a single computer or multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0117] On the other hand, computer readable media suitable for storing computer program instructions and data may include all forms of non-volatile memory, media and storage devices, including semiconductor memory devices such as EPROM, EEPROM and flash memory devices, magnetic disks such as internal hard disks or external disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated in special purpose logic circuitry.

[0118] Implementations of the subject matter described herein may be implemented in a computing system that includes a back-end component such as a data server, a middleware component such as an application server, a front-end component such as a client computer with a web browser or graphical user interface (through which a purchaser may interact with the implementation of the subject matter described herein), or any combination of one or more of these back-end components, middleware components, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communications network.

[0119] Although this specification contains many specific implementation details, these details should not be interpreted as limiting the scope of any disclosed or potentially claimed content, but should be interpreted as a description of features peculiar to a specific disclosed specific embodiment. Certain features described in the specification in the context of a separate embodiment may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a change in the sub-combination.

[0120] In addition, although this specification describes the operations performed in a predetermined order with reference to the accompanying drawings, it should not be understood that these operations need to be performed sequentially or in a predetermined order, which is shown in order to obtain preferred results, or all the operations shown need to be performed. In some cases, multitasking and parallel processing may be advantageous. In addition, it should not be understood that in all types of implementations, various system components need to be divided. It should be understood that the described program components and systems are usually integrated into a single software product or packaged as multiple software products.

[0121] While the present disclosure has been particularly shown and described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as defined by the appended claims.

[0122] Industrial Applicability

[0123] According to the solar panel defect inspection device and the operation method thereof of the present invention, it is possible to more accurately determine whether a solar panel is defective while using a lower-cost training model. Therefore, the present invention is an invention with industrial applicability because it overcomes the limitations of the prior art and therefore has sufficient potential for commercialization or sales of devices applicable to and usable for related technologies and can be clearly implemented in reality.

[0124] [reference numerals]

[0125] 100: Defect detection device

[0126] 110: Communication module

[0127] 120: Output module

[0128] 130: Model providing unit

[0129] 140: Processing module

[0130] 200: Image sensor

Claims

1. A device for detecting defects of a solar panel, the device comprising: A model providing unit, wherein the model providing unit provides a pre-trained model; a communication module that receives a captured image of the solar panel from the image sensor; as well as a processing module, the processing module being functionally connected to the model providing unit and the communication module, The processing module is configured to collect the captured images of the solar panel, generate an attention map by applying the captured images to the pre-trained model, and determine whether there is a defect in the solar panel according to whether the attention map is abnormal.

2. The device according to claim 1, wherein: The processing module applies an attention mechanism including convolutional neural network training to a large-scale public dataset.

3. The device according to claim 1, wherein: The processing module is configured to extract at least one statistical feature from the attention map, detect a correlation between the extracted statistical feature and a label value indicating a defective solar panel, and determine whether the solar panel is defective based on a degree of the correlation.

4. The device according to claim 3, wherein: The processing module is configured to extract a standard deviation of surface flatness of the solar panel from the attention map, and detect a correlation between the standard deviation of surface flatness of the defective solar panel and the standard deviation of surface flatness extracted from the attention map.

5. The device according to claim 4, wherein: The processing module is configured to extract at least one of a mean value of the surface flatness, a kurtosis of the surface flatness, and a skewness of the surface flatness, and detect a correlation between the extracted at least one of the mean value, the kurtosis, and the skewness and at least one of the mean value, the kurtosis, and the skewness in a map generated from an image of the defective solar panel.

6. The device according to claim 1, further comprising: An output module outputs a guidance message indicating whether the solar cell panel is defective.

7. A method for operating a defect detection device for a solar panel, the method comprising the steps of: Prepare pre-trained models; collecting captured images of the solar panel; generating an attention map by applying the captured image to the pre-trained model; and Whether there is a defect in the solar cell panel is determined according to whether the attention map is abnormal.

8. The method according to claim 7, wherein: The step of preparing the pre-trained model comprises the following steps: applying an attention mechanism including convolutional neural network training to a large-scale public dataset.

9. The method according to claim 7, wherein: The steps to determine if a defect exists include the following: extracting at least one statistical feature from the attention map; detecting a correlation between the extracted statistical features and a label value indicative of a defective solar panel; and Whether the solar cell panel is defective is determined based on the degree of the correlation.

10. The method according to claim 9, wherein: The step of extracting the statistical features comprises the steps of: extracting the surface flatness of the solar panel from the attention map, and The step of detecting the correlation includes the step of detecting a correlation between a surface flatness of the defective solar cell panel and a surface flatness extracted from the attention map.

11. The method according to claim 10, wherein: The step of extracting the statistical feature comprises the steps of: extracting at least one of the average value of the surface flatness, the kurtosis of the surface flatness, and the skewness of the surface flatness, and The step of detecting the correlation includes the step of detecting a correlation between the extracted at least one of the mean, kurtosis, and skewness and at least one of the mean, kurtosis, and skewness in a map generated from the image of the defective solar panel.

12. The method according to claim 7, further comprising the steps of: A guidance message indicating whether the solar cell panel is defective is output.

Images