Device for inspecting solar cell panel

Abstract

To provide a device for inspecting a solar cell panel which is capable of highly accurate inspection operation.SOLUTION: Provided is a device for inspecting solar cell panels, with which it is possible to generate a non-defective article learning model by unsupervised learning using a non-defective article data group including image data of solar cell panels determined to be non-defective in the past and not including image data of solar cell panels not determined to be non-defective in the past, as well as generate a defect classification model by supervised learning using a data set of the past image data of solar cell panels and data related to the presence or absence of defects in the image data. When the solar cell panel of an inputted image is determined to be defective by determination using the non-defective article learning model and not determined to have a defect by determination using the defect classification model, the solar cell panel of the input image is classified as a solar cell panel impossible to be determined.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an inspection device for a solar cell panel. [Background technology]

[0002] At solar cell panel manufacturing sites, devices are used to inspect and detect defects in solar cell panels so that the manufactured solar cell panels maintain a stable level of quality. One such device is the detection device disclosed in Patent Document 1.

[0003] The detection device of Patent Document 1 has a candidate area extraction unit, an image generation unit, a first learning device, and a detection unit. The detection device of Patent Document 1 uses the candidate area extraction unit to process a first image to extract a candidate area containing a detection target candidate, and uses the image generation unit to generate a second image containing the candidate area. Furthermore, the detection device inputs the generated second image to a first learning device consisting of a neural network to detect the presence or absence of the defect that is the detection target.

[0004] This detection device detects defects such as broken fingers, cleaning marks, and cracks in the cells that make up a solar cell module. In addition, this detection device configures the first learning device with a multi-layer neural network (deep learning), and furthermore, the second image input to the first learning device contains only a small number of candidate regions, making it possible to detect minute targets that are difficult to detect with high accuracy. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2020 / 158635 Summary of the Invention [Problem to be solved by the invention]

[0006] In conventional devices, the accuracy of defect detection may be reduced if the materials or manufacturing process of the solar cell panel being manufactured is changed. In such cases, engineers must perform maintenance on the equipment to adapt it to changes in materials or manufacturing processes and improve the accuracy of defect detection.However, it is difficult to get engineers to perform maintenance on the equipment immediately every time a change is made to the materials or manufacturing process.

[0007] In other words, there is a problem of a shortage of top engineers who are familiar with both solar panel manufacturing and information technology and have the advanced knowledge and skills to perform maintenance properly. Furthermore, if such top engineers were to perform maintenance, the maintenance costs of the equipment would increase significantly. For these reasons, there has been a demand for a solar panel inspection device that can be easily operated continuously even when the materials or manufacturing process of solar panels are changed, without necessarily requiring immediate response by engineers. Furthermore, conventional detection devices have room for improvement in terms of performing highly accurate inspections.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a solar cell panel inspection device capable of performing highly accurate inspection operations. [Means for solving the problem]

[0009] One aspect of the present invention for solving the above problem includes an image input unit that inputs image data of a solar cell panel, a first learning model generation unit, a second learning model generation unit, a first judgment unit, a second judgment unit, and a classification unit, wherein the first learning model generation unit is capable of generating a good product learning model by unsupervised learning using a group of good product image data that includes image data of solar cell panels that have been judged to be good products in the past and does not include image data of solar cell panels that have not been judged to be good products in the past, and the first judgment unit uses the good product learning model to judge whether the solar cell panel in the image input unit is good products or not, and the second learning model generation unit generates ... second judgment unit generates a good product learning model by unsupervised learning using the good product learning model. The model generation unit is capable of generating a defect classification model through supervised learning using a dataset of past image data of solar cell panels and data regarding the presence or absence of defects in the image data, and the second judgment unit uses the defect classification model to determine whether or not the solar cell panel in an image input to the image input unit has a defect, and the classification unit classifies the solar cell panel in the input image as an indeterminable solar cell panel when the input image input to the image input unit is judged to be defective by the first judgment unit and not judged to have a defect by the second judgment unit, in an inspection device for solar cell panels.

[0010] The solar panel inspection device of this aspect performs judgment using both a good product learning model and a defect classification model, which are generated using different learning methods. If the first judgment unit judges a solar panel as defective and the second judgment unit judges it as not defective, the solar panel in the input image is classified as an unclassifiable solar panel. That is, if judgment were performed using only one model generated by one learning method, a solar panel would be judged as good without being judged as defective, but according to this aspect, it is classified as unclassifiable. This enables improved accuracy compared to judgment using only one model generated by one learning method. More specifically, judgment using a defect classification model generated by supervised learning enables highly accurate judgment of defective products due to known defects, while judgment using a good product learning model generated by unsupervised learning using a group of good product image data enables highly accurate judgment of defective products due to unknown defects. Furthermore, enabling such high-precision inspections prevents accuracy from decreasing below a certain level due to disturbances such as changes in materials or manufacturing processes.

[0011] Preferably, the good product learning model has a comparison image creation unit that creates a comparison image based on the results of the unsupervised learning, creates an error image from the difference between the input image input to the image input unit and the comparison image, applies a sigmoid function or binarization processing to the error image to create a processed image, and further uses the processed image to determine whether the solar cell panel is good product.

[0012] According to this aspect, the accuracy of the judgment using the non-defective product learning model can be further improved, and the inspection operation can be made more accurate.

[0013] Preferably, the unsupervised learning uses at least one framework selected from the group consisting of an autoencoder, a variational autoencoder, and a generative adversarial network.

[0014] Preferably, the group of non-defective image data does not include images of solar cell panels that have been previously determined to be non-defective and that do not meet predetermined standards for non-defectiveness.

[0015] Preferably, the group of non-defective image data includes image data obtained by performing gamma correction or product-sum calculation on images of solar cell panels that have been previously determined to be non-defective.

[0016] In these aspects, the accuracy of judgment using the non-defective product learning model can be further improved, and the inspection operation can be made more accurate.

[0017] Preferably, when the input image input to the image input unit is determined to be a non-defective product by the first determination unit, the classification unit classifies the solar cell panel of the input image as a non-defective solar cell panel.

[0018] According to this aspect, it is possible to increase the accuracy of sorting into non-defective solar cell panels.

[0019] Preferably, when the input image input to the image input unit is determined by the second determination unit to have a defect, the classification unit classifies the solar cell panel of the input image as a defective solar cell panel.

[0020] According to this aspect, it is possible to increase the accuracy of separating defective and non-defective solar cell panels.

[0021] Another aspect of the present invention is a solar cell panel inspection device that includes an image input unit that inputs image data of a solar cell panel, a first learning model generation unit, and a first judgment unit, wherein the first learning model generation unit is capable of generating a good product learning model through unsupervised learning using a group of good product image data that includes image data of solar cell panels that have been previously judged to be good products, but does not include image data of solar cell panels that have not previously been judged to be good products, and the first judgment unit uses the good product learning model to judge whether the solar cell panel of an image input to the image input unit is good product or not, and the good product learning model has a comparison image creation unit that creates a comparison image based on the results of the unsupervised learning, creates an error image from the difference between the input image input to the image input unit and the comparison image, applies a sigmoid function or binarization processing to the error image to create a processed image, and further uses the processed image to judge whether the solar cell panel is good product or not.

[0022] According to this aspect, a processed image is generated based on the error image, and the processed image is used to determine whether the solar panel is a good product. This makes it possible to identify defective objects with higher accuracy, thereby improving inspection accuracy.

[0023] The above aspects may be made dependent on each other, or some of the configurations may be quoted or substituted for each other, as long as they are included in the technical scope of the present invention. [Effects of the Invention]

[0024] According to the present invention, it is possible to provide a solar cell panel inspection device capable of performing highly accurate inspection operations. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a block diagram showing an inspection device according to an embodiment of the present invention. [Figure 2](a) is an image showing an example of a cell image classified as non-defective on the right, and the left is an explanatory diagram tracing the right image. (b) is a comparison image generated from the non-defective cell image of (a) on the right, and the left is an explanatory diagram tracing the right image. (c) is an error image generated based on the non-defective cell image of (a) and the comparison image of (b), and the left is an explanatory diagram tracing the right image. (d) is a processed image generated based on the error image of (c), and the left is an explanatory diagram tracing the right image. [Figure 3] (a) is an image showing an example of a cell image containing a defect object on the right, and the left is an explanatory diagram tracing the right image. (b) is a comparison image generated from the cell image of (a) on the right, and the left is an explanatory diagram tracing the right image. (c) is an error image generated based on the cell image of (a) and the comparison image of (b) on the right, and the left is an explanatory diagram tracing the right image. (d) is an explanatory diagram showing a processed image generated based on the error image of (c) on the right, and the left is an explanatory diagram tracing the right image. [Figure 4] 1 is an explanatory diagram showing a machine learning unit that generates a non-defective learning model as the first learning model generation unit of FIG. [Figure 5] FIG. 5 is an explanatory diagram schematically showing how a solar cell in an input image is determined to be a non-defective or defective product using the non-defective product learning model generated by the machine learning unit in FIG. 4. [Figure 6] 1. FIG. 4 is an explanatory diagram showing a machine learning section by which the second learning model generation section of FIG. 1 generates a defect classification model. [Figure 7] 2 is a flowchart showing an inspection operation performed by the inspection device of FIG. 1. [Figure 8] 1A and 1B are explanatory diagrams showing solar cell panels that can be inspected by an inspection device according to an embodiment of the present invention, where (a) and (b) show different solar cell panels. [Figure 9] FIG. 5 is an explanatory diagram showing a machine learning unit different from that shown in FIG. 4. [Figure 10] 10 is an explanatory diagram schematically showing how a solar cell in an input image is determined to be a non-defective or defective product using the non-defective product learning model generated by the machine learning unit in FIG. 9. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, embodiments of the present invention will be described in detail.

[0027] 1, an inspection device 1 (solar panel inspection device) according to an embodiment of the present invention has a control unit 2, a memory unit 3, a communication unit 4, an input unit 5, and a display unit 6. Furthermore, this inspection device 1 has, as its main components, an image input unit 7, a first learning model generation unit 8, a second learning model generation unit 9, a first judgment unit 10, a second judgment unit 11, and a classification unit 12.

[0028] The control unit 2 is mainly composed of a CPU (Central Processing Unit) that performs various arithmetic processing, a ROM that is the main memory, and a RAM for temporarily storing various data, and comprehensively controls each part that makes up the inspection device 1. In other words, the CPU performs arithmetic processing using the programs and data stored in the ROM and RAM, thereby controlling each part and realizing each function.

[0029] The storage unit 3 has a storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and stores various information including databases and various programs.

[0030] The communication unit 4 has a communication I / F, which means that the inspection device 1 can send and receive information (data, signals) to and from external devices via a network. The input unit 5 is mainly composed of input devices such as a keyboard and a mouse, and receives input to the inspection device 1 from the user. The display unit 6 is configured with a display device such as a display, and displays various types of information.

[0031] The image input unit 7 is a part into which image data is input, and is realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in the ROM, RAM, etc. More specifically, the image input unit 7 is a part into which image data used to construct each model, which will be described later, and image data used to inspect the solar cell panel, which will be described later, are input.

[0032] In this embodiment, the image data input to the image input unit 7 is image data formed by photographing a part or the whole of the solar cell panel with a photographing means such as a camera, and is a cell image obtained by photographing the solar cell constituting the solar cell panel. In detail, the image data input to the image input unit 7 is a cell image obtained by photographing the solar cell obtained by emitting light (see Fig. 2(a), Fig. 3(a), etc.).

[0033] That is, the image data of this embodiment is generated by passing a current through a solar cell panel to make it emit light, and then photographing the surface of the light-emitting part of the solar cell panel with a camera, similar to image data used in EL (electroluminescence) inspection. More specifically, the image data of this embodiment is image data obtained by passing a current through the solar cell to make the solar cell itself emit light, and then photographing the surface of the solar cell with a camera.

[0034] The first learning model generation unit 8 is a part that generates the good product learning model 20 (see Figure 5), and is realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in ROM, RAM, etc. In detail, the first learning model generation unit 8 has a function of learning by so-called unsupervised learning, and is capable of performing unsupervised learning in accordance with a predetermined method. In this embodiment, a generative adversarial network (GAN) is adopted as the unsupervised learning method performed by the first learning model generation unit 8, and more specifically, an efficient GAN is adopted. That is, the first learning model generation unit 8 performs unsupervised learning using a generative adversarial network framework or a framework based thereon. The unsupervised learning method may be an autoencoder or a variational autoencoder. That is, unsupervised learning may be performed using an autoencoder framework, a variational autoencoder framework, or a framework based on them. In other words, unsupervised learning may be performed using at least one framework selected from the group consisting of an autoencoder, a variational autoencoder, and a generative adversarial network.

[0035] Here, "unsupervised learning" is a technique for learning features of training data by providing the data without results. In this embodiment, as shown in Fig. 4, by providing learning data to the machine learning unit, bidirectional learning is performed in which the unit learns to generate an image from noise and simultaneously learns to generate noise from an image. That is, the generator unit, which is a network that generates an image, and the discriminator unit, which is a network that determines whether the image created by the generator unit is real or fake, compete to learn. That is, the machine learning unit has a noise generation unit, a generator unit, an image data input unit, an encoder unit, and a discriminator unit. Then, based on the noise generated by the noise generation unit, the generator unit creates a fake image, and a pair of the noise and the fake image generated based on the noise is input to the discriminator unit. Meanwhile, a real image, which is learning data, is input to the image data input unit and input from the image data input unit to the encoder unit. Furthermore, the encoder unit generates noise based on the real image. Then, a pair of the noise generated based on the real image and the real image is input to the discriminator unit. The discriminator unit determines whether the input pair of noise and image is a pair of real images or a pair of fake images.

[0036] Here, the first learning model generation unit 8 of this embodiment uses a group of non-defective image data as a dataset, learns features in the dataset, and generates a non-defective learning model 20, which is a machine learning model.

[0037] Here, the non-defective image data group is a group consisting of a plurality of cell images, and is a group of cell images (hereinafter also referred to as non-defective cell images) of solar cells previously determined to be non-defective. In other words, the non-defective image data group does not include cell images (hereinafter also referred to as defective cell images) of solar cells previously determined to be non-defective.

[0038] More specifically, a good cell image is an image in which a predefined defect object is not included in the image, and a bad cell image is an image in which a predefined defect object is included in the image. Defects include, for example, broken wires (finger broken wires), water marks (marks left after cleaning), various cracks of different shapes (transverse cracks, cross cracks, right-angle cracks, leaf-vein cracks, parallel cracks, etc.), and single cell abnormalities. A single cell abnormality (cell abnormality) is a defect in which the solar cell does not emit light or emits a significantly reduced amount of light. That is, a defective cell image of a solar cell with a break or a solar cell with water marks will have black areas indicating the break or water marks. Also, a defective cell image of a solar cell with a crack will have black lines indicating the crack. Furthermore, a defective cell image of a solar cell with a cell abnormality will have, for example, an entirely black image or an image in which the white light-emitting areas are significantly darker than normal.

[0039] Furthermore, the non-defective image data group of this embodiment does not include, among the non-defective cell images, cell images (images of solar cell panels) that do not meet predetermined non-defective standards. More specifically, in this embodiment, the good cell images are not included if they have no parts that are defect targets and have a specified number (e.g., three) or more parts that are similar to the defect targets (parts that are suspected of being defect targets). Note that the good cell images included in the good image data group of this embodiment are those that have no parts that are defect targets and have no parts that are similar to the defect targets (there are zero parts that are similar to the defect targets). In other words, the good cell images in the good image data group are not included if they are close to being defective. Furthermore, the good cell images included in the good image data group may be those that have no parts that could be defective and have a small distribution of brightness among the solar cell images. In other words, solar cell images with a large distribution of brightness among the light-emitting parts of the solar cell in the image (those with a large variation in the brightness of each part in the light-emitting part) may be considered to be good products that are close to being defective, and good cell images that are close to being defective may not be included.

[0040] Furthermore, in this embodiment, one or more non-defective cell images (part or all of the non-defective cell images) belonging to the non-defective image data group are images that have been subjected to image correction processing. In particular, one or more non-defective cell images of this embodiment are subjected to image correction processing in which the contrast of the image is adjusted by gamma correction. Note that the image correction process may be performed using a product-sum operation instead of gamma correction. That is, when the pixel value at the position (x, y) of the image is src(x, y), the brightness and / or contrast may be changed using the following formula (1): dst(x,y)=αsrc(x,y)+β···(1) Note that α and β are constants, with α representing contrast and β representing brightness. Therefore, changing the value of α changes the contrast, and changing the value of β changes the brightness. It is preferable to increase the contrast and brightness of all good cell images belonging to the good image data group in order to improve the accuracy of the generated good learning model 20.

[0041] 5, the good product learning model 20 is a model used to determine (estimate) whether a solar cell (solar panel) represented by input image data (hereinafter also referred to as an input image) is a good product or a defective product. In detail, the good product learning model 20 has a comparison image creation unit 21 and an error image creation unit 22, and generates a comparison image and an error image based on the input image.

[0042] The comparison image creation unit 21 is part (or all) of the generator unit (image generator) realized by the unsupervised learning described above, and generates a comparison image (see, for example, FIG. 2(b)) based on an input image (see, for example, FIG. 2(a)). In detail, the comparison image creation unit 21 generates a comparison image based on noise generated from the input image. This comparison image creation unit 21 is an image generator that has been trained so that the comparison image it generates exhibits the characteristics of the good cell image used as the learning image. Therefore, the generated comparison image is as if the defective portion or a portion similar to the defective portion had been removed from the input image. For example, if the cell image shown in Figure 2(a) is the input image, an image is generated as a comparison image (see Figure 2(b)) from which parts similar to the defective object (parts that may be mistaken for defective objects) have been removed. As another example, if the cell image shown in Fig. 3(a) is the input image, the image shown in Fig. 3(b) is generated as the comparison image. In this case as well, it can be seen that an image is generated in which parts similar to the defective object (parts that may be mistaken for defective objects) have been removed.

[0043] The error image generating unit 22 generates an error image from the difference between the input image and the comparison image generated based on the input image by the comparison image generating unit 21. Therefore, the generated error image is an image in which the part of the input image that is the defect object or a part similar to the defect object is emphasized. As an example, when the cell image shown in Figure 2(a) is used as the input image, an error image (see Figure 2(c)) is formed based on the input image (see Figure 2(a)) and a comparison image (see Figure 2(b)) generated based on the same input image. As another example, when the cell image shown in Figure 3(a) is used as the input image, an error image (see Figure 3(c)) is formed based on the input image (see Figure 3(a)) and a comparison image (see Figure 3(b)) generated based on the same input image.

[0044] The second learning model generation unit 9 is a part that generates a defect classification model, and is realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in the ROM, RAM, etc. In detail, the second learning model generation unit 9 generates a defect classification model through supervised learning using a data set of past image data and data on the presence or absence of defects in the past image data. Here, "supervised learning" refers to the process of providing a large amount of training data, i.e., a set of data consisting of certain inputs (explanatory variables) and results (target variables), to a machine learning unit, which then learns the characteristics of these data sets and inductively acquires a model (error model) that estimates the results from the inputs, i.e., the relationship between the inputs and the results. In other words, as shown in Figure 6, the machine learning unit has an input unit and an output unit, and by performing machine learning based on training data, it is possible to construct a machine learning model that calculates the objective variable output from the output unit from the explanatory variable input to the input unit.

[0045] In this embodiment, the past image data is used as training data, and data regarding the presence or absence of defects in the past image data is used as result data. The past image data is a group of cell images of solar panel cells that have been previously determined to be good or bad. In other words, the past image data is a group that includes both good cell images and defective cell images. The data regarding the presence or absence of defects in the past image data is data indicating the past judgment results associated with each cell image belonging to the past image data. For example, the first cell image belonging to the past image data corresponds to a defect-free, non-defective cell image, the second cell image corresponds to a defect-defective cell image, etc., and is data indicating the past judgment results of each cell image belonging to the past image data.

[0046] More specifically, the second learning model generation unit 9 generates a defect classification model using a convolutional neural network (CNN). The defect classification model includes an input layer, an output layer, and one or more intermediate layers (hidden layers) between them. The intermediate layers in this embodiment include multiple convolutional layers, multiple pooling layers, and a fully connected layer. From the viewpoint of preventing overfitting, it is preferable to provide a dropout layer and / or a batch normalization layer in the convolutional neural network. Also, from the viewpoint of preventing overfitting, it is preferable to set the number of layers of the convolutional neural network to 7 to 11. To explain in more detail, for example, let us say that the intermediate layers of a convolutional neural network are eight layers, from the first layer to the eighth layer. In this case, let us say that the first layer to the sixth layer are layers that execute the processes consisting of convolution, batch normalization, activation (a nonlinear function, e.g., relu), and max pooling in this order. Furthermore, let us say that the seventh layer is a layer that executes the processes consisting of convolution, batch normalization, and activation (a nonlinear function, e.g., relu) in this order. Furthermore, let us say that the eighth layer is a layer that executes the processes consisting of convolution (a filtering process), batch normalization, and average pooling in this order.

[0047] From the above, when an input image is input, the defect classification model outputs the certainty of the inference that the solar cell shown in the input image is a good product and the certainty that it is a defective product. In other words, it outputs the probability that the solar cell shown in the input image belongs to each of the two categories, "good product" and "defective product." To give a specific example of the output result, the defect classification model outputs results such as good (0.97) and bad (0.03). In other words, the solar cell shown in the input image has a 97 percent probability of being good and a 3 percent probability of being bad. The defect classification model of this embodiment is an EDL (Evidential Deep Learning) model formed by applying the content disclosed in Murat Sensoy, Lance Kaplan, Melih Kandemir, "Evidential Deep Learning to Quantify Classification Uncertainty," 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, Canada.

[0048] The first judgment unit 10 is a part that uses the above-mentioned good product learning model 20 to judge whether the solar cell (solar panel) in the input image is a good product or not, and is realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in ROM, RAM, etc. In detail, the first determination section 10 has a processed image creation section 23 (see FIG. 5) and a determination section 24 (see FIG. 5).

[0049] The processed image creation unit 23 generates a processed image by performing a sharpening process on the error image based on the error image. Note that the "sharpening process" here refers to a process of making an image sharp by emphasizing changes in pixel values ​​(changes in shading) of the image. That is, in this embodiment, an image in which the luminance weight of the error image is emphasized is generated as the processed image. In this embodiment, a sigmoid function is used for the sharpening process. That is, if the brightness is greater than a reference value, the image is made brighter, and if it is less than a reference value, the image is made darker, and the amount of change is calculated using the sigmoid function. As an example, by performing a sharpening process on an error image (see Figure 2(c)) generated based on the cell image shown in Figure 2(a), a processed image (see Figure 2(d)) with enhanced brightness is formed. As another example, when a sharpening process is performed on an error image (see Figure 3(c)) generated based on the cell image shown in Figure 3(a), a processed image (see Figure 3(d)) with enhanced brightness is similarly formed. The sharpening process by the processed image creation unit 23 may be configured to use binarization instead of the sigmoid function described above. That is, the error image may be converted into black and white (0 and 1) based on a predetermined threshold value.

[0050] The determination unit 24 executes an operation of determining whether the solar cell panel is a non-defective product or a defective product using the processed image generated by the processed image generation unit 23. In detail, the determination unit 24 determines that the solar cell panel is a non-defective product when the defect score (abnormality score) of the processed image is less than a predetermined threshold (first threshold), and determines that the solar cell panel is a defective product when the defect score is equal to or greater than the threshold. The defect score is a value that quantifies the degree of defect in the target image (processed image). The defect score in this embodiment is a value calculated based on the proportion of the area of ​​parts with a brightness higher than a certain level in the image and the brightness of the parts with a brightness higher than a certain level, and is calculated using a predetermined formula. For example, the defect score value increases as the proportion of the area of ​​parts with a brightness higher than a certain level increases. Furthermore, even if the proportion of the area of ​​parts with a brightness higher than a certain level is the same, the defect score value increases as the brightness of the parts with a brightness higher than a certain level increases. In this embodiment, the brightness value indicated by 256 levels (256 gradations) of each pixel constituting the processed image is acquired, and the total value of these brightness values ​​is used.

[0051] From the above, in the judgment operation by the first judgment unit 10 using the good product learning model 20, the good product learning model 20 generates a comparison image and an error image based on an input image, as shown in Fig. 5. Then, the first judgment unit 10 generates a processed image based on the error image generated by the good product learning model 20, and calculates a defect score for the processed image. Then, the first judgment unit 10 judges (estimates) whether the solar cell shown in the input image (the solar cell shown in the input image) is a good product or a defective product based on the calculated defect score. For example, if the input image is a cell image (reference) such as that shown in Figure 2(a), a processed image without (or almost without) high brightness areas, such as that shown in Figure 2(d), is generated. In this case, the defect score will be 0 (or a value close to 0), and the solar cell in the input image will be determined to be a non-defective product. On the other hand, if the input image is a cell image (reference) such as that shown in Figure 3(a), a processed image with some areas of high brightness will be formed, as shown in Figure 3(d). In this case, the defect score will exceed a certain value, and the solar cell in the input image will be determined to be defective.

[0052] Here, each functional unit of the inspection device 1 of this embodiment may be implemented individually or in part on different hardware, as long as the inspection device 1 as a whole has each functional unit. For example, the first determination unit 10 and the second determination unit 11 may be implemented on different hardware. Note that the functional unit is, for example, a function realized by a processor reading a program stored in a memory. Therefore, as described above, the first determination unit 10 is not limited to a configuration including the processed image creation unit 23 and the determination unit 24, as long as the inspection device 1 includes the processed image creation unit 23 and the determination unit 24 as its main components. That is, the first determination unit 10 may be a functional unit that calculates a defect score based on a processed image generated by the processed image creation unit 23, which is an external functional unit, and determines whether the product is good or defective. Furthermore, the first determination unit 10 may be a functional unit in which the generation of the processed image and the calculation of the defect score are performed by an external functional unit (the processed image creation unit 23 and a score calculation unit that calculates the defect score), and the determination is made based on the defect score calculated by the external functional unit.

[0053] The second judgment unit 11 is a part that uses the above-mentioned defect classification model to judge whether or not there is a defect in the solar cell (solar panel) in the input image, and is realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in the ROM, RAM, etc. In this embodiment, if the output result of the defect classification model indicates that the product is defective, the solar cell in the input image is determined to have a defect if the probability is greater than or equal to a predetermined threshold (second threshold), and if the probability is less than the threshold, the solar cell in the input image is determined to have no defect.

[0054] The classification unit 12 is a part that classifies whether the solar cell (solar panel) in the input image is a good product, a defective product with a known defect, or an imperfect good product or a defective product with an unknown defect, based on the respective judgment results of the first judgment unit 10 and the second judgment unit 11. This classification unit 12 is also realized by the CPU of the control unit 2 performing arithmetic processing using programs and data stored in the ROM, RAM, etc.

[0055] Next, the solar cell panel inspection operation performed by the inspection device 1 of this embodiment will be described in detail. The inspection operation of this embodiment classifies whether the solar cell panel being inspected is a non-defective product, a defective product with a known defect, or an imperfectly non-defective product or a defective product with an unknown defect.

[0056] In the inspection operation, image data showing the solar cell panel to be inspected is input to the image input unit 7. In this embodiment, cell images of solar cells belonging to the solar cell panel to be inspected are input to the image input unit 7. If there are multiple solar cells belonging to the solar cell panel to be inspected, cell images corresponding to each solar cell are input to the image input unit 7 as input images, and inspection is performed on each solar cell.

[0057] In the inspection operation, as shown in FIG. 7, the first judging unit 10 of the inspection device 1 judges whether or not the solar cell in the input image is a non-defective product using the non-defective product learning model 20 (STEP 1). Then, if the solar cell is determined to be a non-defective product as a result of the determination using the non-defective product learning model 20 (Yes in STEP 2), the classification unit 12 of the inspection device 1 determines that the solar cell in the input image is a non-defective product (STEP 3). That is, the classification unit 12 classifies the solar cell panel (solar cell) in the input image as a non-defective solar cell panel.

[0058] On the other hand, if the solar cell is judged to be defective as a result of the judgment using the good product learning model 20 (No in STEP 2), the second judgment unit 11 of the inspection device 1 uses the defect classification model to judge whether or not the solar cell in the input image is defective (STEP 4). Then, if it is determined that the solar cell is defective as a result of the determination using the defect classification model (Yes in STEP 5), the classification unit 12 of the inspection device 1 classifies the solar cell in the input image as a defective product (STEP 6). That is, the classification unit 12 classifies the solar cell panel (solar cell) in the input image as a defective solar cell panel.

[0059] On the other hand, if the result of the judgment using the defect classification model indicates that the solar cell is not defective (No in STEP 5), the classification unit 12 of the inspection device 1 classifies the solar cell in the input image as an imperfect, non-defective product or a defective product having an unknown defect. That is, if the solar cell is determined to be a defective product in the judgment using the non-defective product learning model 20 and is determined to be free of defects in the judgment using the defect classification model, the classification unit 12 classifies it as an imperfect, non-defective product or a defective product having an unknown defect. In addition, a solar cell panel (solar cell) classified as "an imperfect product or a defective product with an unknown defect" can also be said to be an undeterminable solar cell (undeterminable solar cell panel) that cannot be determined as being a good product or a defective product. In other words, if the result in STEP 5 above is No, the classification unit 12 classifies the solar cell as an undeterminable solar cell (undeterminable solar cell panel).

[0060] In the above-described embodiment, the judgment using the non-defective product learning model 20 and the judgment using the defect classification model may be performed in order. That is, the judgment using the defect classification model may be performed before the judgment using the non-defective product learning model 20. In detail, when judgment using the defect classification model is performed first, if the judgment determines that there is a defect, the solar cell panel in the input image is classified as a defective solar cell panel, and if it determines that there is no defect, judgment is performed using the good product learning model 20. Then, if it is determined to be a good product, the solar cell panel in the input image is classified as a good solar cell panel, and if it is determined to be a defective product, the solar cell panel in the input image is classified as an unclassifiable solar cell panel. That is, in this case too, if the judgment using the good product learning model 20 determines that the solar cell panel is defective and the judgment using the defect classification model determines that there is no defect, it is classified as an unclassifiable solar cell panel.

[0061] Here, when operating the above-described inspection device 1, if there is image data containing a defect that was detected by the non-defective product learning model 20 but not detected by the defect classification model, it is preferable to use this image data to perform re-learning by the second learning model generation unit 9 so that the defect classification model can detect this defect. In other words, if a decrease in the accuracy of the trained defect classification model is predicted, it is preferable to automatically perform re-learning by the second learning model generation unit 9. This makes it possible to further improve the accuracy of the inspection operation.

[0062] Furthermore, when operating the above-mentioned inspection device 1, it is preferable to use image data classified as an unclassifiable solar cell panel for re-learning in the first learning model generation unit 8 and / or the second learning model generation unit 9. That is, it is preferable to manually verify the image data of the solar panel classified as unclassifiable, to determine whether it is a good or bad product, and use this for re-learning in the first learning model generation unit 8 and the second learning model generation unit 9. It is preferable to update the good product learning model 20 and the defect classification model by repeating the re-learning in the first learning model generation unit 8 and the second learning model generation unit 9, in order to further improve the accuracy of the inspection operation.

[0063] Furthermore, if it is determined by a person verifying the image data classified as a defective solar cell panel that the solar cell shown in the image data is not defective, it is preferable to use this image data to re-train the second learning model generation unit 9 so that the image data can be determined to be a good product.

[0064] The inspection device 1 of this embodiment is capable of inspecting whether a solar cell panel (see FIG. 8(a), hereinafter also referred to as the first solar cell panel 100) in which solar cells of the same shape are arranged in a matrix is ​​a non-defective product by performing the above-described solar cell panel (solar cell) inspection operation. In the first solar cell panel 100 to be inspected, some solar cells may be arranged in a different orientation from the others. For example, as shown in Fig. 8(a), the first solar cell panel 100 to be inspected may have solar cells in a predetermined row (second row) of solar cells arranged in multiple rows in the short side direction, and each solar cell in the second row may be arranged in a different orientation from the solar cells in the other rows. In this first solar cell panel 100, the solar cells in the second row are arranged in an orientation rotated 180 degrees from the solar cells in the other rows in a plan view. The first solar cell panel 100 to be inspected is not limited to this, and may have all solar cells arranged in the same orientation.

[0065] When the first solar cell panel 100 or a work-in-progress thereof is the object of inspection, the above-described operation may be used to determine whether each solar cell (solar cell panel) belonging to the object of inspection is a pass or fail. If a predetermined number N1 or more (N1 is a positive integer, for example, 1) of solar cells are determined to be defective solar cell panels, the object of inspection may be determined to be defective. Alternatively, or in addition to this, if a predetermined number N2 or more (N2 is a positive integer, for example, 3) of solar cells are determined to be unqualified solar cell panels, the object of inspection may be determined to be defective. That is, the inspection device 1 may determine that the object of inspection is defective if a predetermined number or more of at least one of defective solar cell panels and unqualified solar cell panels are present.

[0066] The cell image described above may be formed by photographing the first solar cell panel 100 or a work-in-progress thereof. More specifically, a work-in-progress of the first solar cell panel 100 is formed by arranging a plurality of solar cells and an encapsulant (encapsulating sheet) on a back surface substrate that forms the back surface side, and then arranging a light-transmitting substrate such as a glass substrate that forms the light-receiving surface side on top of that. Then, in this state, a current is passed through the plurality of solar cells to cause them to emit light, and an image is taken from the light-transmitting substrate side. In other words, the cell image may be obtained by photographing with the light-transmitting substrate disposed between the plurality of solar cells and the photographing means (camera).

[0067] Furthermore, the inspection device may be capable of inspecting whether a solar cell panel (see FIG. 8(b), hereinafter also referred to as the second solar cell panel 101) formed with multiple types of solar cells arranged side by side is a non-defective product by inspecting the solar cell (solar cell panel).

[0068] In this case, the number of solar cells included in one cell image obtained by capturing an image of the solar cell may be one as described above, or may be multiple. Also, the cell image may be formed by capturing an image of the second solar cell panel 101 or a work-in-progress thereof, as described above. That is, as described above, the image data of the solar cell panel may be image data formed by photographing a part or the whole of the solar cell panel with a photographing means such as a camera, and a plurality of solar cells may be included in one cell image. In this case, the inspection device may determine whether each solar cell is pass or fail as described above, or may determine whether each plurality of solar cells (for example, one cell row) is pass or fail.

[0069] The following describes in detail the inspection operation for inspecting the second solar cell panel 101, assuming that a plurality of solar cells can be contained in one cell image. Note that the same parts as those in the above-described embodiment are given the same reference numerals, and redundant explanations will be omitted.

[0070] In the inspection device of this embodiment, the non-defective product learning model 120 (see FIG. 10) generated by the first learning model generation unit 8 is different from the non-defective product learning model 20 (see FIG. 5) described above. In the inspection device of this embodiment, the first learning model generation unit 8 employs a conditional efficient GAN instead of the above-described efficient GAN as the unsupervised learning method for generating the good learning model 120. In this way, the generative adversarial network (GAN) employed in generating the good learning model 120 may be a conditional efficient GAN, or more specifically, a conditional GAN.

[0071] In this embodiment, too, learning data is provided to the machine learning unit, so that the unit learns to generate an image from noise, and at the same time learns to generate noise from the image, thereby performing two-way learning, as shown in Fig. 9. Here, the machine learning unit of this embodiment differs from the above-described machine learning units in that it has a label input unit.

[0072] That is, in the machine learning unit of this embodiment, as shown in FIG. 9, a real image serving as learning data is input to an image data input unit, and a label corresponding to this real image (information indicating a condition label, described in detail later) is input to a label input unit. Then, a generator unit creates a fake image based on the noise generated by the noise generation unit and the input label. Then, a set of noise, a fake image generated based on the noise, and a label is input to a discriminator unit. Meanwhile, a real image serving as learning data is input to an image data input unit, and input from the image data input unit to an encoder unit. Furthermore, the encoder unit generates noise based on the real image. Then, a set of noise generated based on the real image, the real image, and a label is input to a discriminator unit. The discriminator unit determines whether the input set of noise, image, and label is a set of a real image or a set of a fake image.

[0073] In this embodiment, a group of non-defective image data is used as a dataset, and features in the dataset are learned. That is, the machine learning unit of this embodiment receives, as learning data, a group of non-defective image data and labels corresponding to each cell image (real image) belonging to the group of non-defective image data. Here, the cell image of this embodiment contains a plurality of solar cells in one cell image as described above, and more specifically, contains a cell row 105 (see FIG. 8(b)) formed by electrically connecting a plurality of solar cells. In other words, the cell image of this embodiment is an image containing one cell row 105.

[0074] 8(b), the second solar cell panel 101 has a plurality of cell rows 105 (six in this embodiment) arranged in different positions. The second solar cell panel 101 of this embodiment has, from one side in the longitudinal direction, a first cell row 105a, a second cell row 105b, ..., and a sixth cell row 105f. As a result, the cell image is an image that includes any of the first cell row 105a to the sixth cell row 105f.

[0075] The first cell row 105a and the sixth cell row 105f are cell rows of the same shape but are arranged in different orientations, i.e., one of the first cell row 105a and the sixth cell row 105f is rotated 180 degrees from the other in a plan view. Furthermore, the second cell column 105b to the fifth cell column 105e are cell columns 105 having a different shape from the first cell column 105a and the sixth cell column 105f. Specifically, the shapes of the solar cells belonging to the first cell column 105a and the sixth cell column 105f are different from those of the second cell column 105b to the fifth cell column 105e. That is, each of the solar cells belonging to the first cell column 105a and the sixth cell column 105f is a solar cell having a substantially horizontally rectangular shape with two corners spaced apart in the longitudinal direction missing. In contrast, each of the solar cells belonging to the second cell column 105b to the fifth cell column 105e is a solar cell having a substantially horizontally rectangular shape with four corners.

[0076] As described above, the label is information indicating the condition label, and in this embodiment, the label is information that identifies the cell string 105 contained in the corresponding cell image. That is, the label is information that identifies which of the first cell string 105a to sixth cell string 105f the cell string 105 contained in the cell image is. That is, the label is information that can identify in which part of the second solar cell panel 101 the cell string 105 contained in the corresponding cell image is arranged.

[0077] In this way, by inputting images associated with their corresponding labels as learning data, the generator unit (generator) and discriminator unit (identifier) ​​perform learning taking the labels (conditions) into consideration. As a result, when photographing a work-in-progress of the second solar cell panel 101 to acquire cell images, learning is possible that includes distortion of the solar cell due to differences in the placement location of the cell row 105, the presence or absence of missing corners of the solar cell, etc. This makes it possible to improve the accuracy of the generated non-defective product learning model 120. Furthermore, there is no need to generate a learning model for each type (e.g., each shape) of cell row 105 (solar cell), making it easier to build, manage, and operate the inspection device 1.

[0078] The non-defective image data group of this embodiment is also a group consisting of multiple cell images, and is a group of cell images (non-defective cell images) of cell rows 105 that were previously determined to be non-defective. In other words, the non-defective image data group does not include cell images (defective cell images) of cell rows 105 that were previously determined to be non-defective. That is, this good cell image may be an image that does not include a predetermined defect object, as described above, or may not include a cell image that does not meet a predetermined standard for good quality. Also, as described above, the good cell image may be an image that does not include a portion that is a defect object and has a small distribution of brightness of the solar cell within the image, or may be an image in which one or more good cell images belonging to the good image data group have been subjected to image correction processing.

[0079] 10, the good product learning model 120 is a model used to determine (estimate) whether a cell row 105 (solar panel) represented by input image data (hereinafter also referred to as an input image) is a good product or a defective product. This good product learning model 120 has a comparison image creation unit 121 and an error image creation unit 122, and generates a comparison image and an error image based on the input image and a label. Furthermore, the good product learning model 120 of this embodiment differs from the good product learning model 20 of the above-described embodiment in that it has a label input unit.

[0080] The comparison image creation unit 121 of this embodiment generates a comparison image based on noise generated from an input image and a label corresponding to the input image. This comparison image creation unit 121 is also an image generator that has been trained so that the comparison image it generates exhibits the characteristics of the good cell image used as the learning image. Therefore, the generated comparison image is an image that appears as if the defective portion or a portion similar to the defective portion has been removed from the input image.

[0081] The error image generating unit 122 generates an error image from the difference between the input image, the label corresponding to the input image, and the comparison image generated based on the input image by the comparison image generating unit 121. Therefore, the generated error image is an image in which the part of the input image that is a defect target or a part similar to a defect target is emphasized.

[0082] That is, in the judgment operation using the good product learning model 120 by the first judgment unit 10 of this embodiment, as shown in Fig. 10, the good product learning model 120 generates a comparison image and an error image based on an input image and a label corresponding to the input image. Then, as in the above-described embodiment, the first judgment unit 10 judges (estimates) whether the cell row shown in the input image (the solar cell shown in the input image) is a good product or a defective product based on the calculated defect score. In the judgment operation using the good product learning model 120, the operation after generating the error image is the same as the judgment operation using the good product learning model 20 described above, and therefore a redundant description will be omitted.

[0083] When the second solar cell panel 101 or a work-in-progress thereof is the object of inspection, the inspection device may determine whether each cell string 105 belonging to the object of inspection is a pass or fail product. The inspection device may then determine that the object of inspection is a fail product if a predetermined number or more of at least one of defective solar cell panels and unclassifiable solar cell panels are present. In addition, when the second solar cell panel 101 or a work-in-progress thereof is the subject of inspection, it is also possible to determine whether each solar cell belonging to the subject of inspection (each cell belonging to the cell column 105) is a good product or a defective product. In this case, the label may be information indicating the arrangement position of the solar cell. For example, if the solar cell is arranged in the Xth row and Yth column of the whole, the label may be information (X, Y) indicating the elements of the matrix that are the arrangement position. [Experimental Example]

[0084] Hereinafter, an experiment conducted by the present inventor when creating the inspection device 1 according to the above embodiment will be described.

[0085] (Experiment 1) Machine learning models were constructed corresponding to Experimental Example 1, Experimental Example 2, and Comparative Example 1. The machine learning models corresponding to Experimental Example 1, Experimental Example 2, and Comparative Example 1 all determine whether or not a solar cell in an input image is a non-defective product, and different learning datasets were used for construction. The conditions for constructing each machine learning model were all listed below (1) to (6). (1) Optimization algorithm: Adam (2) Learning coefficient: 0.003 (3) Loss function: logloss (4) Number of epochs: 150 (5) Number of layers: 8 (6) Bath size: 64 In addition, the constructed machine learning model did not have a dropout layer, but had a batch normalization layer.

[0086] 857 defective cell images and 15,000 good cell images were prepared as the training dataset for Experimental Example 1. Furthermore, 286 defective cell images and 5,000 good cell images were prepared as the verification dataset for Experimental Example 1. Note that the defective cell images all had defects of broken wires or water marks. As the training dataset for Experimental Example 2, 857 defective cell images, 15,000 good cell images, and an additional 14,000 defective cell images were prepared. Here, the additional defective cell images were normal defective cell images that had been processed by moving, rotating, etc. Furthermore, as the verification dataset for Experimental Example 2, 286 defective cell images and 5,000 good cell images were prepared. 857 defective cell images and 900 good cell images were prepared as the training dataset for Comparative Example 1. Furthermore, 286 defective cell images and 300 good cell images were prepared as the verification dataset for Comparative Example 1. Note that the defective cell images all had defects of broken wires or water marks. Furthermore, as test data, we prepared 8,846 cell images that were a mixture of good and defective cell images.

[0087] Then, machine learning models corresponding to Experimental Example 1, Experimental Example 2, and Comparative Example 1 were constructed using the training dataset and verified using the validation dataset. The constructed three machine learning models were then used to determine whether each cell image belonging to the test data was a non-defective product. The accuracy rate (acc) and Matthews Correlation Coefficient (MCC) during validation, as well as the accuracy rate (acc) and overdetection rate during testing, were obtained. The results shown in Table 1 below were obtained.

[0088] [Table 1]

[0089] The results of Experiment 1 showed that increasing the number of images in the training dataset and the proportion of good cell images in the training dataset are effective in reducing overdetection, but simply increasing the number of images in the training dataset has little effect on reducing overdetection.

[0090] (Experiment 2) A machine learning model similar to the above-described good product learning model was constructed using a generative adversarial network (Efficient GAN). In Experimental Examples 3, 4, and 5, the gain values ​​of the sigmoid function were set to 2.0, 1.0, and 0.5, respectively. Furthermore, a machine learning model that performs judgment without sharpening processing was constructed as Comparative Example 2. The machine learning model of Comparative Example 1 differs from the machine learning model of the Examples in that it does not have a processed image creation unit and the judgment unit uses an error image, but is otherwise similar to the machine learning model of the Examples. Then, the AUC score was calculated using the RCO curve. The ROC-AUC scores were 0.86, 0.84, 0.81, and 0.71 for Experimental Example 3, Experimental Example 4, Experimental Example 5, and Comparative Example 2, respectively. This indicates that the problem of small defects going undetected can be improved by performing sharpening processing with a sigmoid function gain value between 0.5 and 2.0.

[0091] In the above-described embodiments, each component can be freely substituted or added between the respective embodiments as long as it falls within the technical scope of the present invention. [Explanation of symbols]

[0092] 1. Inspection equipment (solar panel inspection equipment) 7 Image input section 8. First learning model generation unit 9 Second learning model generation unit 10 1st judgment part 11 Second Judgment Section 12 Sorting section 20 Good product learning model 21 Comparison Image Creation Department

Claims

1. The image processing device includes an image input unit that inputs image data of a solar panel, a first learning model generation unit, a second learning model generation unit, a first determination unit, a second determination unit, and a classification unit, the learning model generation unit is capable of generating a good product learning model through unsupervised learning using a group of good product image data that includes image data of solar cell panels that have been previously determined to be good products, but does not include image data of solar cell panels that have not been previously determined to be good products; the first determination unit determines whether or not the solar cell panel in the image input unit is a non-defective product by using the non-defective product learning model, the second learning model generation unit is capable of generating a defect classification model through supervised learning using a dataset of past image data of the solar cell panel and data regarding the presence or absence of defects in the image data; the second determination unit determines whether or not there is a defect in the solar cell panel of the image input unit by using the defect classification model, The classification unit classifies the solar cell panel in the input image as an unclassifiable solar cell panel when the input image input to the image input unit is determined to be defective by the first judgment unit and is not determined to have a defect by the second judgment unit.

2. 2. The solar cell panel inspection device according to claim 1, wherein the non-defective product learning model has a comparison image creation unit that creates a comparison image based on the results of the unsupervised learning, creates an error image from the difference between the input image input to the image input unit and the comparison image, applies a sigmoid function or binarization processing to the error image to create a processed image, and further determines whether the solar cell panel is non-defective using the processed image.

3. 3. The solar panel inspection device according to claim 1, wherein the unsupervised learning uses at least one framework selected from the group consisting of an autoencoder, a variational autoencoder, and a generative adversarial network.

4. 3. The solar cell panel inspection device according to claim 1, wherein the non-defective image data group does not include images of solar cell panels that have been previously determined to be non-defective and do not meet predetermined standards for non-defectiveness.

5. 3. The solar cell panel inspection device according to claim 1, wherein the non-defective image data group includes image data obtained by performing gamma correction or product-sum calculation on images of solar cell panels previously determined to be non-defective.

6. 3. The solar cell panel inspection device according to claim 1, wherein the classification unit classifies the solar cell panel of the input image as a good solar cell panel when the input image input to the image input unit is determined to be a good product by the first determination unit.

7. 3. The solar cell panel inspection device according to claim 1, wherein when the input image input to the image input unit is determined to be defective by the second determination unit, the classification unit classifies the solar cell panel of the input image as a defective solar cell panel.

8. The image processing device includes an image input unit that inputs image data of a solar panel, a first learning model generation unit, and a first determination unit, the first learning model generation unit is capable of generating a good product learning model through unsupervised learning using a good product image data group that includes image data of solar cell panels that have been previously determined to be good products, but does not include image data of solar cell panels that have not been previously determined to be good products; the first determination unit determines whether or not the solar cell panel in the image input unit is a non-defective product by using the non-defective product learning model, The non-defective product learning model has a comparison image creation unit that creates a comparison image based on the results of the unsupervised learning, creates an error image from the difference between the input image input to the image input unit and the comparison image, applies a sigmoid function or binarization processing to the error image to create a processed image, and further uses the processed image to determine whether the solar panel is non-defective.

Images