Snow cover determination device, snow cover determination system, and snow cover determination method

Abstract

To provide a snow accumulation determination device, a snow accumulation determination system, and a snow accumulation determination method capable of improving accuracy of a state of snow accumulation existing on a solar panel even after introduction of the device.SOLUTION: According to one embodiment, a snow accumulation determination device includes an input unit that receives input of an image at a first time serving as a reference. The snow accumulation determination device further includes a detection unit that detects a region where snow accumulation is present on the solar panel in the image at the first time, based on a detection model that includes an input layer and an output layer and has been trained to be capable of detecting a region where snow accumulation is present on the solar panel. The snow cover determination device further includes a determination unit that determines a state of snow cover on the solar panel. The snow accumulation determination device further includes an updating unit that updates the detection model using the image at the first time and training data in which an attribute of a subject is added to the image at the first time.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a snow accumulation determination device, a snow accumulation determination system, and a snow accumulation determination method. [Background technology]

[0002] Solar power generation requires a guarantee of the amount of power generated during actual operation of the power plant, relative to the power generation forecast made during operation planning. In areas with snowfall, snow accumulation on the solar panels can cause unstable power generation, so it is desirable to exclude snow accumulation from the output guarantee.

[0003] Snow gauges are sometimes used to determine snow accumulation, but due to the high cost of installation, determinations based on meteorological data are widely used. However, snow remains on solar panels even after snowfall, and the amount of power generated becomes unstable even on sunny days due to the remaining snow, so determinations based on meteorological data lack reliability.

[0004] Currently, known methods for determining snow accumulation on solar panels include systems that compare the amount of electricity generated by solar panels to determine snow accumulation, and systems that use camera images to determine the snow accumulation on roofs and road surfaces. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-154321 Summary of the Invention [Problem to be solved by the invention]

[0006] In addition, technology is being developed that uses machine learning to determine the state of snow accumulation on solar panels, and there is a demand for a system that can easily improve the accuracy of snow detection even after a solar power plant begins operation.

[0007] Therefore, an embodiment of the present invention provides a snow accumulation determination device, a snow accumulation determination system, and a snow accumulation determination method that can improve the accuracy of determining the state of snow accumulation on solar panels even after the device is installed. [Means for solving the problem]

[0008] According to one embodiment, a snow accumulation determination device includes an input unit that accepts input of an image at a first reference time. The snow accumulation determination device further includes a detection unit that includes an input layer and an output layer and detects areas of snow on a solar panel in an image at the first time based on a detection model trained to detect areas of snow on the solar panel. The snow accumulation determination device further includes a determination unit that determines the state of snow on the solar panel. The snow accumulation determination device further includes an update unit that updates the detection model using the image at the first time and training data in which subject attributes are assigned to the image at the first time. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of a snow accumulation determination device according to a first embodiment. [Figure 2] FIG. 3 is another schematic configuration diagram of the snow accumulation determination device in the first embodiment. [Figure 3] 1 is a functional block diagram of a snow accumulation determination device according to a first embodiment. [Figure 4] 4 is a flowchart of snow accumulation determination processing in the first embodiment. [Figure 5] 4 is an example of an output result of a detection model in the first embodiment. [Figure 6] FIG. 10 is a schematic configuration diagram of a snow accumulation determination system according to a second embodiment. [Figure 7] FIG. 10 is a functional block diagram of a snow accumulation determination system according to a second embodiment. [Figure 8] 10 is a flowchart of a snow accumulation determination system according to a second embodiment. [Figure 9]10 is a diagram illustrating a container structure of a second snow accumulation determination device in a second embodiment. FIG. [Figure 10] FIG. 10 is a hardware configuration diagram of a snow accumulation determination device according to a third embodiment. [Figure 11] FIG. 10 is a functional block diagram of a snow accumulation determination device according to a fourth embodiment. [Figure 12] 10 is a flowchart of snow accumulation determination processing in the fourth embodiment. [Figure 13] FIG. 11 is a functional block diagram of a snow accumulation determination system according to a modified example of the fourth embodiment. [Figure 14] 13 is a flowchart of snow accumulation determination processing in a modified example of the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The present invention is not limited to these embodiments. The drawings are schematic or conceptual, and the proportions of the various parts are not necessarily the same as those in reality. In the specification and drawings, elements similar to those described above with reference to the previous drawings are designated by the same reference numerals, and detailed descriptions thereof will be omitted as appropriate.

[0011] Hereinafter, the reference time will also be referred to as the first time. For example, the first time is the reference time for determining snow accumulation. Furthermore, a time before the first time will be referred to as the second time. An image including the solar panel at the imaging target location at the second time will also be simply referred to as a past image.

[0012] (First embodiment) FIG. 1 is a schematic diagram of a snow accumulation determination device 2 according to the first embodiment.

[0013] The snow accumulation determination device 2 in this embodiment is installed in a data center or the like away from the solar power plant 1, and is connected to an imaging device 6 installed in the solar power plant 1 via a network 30. The snow accumulation determination device 2 detects snow accumulation on the solar panel 100 using a detection model 12 based on an image captured by the imaging device 6. Furthermore, the snow accumulation determination device 2 updates the detection model 12 at a predetermined timing after the solar power plant 1 starts operation, thereby continuously improving the snow accumulation detection accuracy even after the solar power plant 1 starts operation. The snow accumulation determination device 2 may be provided as a so-called cloud service. Furthermore, the service provider of the snow accumulation determination device 2 and the user of the snow accumulation determination device 2 may be different or may be the same.

[0014] In FIG. 1, a string configuration 500 of solar panels 100 includes multiple solar panels 100 connected in series. Furthermore, in FIG. 1, an array configuration 700 of solar panels 100 includes multiple string configurations 500 connected in parallel. The array configuration 700 shown in FIG. 1 includes multiple string configurations 500 extending laterally and adjacent to each other in the depth direction, and each string configuration 500 includes multiple solar panels 100 arranged in one or more rows in the horizontal direction. In this embodiment, snow accumulation determination is performed using, for example, an image of a portion of the array configuration 700 captured by the imaging device 6. The snow accumulation analysis in this embodiment can be applied not only to an image of a portion of the array configuration 700 captured by the imaging device 6, but also to an image of a portion of each solar panel 100 or each string configuration 500 captured by the imaging device 6.

[0015] The imaging device 6 is, for example, at least one monitoring camera installed on the roof of a building 400 in the solar power plant 1 or in the solar power plant 1. The imaging device 6 is also installed on a mobile object such as a robot or a drone. In this case, the imaging device 6 and the snow accumulation determination device 2 may be electrically connected by wireless connection. The imaging device 6 is, for example, a monocular camera or a fisheye camera equipped with a lens and an image sensor. The imaging device 6 is, for example, a visible light camera that captures reflected light using visible light, an infrared camera, a camera that can acquire a depth map, a distance sensor, or the like.

[0016] The images captured by the imaging device 6 may be either moving images or still images. When capturing moving images, subsequent processing may be performed for each frame, or may be performed based on at least one of the multiple frames. Further, subsequent processing may be performed based on an image obtained by performing arithmetic processing such as averaging on multiple frames, or on an image obtained by performing arithmetic processing such as panoramic stitching on multiple images acquired from multiple surveillance cameras. Furthermore, the images captured by the imaging device 6 may be three-channel color images composed of R, G, and B images, or may have a color system different from the RGB color system through color system conversion processing. The different color system may be, for example, the HLS color system. Alternatively, the images may be monochrome images obtained by multiplying each channel information of a color image by a predetermined coefficient to convert it into a single channel, and the pixel Y constituting the monochrome image may be calculated, for example, using Equation (1). Y=0.2126×R+0.7512×G+0.0722×B (1)

[0017] Furthermore, the snow accumulation determination device 2 may share the imaging device 6 with the monitoring system 3 that monitors the status of the solar power plant 1. In this case, the snow accumulation determination device 2 may acquire images captured by the imaging device 6 via the network device 20, or may acquire images of the imaging device 6 stored in the memory unit 4 of the monitoring system 3. By sharing the imaging device 6 and other equipment, the snow accumulation determination device 2 can reduce the cost of introducing the equipment.

[0018] In this example, the result of snow accumulation determination made by the snow accumulation determination device 2 is displayed on the display device 5 of the monitoring system 3, but the output destination of the snow accumulation determination result is not limited to this. For example, the snow accumulation determination device 2 may be provided with a dedicated display device.

[0019] FIG. 2 is another schematic diagram of the snow accumulation determining device 2 in the first embodiment.

[0020] Generally, when the solar power plant 1 is constructed in a mountainous region or the like, the transmission capacity of the lines that make up the network 30 tends to be smaller than in urban areas. Since transmitting images places a heavy load on the network 30, the processing of the snow accumulation determination device 2 may be performed at the solar power plant 1 rather than via the network 30.

[0021] In this case, for example, the snow accumulation determination device 2 is installed in a building 400 in the solar power plant 1, and acquires images from the imaging device 6 via Ethernet or the like. In this example, the network device 20 is used to establish an Ethernet in the solar power plant 1.

[0022] In the following example, the snow accumulation determination device 2 will be described using the configuration of FIG. 1, but the processing of this device is similar to that of the configuration of FIG.

[0023] FIG. 3 is a functional block diagram of the snow accumulation determination device 2 in the first embodiment.

[0024] The snow accumulation determination device 2 includes an input unit 11, a detection model 12, a detection unit 13, a determination unit 14, an output unit 15, and an update unit 16. The snow accumulation determination device 2 can be realized, for example, by installing a program for the snow accumulation determination device 2 in a PC (Personal Computer). A CPU (Central Processing Unit) in the snow accumulation determination device 2 executes the program for the snow accumulation determination device 2, thereby realizing the functions of the input unit 11, the detection model 12, the detection unit 13, the determination unit 14, the output unit 15, and the update unit 16.

[0025] The input unit 11 receives an input of one image including the solar panel 100 at the image capture location at a first time from the imaging device 6 via the network 30. The image input here is an image that is used to determine whether or not snow is present on the solar panel 100. The input unit 11 also acquires the image via the network 30. Note that the illustration of the network device 20 is omitted. The input unit 11 receives input of images from time to time, but below, the processing performed by the snow accumulation determination device 2 when it receives input of one image will be mainly described.

[0026] The detection unit 13 detects areas on the solar panel 100 in the image where snow is present, using the detection model 12 that has been trained to be able to detect areas on the solar panel 100 where snow is present.

[0027] The area where snow is present on the solar panel is data relating to the area of ​​snow-covered pixels on the solar panel, and is calculated based on area segmentation (e.g., semantic segmentation). Alternatively, it may be calculated by solving an anomaly detection binary problem in which the area of ​​the solar panel without snow is considered to be a steady area and the other areas are considered to be anomaly areas. This anomaly detection is a method for solving unspecified anomalies, but another method for solving anomaly detection may be specific anomaly detection using a detection model 12 trained on the snow-covered area on the solar panel as an anomaly area. In this embodiment, an example calculated by area segmentation will be described. Area segmentation is one of the means for estimating which area of ​​the image to be processed has which attribute, and in this embodiment, it is performed by a pre-trained detection model 12.

[0028] The detection model 12 is a trained model that has been trained using at least one or more past images taken at the same location as the input image and training data that corresponds to the past images and has snow on the solar panel 100 as a training target. The training data is, for example, attribute data in which a user has assigned attributes to subjects included in the image. For example, the attribute data may be in a file format (.jpg, .png, .bmp, .tiff) of an image in which snow-covered locations are indicated, or in a text file format (.xml, .json, .txt) that describes the coordinates and label information of the snow-covered locations on the image, or in other formats.

[0029] The detection model 12 has an input layer and an output layer, and the input layer receives an image at a first time as input. The output layer outputs attribute data as a detection result of areas where snow is present on the solar panel 100 by area division. A determination unit 14 (described later) determines the presence or absence of snow on the solar panel 100 according to areas to which the attribute data is assigned as snow. The attribute data output from the detection model 12 using the image at the first time as input is also referred to as first attribute data.

[0030] The subject refers to something that exists regardless of the shooting conditions of the imaging device 6 when the image is acquired, such as the "solar panel 100," the "mounting," or the "background such as the sky or ground." The subject also refers to information that appears in the image depending on the environment at the time of image capture, such as "external disturbances such as blown-out highlights, ghosts, lens flare, halation, or shadows." The two may be defined independently by the user as attributes, or the latter may be included in the former. In this case, attributes such as "external disturbances such as blown-out highlights, ghosts, lens flare, halation, or shadows" are not defined.

[0031] In addition, in the case of an image, the attribute data may be composed of color information (for example, represented by RGB data such as R=255, G=0, B=0) that represents each attribute for each pixel, or indexed information (for example, the attribute solar panel 100 is assigned a serial ID=1, and the pixel information of the indexed image is assigned RGB data corresponding to R=1, G=1, B=1).

[0032] To generate a trained model, the detection model 12 performs a convolution operation on a past image and, based on the feature map resulting from the convolution operation, expresses the probability of which attribute each pixel in the past image belongs to. Based on the error between the attribute output as the most probable attribute and the attribute indicated by the training data, parameters in the detection model 12, such as the kernel and bias used during the convolution operation, are updated. The feature map of the past image may be obtained by arbitrarily changing the calculation formula, such as the size and number of kernels, and whether or not bias is used. The detection model 12 may also perform padding during the convolution operation, or a pooling operation such as max pooling or average pooling. In the case of a three-channel color image, the detection model 12 may perform a convolution operation on each of the R, G, and B images and combine the results of each operation, for example, by summing each element. Areas of snow on a solar panel can be detected based on the feature map, for example, by performing a transposed convolution operation. These areas can be identified pixel by pixel. The attribute data obtained by the detection model 12 assigning attributes to past images is also referred to as second attribute data.

[0033] The determination unit 14 receives the region segmentation result from the detection unit 13 as input and determines the state of snow accumulation on the solar panel 100. For example, the state of snow accumulation on the solar panel 100 is determined based on the snow accumulation rate. The snow accumulation rate on the solar panel 100 is the ratio of the area of ​​pixels corresponding to snow on the solar panel 100 to the area of ​​pixels corresponding to the solar panel 100. The area of ​​pixels corresponding to snow on the solar panel 100 is calculated by performing region segmentation on the captured image. On the other hand, in a fixed imaging device 6, the area of ​​pixels corresponding to the solar panel 100 can be calculated by having the snow accumulation determination device 2 recognize the area in advance using, for example, manual teaching or affine transformation, and storing the area in a memory unit (not shown). The determination unit 14 can calculate the snow accumulation rate using these two values. For example, the determination unit 14 compares the calculated snow accumulation rate with a threshold, and can determine that snow exists on the solar panel 100 if the snow accumulation rate exceeds the threshold. Hereinafter, the predetermined threshold value used by the determining unit 14 to compare the snow accumulation rate will also be referred to as a first threshold value.

[0034] The output unit 15 outputs the determination result of the snow accumulation state. For example, when it is determined that there is snow accumulation on the solar panel 100, the output unit 15 may be configured to output a signal based on the determination of the snow accumulation state. An example of a signal based on the occurrence of snow accumulation is an alarm signal. The output alarm signal may be displayed on the display device 5 via the network 30. Furthermore, instead of displaying the output alarm signal on the display device 5, it may be output to an alarm device (not shown) other than one that displays on a screen, such as an indicator or a warning light.

[0035] The update unit 16 retrains and updates the detection model 12 using the image received by the input unit 11 and the teaching data that teaches the subject in this image. The detection model 12 may be updated using the same method as the learning method for the detection model 12 before the update. For example, if a neural network is used for the detection model 12, parameters such as weights and biases of the detection model 12 before the update may be set as initial values ​​by auto-tuning, and then the parameters such as weights and biases of all layers may be updated, or the parameters such as weights and biases of only the output layer may be updated. The parameters may be set by a user instead of by auto-tuning. In the configuration of FIG. 1, the detection model 12 is updated on the cloud side, and in the configuration of FIG. 2, the detection model 12 is updated on the local side.

[0036] The images that the update unit 16 refers to when updating may be at least one or more previous images taken in advance at the same location but at a different date and time as the input image, or images that are input to the input unit 11 from time to time may be used.

[0037] FIG. 4 is a flowchart of the snow accumulation determination process in the first embodiment.

[0038] In this flowchart, after the input unit 11 receives input of an image at a first time, the snow accumulation determination device 2 determines the state of snow accumulation on the solar panel 100 contained in this image. After determining the snow accumulation, the snow accumulation determination device 2 updates the detection model 12 using this image.

[0039] In step S1, the input unit 11 receives an input of an image from the imaging device 6 via the network 30. This image is an image including the solar panel 100, captured by a fixed imaging device 6 installed in the solar power plant 1. In step S2, the detection unit 13 receives an input of the image into the detection model 12, and receives an input of the result of identifying an area where snow is present on the image including the solar panel 100, output from the detection model 12.

[0040] In step S3, the determination unit 14 determines the state of snow accumulation on the solar panel 100 using the image output from the detection model 12. For example, if the snow accumulation rate exceeds a predetermined threshold, it is determined that there is snow, and if the snow accumulation rate does not exceed the predetermined threshold, it is determined that there is no snow. In step S4, the output unit 15 outputs the determination result of the snow accumulation state. In this example, the determination result is output to the display device 5. For example, if the snow accumulation rate exceeds a predetermined threshold, text information indicating that there is snow is output, and if the snow accumulation rate does not exceed the predetermined threshold, text information indicating that there is no snow is output.

[0041] In step S5, the update unit 16 updates the detection model 12 using the image received as input by the input unit 11. The update unit 16 may set parameters such as weights and biases of the neural network in the detection model 12 before the update as initial values, and then update parameters such as weights and biases of all layers, or may update parameters such as weights and biases of only the output layer. The learning data may also be additionally saved in a database.

[0042] FIG. 5 shows an example of an output result of the detection model 12 in the first embodiment.

[0043] Fig. 5A is an example of an image received as input by the input unit 11, and Fig. 5B is the result of region segmentation output by the detection model 12. In this example, for simplicity of explanation, the result of region segmentation using one solar panel 100 will be explained, rather than the result of region segmentation using an array configuration 700.

[0044] The image in Fig. 5A captures the solar panel 100, the ground 130, the sky 140, and the mounting base 150. In the region segmentation result in Fig. 5B, the solar panel 100 is color-coded green, the ground 130 is color-coded pink, the sky 140 is color-coded blue, and the mounting base 150 is color-coded black. For the sake of explanation, the green-colored parts are represented by a single gray color, the pink-colored parts are represented by thin diagonal hatching, the blue-colored parts are represented by thick diagonal hatching, and the black-colored parts are represented by dotted hatching.

[0045] The update unit 16 also uses sets of teaching data such as those shown in FIGS. 5A and 5B. The teaching data in FIG. 5B differs in that attributes have been assigned in advance as correct answer data by, for example, a user. The update unit 16 updates the detection model 12 based on the teaching data. For example, the update unit 16 re-learns the detection model 12 using teaching data that corresponds to the input image, the result of region segmentation, and the input image, and that has snow accumulation on the solar panel 100 as a teaching target. The update unit 16 updates the parameters of the detection model 12 by, for example, re-learning.

[0046] According to this embodiment, the snow accumulation determination device 2 can update the detection model 12 based on the image captured by the imaging device 6. This allows the user of the snow accumulation determination device 2 to easily obtain highly accurate evidence of snow accumulation.

[0047] Furthermore, according to this embodiment, by performing the processing of the snow accumulation determination device 2 on the solar power plant 1 side, the load on the network 30 associated with the transmission of images can be reduced, and evidence of snow accumulation can be retained even in solar power plants 1 with limited line capacity.

[0048] (Second embodiment) FIG. 6 is a schematic diagram of a snow accumulation determination system according to the second embodiment.

[0049] The snow accumulation determination system in this embodiment includes a first snow accumulation determination device 2' and a second snow accumulation determination device 2". The first snow accumulation determination device 2' is installed, for example, in a data center or the like, and is connected to an imaging device 6 and a second snow accumulation determination device 2" installed in a solar power plant 1 via a network 30. The first snow accumulation determination device 2' may be provided as a cloud service, as in the first embodiment. The second snow accumulation determination device 2" is installed, for example, in a building 400 of the solar power plant 1, and operates in a different environment from the first snow accumulation determination device 2'. In addition, the second snow accumulation determination device 2" is connected to the first snow accumulation determination device 2' via the network 30.

[0050] The first snow accumulation determination device 2′ receives an image captured by the imaging device 6 as an input, and detects snow accumulation on the solar panel 100 using the first detection model 29 based on the image captured by the imaging device 6.

[0051] The second snow accumulation determination device 2'' generates a pseudo-snow accumulation image by converting pixels in a past image into pixels corresponding to snow accumulation. Furthermore, before detecting snow accumulation, the first snow accumulation determination device 2' trains the first detection model 29 using the pseudo-snow accumulation image generated in advance by the second snow accumulation determination device 2''.

[0052] The second snow accumulation determination device 2'' also updates the second detection model 27 based on the image at the first time. The second detection model 27 is updated at a predetermined timing, and updates to parameters, etc. are reflected in the first detection model 29 present on the network 30 by file sharing.

[0053] The second snow accumulation determination device 2'' operates as a computer that exists in an environment different from the cloud. A computer that exists in an environment different from the cloud may be, for example, a local machine such as a local computer server that is installed near a user, such as the building 400 of the solar power plant 1, a management center, or a control center, and that the user operates directly at hand. Furthermore, these computers may be containers virtually constructed on a local machine, or may be containers that are operated by a user through a platform that can manage and automate multiple containers.

[0054] FIG. 7 is a functional block diagram of the snow accumulation determination system according to the second embodiment.

[0055] Since the first snow accumulation determination device 2' has the same configuration as the snow accumulation determination device 2, the configuration of the second snow accumulation determination device 2'' will be mainly described. In this diagram, for the sake of explanation, the detection model 12 in the above-mentioned embodiment is depicted as a first detection model 29. Furthermore, functional blocks included in one of the first snow accumulation determination device 2' and the second snow accumulation determination device 2'' may be included in the other device. For example, the update unit 16 may be included in the second snow accumulation determination device 2''. The input layer of the first detection model 29 is also referred to as the first input layer, and the output layer is also referred to as the first output layer.

[0056] The second snow accumulation determination device 2'' comprises a training image input unit 21, a pseudo-snow accumulation image generation unit 22, a generation unit 23, a control unit 24, a correction unit 25, a pseudo-snow accumulation image database 26, a second detection model 27, and an update database 28. The second snow accumulation determination device 2'' can be realized, for example, by installing a program for the second snow accumulation determination device 2'' on a PC. When the CPU in the second snow accumulation determination device 2'' executes the program for the second snow accumulation determination device 2'', the functions of the training image input unit 21, the pseudo-snow accumulation image generation unit 22, the generation unit 23, the control unit 24, the correction unit 25, the pseudo-snow accumulation image database 26, the second detection model 27, and the update database 28 are realized. The input layer of the second detection model 27 is also called the second input layer, and the output layer is also called the second output layer.

[0057] The learning image input unit 21 receives as input a past image taken at the same location as the image taken at the first time by the imaging device 6. For example, it is conceivable that a past image database is provided in the second snow accumulation determination device 2'', and the learning image input unit 21 uses the past images stored in this database as input.

[0058] The pseudo-snow-covered image generation unit 22 converts pixels in the image acquired by the learning image input unit 21 into pixels corresponding to snow, thereby generating a pseudo-snow-covered image in which snow has been generated in the image. Pixels corresponding to snow may be realized, for example, by comparing the luminance distribution of snow-covered areas with that of non-snow-covered areas and converting the luminance values ​​of pixels in a previous image, or by replacing the luminance of the non-snow-covered areas with the luminance of the snow-covered areas. In this case, the luminance value of the snow-covered area may be set by obtaining it from, for example, an image of the solar panel 100 with snow, acquired in the past at another solar power plant 1. Furthermore, the luminance value of the snow-covered area may be changed to change the values ​​of each RGB channel so as to represent the color of actual snow. The RGB values ​​capable of representing the color of snow may be set by obtaining it from, for example, an image of the solar panel 100 with snow, acquired in the past at another solar power plant 1.

[0059] When generating the pseudo-snow-covered image, the pseudo-snow-covered image generating unit 22 may perform conversion on pixels that have brightness values ​​within a predetermined range among the pixels present on the solar panel 100 in the image. Also, the area of ​​the solar panel 100 that should be converted may be explicitly indicated by the pseudo-snow-covered image generating unit 22 or the user.

[0060] Furthermore, the pseudo-snow-covered image generating unit 22 may adjust the brightness of the pseudo-snow or the values ​​of each RGB channel in accordance with the brightness of the entire image to generate a more natural pseudo-snow-covered image.

[0061] The processing up to this point has been a method for generating pseudo-snow-covered images based on basic image processing methods, but the pseudo-snow-covered image generation unit 22 may also generate pseudo-snow-covered images using machine learning, particularly deep learning. A generative adversarial network, for example, may be used as a deep learning method. The number of pseudo-snow-covered images to be generated may be set automatically by the pseudo-snow-covered image generation unit 22, or may be set arbitrarily by the user.

[0062] The pseudo-snow-covered image generating unit 22 also assigns attributes to all pixels that make up the pseudo-snow-covered image to create attribute data. The attribute data generated by the pseudo-snow-covered image generating unit 22 is also called third attribute data.

[0063] Furthermore, the pseudo-snow-covered image generating unit 22 stores the generated pseudo-snow-covered image and attribute data corresponding to the pseudo-snow-covered image in the pseudo-snow-covered image database 26 in addition to the past images and their attribute data.

[0064] Also, unlike the first embodiment, the first detection model 29 performs learning using past images and attribute data corresponding to those past images, and pseudo-snow-covered images and attribute data corresponding to those pseudo-snow-covered images. The detection model 12 performs learning using the attribute data as training data. Learning by the detection model 12 may be performed in advance, before the snow accumulation determination system is installed.

[0065] The generation unit 23 receives an input of an image captured at a first time by the imaging device 6. In this figure, the generation unit 23 is configured to receive an input of an image from the first snow accumulation determination device 2′, but the generation unit 23 may receive an input of an image directly from the imaging device 6 without going through the first snow accumulation determination device 2′.

[0066] The generation unit 23 creates attribute data for this image using a second detection model 27 that operates in an environment different from that of the first detection model 29. The input layer of the second detection model 27 receives the image at the first time as input, and the output layer outputs the attribute data. Before updating the second detection model 27, which will be described below, the first detection model 29 and the second detection model 27 may share files and synchronize parameters, etc., so that the second detection model 27 can be used as a trained model.

[0067] The control unit 24 determines whether to modify the snow-covered area based on whether the snow-covered area is assigned to a desired area among the attributes assigned by the generation unit 23. The desired area is determined, for example, based on whether the attribute is correctly assigned to the area corresponding to snow among the assigned attribute data.

[0068] For example, determining whether attribute data is assigned to a desired region in an image is performed by comparing the similarity of attributes between an image to which attribute data is assigned and an image (comparison image) to which different attribute data is assigned. The comparison image may be, for example, an image similar to the image to which attribute data is assigned selected from past images containing snow, or several past images immediately preceding the image to which attribute data is assigned, and used for the similarity comparison. For example, the control unit 24 may compare the distribution of attribute data between the image to which attribute data is assigned and the comparison image and calculate the similarity. If the distribution of attribute data between the image to which attribute data is assigned and the comparison image differs significantly, the control unit 24 may determine that the attribute assigned to the snow-covered region is not assigned to the desired region. This comparison may be performed, for example, using a threshold value. This comparison may also be performed by comparing RGB distributions rather than comparing attribute data. The predetermined threshold value used by the control unit 24 to compare the distribution of attribute data is also referred to as a third threshold value.

[0069] Halation may appear as a blurred white image in an image. Even if an image contains halation and the generation unit 23 erroneously attributes the halation in this image as snow, the control unit 24 can determine that the attribute data is not assigned to a desired area in the image by comparing the distribution of attributes between this image and another image. The determination of whether the attribute data is assigned to a desired area in the image may be performed, for example, by a user. Furthermore, if a sensor or the like attached to the solar panels 100 clearly shows a difference in the amount of power generated by each solar panel 100, the control unit 24 may determine whether the attribute data is assigned to a desired area by comparing the amount of power generated by each solar panel 100 to which the attribute data is assigned with the distribution of snow-covered areas on each solar panel 100.

[0070] Correction unit 25 corrects the attribute data when control unit 24 determines that a snow-covered area is not assigned to a desired area in the attribute data and that correction is necessary. Also, correction unit 25 stores the image at the first time point and the corrected attribute data corresponding to this image in update database 28.

[0071] The correction unit 25 may provide an annotation tool via a UI (User Interface) that allows the user to correct data. For example, the annotation tool may be an application that allows the user to correct attribute data pixel by pixel. The user can make corrections using the annotation tool by comparing the attribute data with the image before the attribute data was added.

[0072] If no correction is required, the control unit 24 stores in the update database 28 the image at the first time and an image to which attribute data corresponding to this image has been added.

[0073] Furthermore, the update unit 16 updates the second detection model 27 using the image at the first time stored in the update database 28 and the attribute data corresponding to this image or the corrected attribute data corresponding to this image. The update of the second detection model 27 may be realized by, for example, re-learning similar to that described above. Furthermore, the update unit 16 shares the file of the second detection model 27 with the first detection model 29 by file sharing, and updates the first detection model 29.

[0074] FIG. 8 is a flowchart of the snow accumulation determination system according to the second embodiment.

[0075] In this flowchart, for simplicity's sake, the second snow accumulation determination device 2'' generates one pseudo-snow accumulation image and attribute data for this image based on a past image, and the first detection model 29 performs learning using these learning data. In this example, a learning example based on one pseudo-snow accumulation image is described, but the learning data used by the first detection model 29 may be any number of pseudo-snow accumulation images. Furthermore, the second snow accumulation determination device 2'' creates attribute data using an image at a first time and determines whether or not this data needs to be corrected. Furthermore, the second snow accumulation determination device 2'' updates the first detection model 29 based on the corrected attribute data.

[0076] In step S21, the learning image input unit 21 in the second snow accumulation determination device 2'' accepts input of a past image that includes the solar panel 100 and was taken at the same location as the input image. For example, the learning image input unit 21 accepts input of a past image from a past image database. In step S22, the pseudo-snow-covered image generation unit 22 converts pixels in the past image into pixels corresponding to snow, and generates a pseudo-snow-covered image in which snow has been generated in the image. For example, the pseudo-snow-covered image generation unit 22 compares the brightness of the snow-covered area with the brightness of the non-snow-covered area, and converts the brightness values ​​so that the brightness of the non-snow-covered area on the solar panel 100 in the past image matches the brightness of the snow-covered area. The pseudo-snow-covered image generation unit 22 also creates attribute data for the pseudo-snow-covered image. In addition to the past image and its attribute data, the pseudo-snow-covered image generation unit 22 stores the generated pseudo-snow-covered image and attribute data corresponding to the pseudo-snow-covered image in the pseudo-snow-covered image database 26.

[0077] In step S23, the first detection model 29 in the first snow accumulation determination device 2' performs learning using a past image and attribute data corresponding to that past image, and a pseudo-snow-covered image and attribute data corresponding to that pseudo-snow-covered image. The first detection model 29 learns using each attribute data as training data. In step S24, the generation unit 23 accepts input of an image at the first time. The generation unit 23 may acquire an image directly from the imaging device 6, or may use an image accepted as input by the input unit 11. The generation unit 23 also uses the second detection model 27 to detect a snow-covered area in the image at the first time, and assigns attributes to this snow-covered area as well as to all pixels constituting the image, thereby generating attribute data.

[0078] In step S25, control unit 24 determines whether to modify the snow-covered area based on whether the snow-covered area is assigned to a desired area among the attributes assigned by generation unit 23. If control unit 24 determines that the snow-covered area is assigned to a desired area in the attribute data and that modification of this area is unnecessary (YES in step S25), in step S26, control unit 24 stores the image at the first time point and the attribute data corresponding to this image in update database 28.

[0079] If control unit 24 determines that the snow-covered area is not assigned to a desired area in the attribute data and that this area should be corrected (NO in step S25), correction unit 25 corrects the attribute data in step S27. Similarly to step S26, correction unit 25 stores the image at the first time and the corrected attribute data corresponding to this image in update database 28. In step S28, update unit 16 refers to update database 28 and updates second detection model 27 so that the snow-covered area on solar panel 100 can be detected.

[0080] In step S29, at a predetermined timing, the update unit 16 shares files between the first snow accumulation determination device 2′ and the second snow accumulation determination device 2″, and reflects updates to the parameters of the second detection model 27, etc., in the first detection model 29.

[0081] FIG. 9 is a diagram illustrating the container structure of the second snow accumulation determining device 2'' in the second embodiment.

[0082] This figure shows that the processing executed by the first snow accumulation determination device 2' and the second snow accumulation determination device 2'' is performed by the first container 31 to the fifth container 35. If the update unit 16 is configured to be included in the second snow accumulation determination device 2'', the fifth container 35 may be included in the second snow accumulation determination device 2''.

[0083] For example, in the first container 31 to the third container 33, processing is performed by the generation unit 23, in the fourth container 34, processing is performed by the control unit 24 and the modification unit 25, and in the fifth container 35, processing is performed by the update unit 16.

[0084] The generation unit 23 is made up of a first container 31 to a third container 33. The first container 31 performs a process of acquiring an image at a first time. The second container 32 centrally manages files handled in a series of processes by the generation unit 23. The third container 33 uses the second detection model 27 stored in the second container 32 to detect a snow-covered area on the solar panel 100 in the image at the first time, and performs a process of generating attribute data based on the detected snow-covered area.

[0085] The control unit 24 and the correction unit 25 are configured by a fourth container 34. The fourth container 34 determines whether the snow-covered area generated by the second detection model 27 is assigned to a desired area, and performs processing to correct the snow-covered area based on the determination result.

[0086] The process of determining whether the snow-covered area is assigned to the desired area may be performed by the control unit 24, for example, as described above, by comparing the similarity of the attributes of an image to which attribute data different from that of the image. Alternatively, this may be achieved by providing a display function in the fourth container 34. In this case, the user makes the determination by comparing the displayed attribute data with the image via this display function.

[0087] The correction unit 25 may provide the user with a UI of the annotation tool stored in the fourth container 34. Alternatively, the correction unit 25 may be provided with a function to download the image and attribute data corresponding to the image, allowing the user to make corrections using an annotation tool installed in another container or on another computer.

[0088] The update unit 16 is configured by a fifth container 35. In the fifth container 35, for example, a process is performed to update the second detection model 27 using the image at the first time stored in the update database 28 and the corrected attribute data so that the snow-covered area on the solar panel 100 can be detected.

[0089] The updated second detection model 27 is stored in the second container 32. The second detection model 27 before the update is similarly stored in the second container 32 under version management together with a revision number.

[0090] According to this embodiment, the snow accumulation determination system updates the second detection model 27 using a second snow accumulation determination device 2'' that exists in an environment different from the cloud, and updates the first detection model 29 of the first snow accumulation determination device 2' based on the results of this update. This reduces the amount of data transferred to the first snow accumulation determination device 2' that exists on the network 30, and allows the first detection model 29 to be updated even in a solar power plant 1 with low line capacity. Furthermore, by updating the second detection model 27 using the second snow accumulation determination device 2'', the usage load of the first snow accumulation determination device 2', which serves as a cloud server, can be reduced.

[0091] Furthermore, according to this embodiment, the snow accumulation determination system creates and modifies attribute data of an image at a first time in an environment different from the cloud, and generates the data as learning data, thereby reducing the usage load of the first snow accumulation determination device 2', which serves as a cloud server.

[0092] (Third embodiment) FIG. 10 is a hardware configuration diagram of the snow accumulation determination device 2 and the like in the third embodiment.

[0093] Here, the snow accumulation determination device 2 of the first embodiment will be taken as the hardware configuration, but the first snow accumulation determination device 2' or second snow accumulation determination device 2'' of the second embodiment has a similar configuration. The snow accumulation determination device 2 of Figure 10 comprises a processor 52 such as a CPU, a main memory device 53 such as RAM, an auxiliary memory device 54 such as an HDD, a network interface 55 such as a LAN (Local Area Network) board, a device interface 56 such as a memory slot or memory port, and a bus 57 that connects these devices to each other. The snow accumulation determination device 2 is, for example, a computer such as a PC, and comprises input devices such as a keyboard and a mouse, and an output device such as an LCD (Liquid Crystal Display) monitor.

[0094] In this embodiment, a program for causing a computer to execute the information processing of the snow accumulation determination device 2 of the first embodiment, and the first snow accumulation determination device 2' or second snow accumulation determination device 2'' of the second embodiment is installed in an auxiliary storage device 54 (hereinafter, these devices will be simply referred to as the snow accumulation determination device 2). The snow accumulation determination device 2 deploys this program in the main storage device 53 and executes it using the processor 52. This enables the functions of each block shown in Figure 3 or 7 to be realized within the snow accumulation determination device 2, making it possible to perform the snow accumulation determination described in the first and second embodiments.

[0095] This program can be installed, for example, by attaching an external device 58 on which the program is recorded to the device interface 56 and storing the program from the external device 58 in the auxiliary storage device 54. Examples of the external device 58 include a computer-readable recording medium and a recording device incorporating such a recording medium. Examples of recording media include a CD-ROM (Compact Disk Read Only Memory), a CD-R (Compact Disk Recordable), a flexible disk, a DVD-ROM (Digital Versatile Disk Read Only Memory), and a DVD-R (Digital Versatile Disk Recordable), and an example of a recording device is a HDD. This program can also be installed by downloading it via the network interface 55, for example.

[0096] According to this embodiment, the functions of the snow accumulation determination device 2 in either of the first and second embodiments can be realized by software.

[0097] (Fourth embodiment) FIG. 11 is a functional block diagram of a snow accumulation determination device 2 according to the fourth embodiment.

[0098] The snow accumulation determination device 2 includes an input unit 11, a detection model 12, a detection unit 13, a determination unit 14, an output unit 15, an update unit 16, a selection unit 17, a power plant information database 18, and an update database 28. The following will omit the details explained in the above-mentioned embodiment and will mainly describe the functions that differ from the snow accumulation determination device 2 in Fig. 3. The processing of the snow accumulation determination device 2, for example, updating the detection model 12, may be performed on the cloud side or on the local side.

[0099] Since the image input to the input unit 11 is an image of a portion of the area of ​​the solar power plant 1 captured by the imaging device 6, even if the determination unit 14 determines that there is snow due to snow accumulation in an area within the image, there may be no snow accumulation in other areas and solar panels 100 in other areas may be generating electricity. Conversely, even if there is no snow accumulation in an area within the image and the determination unit 14 determines that there is no snow accumulation, there may be snow accumulation in other areas and solar panels 100 in other areas may not be generating electricity or may be generating less electricity than expected.

[0100] Therefore, in this embodiment, the selection unit 17 uses power plant information for the time period of the image used by the determination unit 14 to determine snow accumulation to determine whether or not to adopt the image as learning data for updating. In this embodiment, the power plant information used is at least one of information about the power generation of the solar panel 100 and information about the weather in the area where the solar panel 100 is installed. More specific examples of power plant information will be described later. If the selection unit 17 determines that the snow accumulation determination result matches the power plant information, it selects the image used for the snow accumulation determination as learning data for updating the detection model 12. The selection unit 17 stores the image selected as learning data for updating in the update database 28.

[0101] The update database 28 may store each image in association with a snow accumulation rate. To eliminate bias in the learning data, the update database 28 may store images with a range of snow accumulation rates, from low to high, so that the number of images for each snow accumulation rate is somewhat equal. For example, the selection unit 17 may store images in the update database 28 so that the number of images corresponding to snow accumulation rates is not biased, such as images with a snow accumulation rate determined to be 0%, images with a snow accumulation rate determined to be 20%, or images with a snow accumulation rate determined to be 40%. If the learning data is biased, the selection unit 17 may discard an image rather than adopt it as learning data for updating, depending on the breakdown of the number of images for each snow accumulation rate. For example, if a certain image for which a snow accumulation rate has been determined already has a predetermined number of other images with the same snow accumulation rate stored in the update database 28, the selection unit 17 may discard that image. To eliminate bias in the learning data, the selection unit 17 may supplement the missing images by adopting them as learning data for updating, depending on the breakdown of the number of images for each snow accumulation rate. Furthermore, if the snow accumulation determination device 2 is provided with a pseudo snow accumulation image database 26 and these images are used as learning data, the selection unit 17 may take into consideration the number of images stored in the pseudo snow accumulation image database.

[0102] When the detection model 12 is updated based on the images received by the input unit 11, the update unit 16 uses the images selected as learning data. Note that the selection unit 17 discards images for which it determines that the snow accumulation determination result does not match the power plant information, without adding them as learning data for updating the detection model 12.

[0103] The selection unit 17 receives input of the snow accumulation determination result from the determination unit 14. The selection unit 17 also extracts power plant information from the power plant information database 18 at the time the image for which snow accumulation was determined was acquired. The selection unit 17 determines whether the snow accumulation determination result conforms to the power plant information using the snow accumulation determination result and the power plant information. If the selection unit 17 determines that the snow accumulation determination result conforms to the power plant information, it selects the image used for the snow accumulation determination as learning data. The images selected as learning data may be either images determined to have snow or images determined to have no snow. Below, an example will be described in which it is determined whether a snow accumulation determination result that indicates the presence of snow conforms to the power plant information.

[0104] The power plant information may include, for example, the amount of power generated at the photovoltaic power plant 1, the PR (Performance Ratio) value (also called the system output coefficient) at the photovoltaic power plant 1, weather data for the area where the photovoltaic power plant 1 is installed, or a combination of these. However, the power plant information is not limited to these examples, and may include, for example, information on current and voltage values ​​such as solar radiation intensity, solar radiation amount, and IV characteristics, environmental information such as outside air temperature, and the panel temperature or humidity of the photovoltaic panels 100, as well as various other data related to the photovoltaic power plant 1. Data acquired at another photovoltaic power plant 1 built nearby may also be used. Values ​​for the amount of power generated and the PR value of an individual photovoltaic panel 100 may also be used. The snow accumulation determination device 2 stores data measured every moment at the photovoltaic power plant 1 in the power plant information database 18 and uses it as power plant information, as well as stores data acquired via the network 30 in the power plant information database 18 and uses it.

[0105] For example, for an image determined to have snow, the selection unit 17 extracts the amount of power generated at the time the image was captured from the power plant information database 18. If the amount of power generated is zero or below a predetermined threshold, it is highly likely that no power was generated during that time period. Therefore, the selection unit 17 determines that the determination of snow is highly valid and conforms to the power plant information. In this case, the selection unit 17 selects the image as learning data for updating the detection model 12. On the other hand, if the amount of power generated is not zero or is greater than the predetermined threshold, it is highly likely that power was generated during that time period. Therefore, the selection unit 17 determines that the determination of snow is low in validity and does not conform to the power plant information. In this case, the selection unit 17 discards the image without selecting it as learning data for updating. The specific value of the power plant information varies depending on the type of information used. However, for simplicity of explanation, the predetermined threshold used by the selection unit 17 for comparison is also referred to as the second threshold.

[0106] For example, for an image determined to have snow, the selection unit 17 extracts the PR value for the time the image was acquired from the power plant information database 18. The relationship between the PR value, the amount of power generation, and the amount of solar radiation is PR value ∝ (amount of power generation / amount of solar radiation). Possible weather conditions for low solar radiation include snow, cloudy weather, and rain. In other words, using the PR value provides an index that takes into account not only the amount of power generation but also the effects of weather. For example, if the amount of power generation is low despite sufficient solar radiation for power generation, there is a possibility that snow has accumulated on the solar panel 100. If the PR value is below a predetermined threshold, it is highly likely that no power generation is occurring during that time period. Therefore, the selection unit 17 determines that the determination of snow accumulation is highly valid and conforms to the power plant information. On the other hand, if the PR value is greater than the predetermined threshold, it is highly likely that power generation is occurring during that time period. Therefore, the selection unit 17 determines that the determination of snow accumulation is low in validity and does not conform to the power plant information.

[0107] Furthermore, for an image determined to have snow accumulation, the selection unit 17 extracts the amount of power generation and the amount of solar radiation at the time the image was acquired from the power plant information database 18. If the amount of power generation is zero and the amount of solar radiation is zero or insufficient for power generation, it is impossible to determine whether the cause of the zero amount of power generation is the influence of snow accumulation or the influence of a lack of solar radiation. Therefore, such an image has many uncertainties, and the selection unit 17 may decide not to select it as learning data.

[0108] Furthermore, for an image determined to have snow accumulation, the selection unit 17 extracts the amount of solar radiation and panel temperature at the time the image was acquired from the power plant information database 18. If the temperature of the solar panel 100 is lower than the environmental temperature of the solar power plant 1, for example, if the panel temperature is zero or below a predetermined threshold, there is a high possibility that the panel temperature has dropped due to snow accumulation and that power generation is not occurring. This information also enables the selection unit 17 to determine that there is no increase in panel temperature due to power generation. Therefore, the selection unit 17 determines that the result of determining that there is snow accumulation is highly valid and conforms to the power plant information. In this case, the selection unit 17 selects the image as learning data for updating the detection model 12.

[0109] For example, if snow has accumulated on a panel, the panel temperature is expected to drop below 0°C. Also, even if snow has accumulated on the panel, if it is exposed to sunlight and begins to melt, power generation will begin on the panel and the panel temperature will rise. Taking this into consideration, the predetermined threshold can be set to a margin of 5°C, but various other values ​​can also be set.

[0110] Furthermore, for example, the selection unit 17 may compare the theoretical power generation amount or theoretical PR value when there is no snow on the solar panel 100 with the actual value of the power generation amount or PR value depending on the amount of solar radiation, and determine whether the snow accumulation determination result conforms to the power plant information. If the actual value of the power generation amount or PR value differs from the theoretical value of the power generation amount or PR value, the selection unit 17 determines that there is snow on the solar panel 100. Comparing the theoretical value and the actual value of the power generation amount or PR value can be used as an index of the extent to which it is affected by snow accumulation.

[0111] For example, for an image determined to have snow, the selection unit 17 extracts from the power plant information database 18 the theoretical power generation amount and the actual power generation amount or the actual PR value at the time the image was acquired. If the theoretical power generation amount and the actual power generation amount are relatively close to each other, it is highly likely that there is no snow and power is being generated. Therefore, the selection unit 17 determines that the result of determining that there is snow is not valid and does not match the power plant information. In this case, the selection unit 17 discards this image without selecting it as learning data for updating. Furthermore, if the actual power generation amount is smaller than the theoretical power generation amount, it is highly likely that power generation is not being generated due to snow on the solar panel 100. Therefore, the selection unit 17 determines that the result of determining that there is snow is highly valid and matches the power plant information. The comparison between the theoretical power generation amount and the actual power generation amount can be performed, for example, using a predetermined threshold value.

[0112] Furthermore, for example, for an image determined to have snow accumulation, the selection unit 17 extracts the current value or voltage value at the time the image was acquired. The current value or voltage value may be measured for each power conditioner (also called PCS: Power Conditioning System), or may be measured for each connection box that serves as a connection point for multiple solar panels 100. The current value or voltage value may also be measured at a connection point with the power grid.

[0113] Furthermore, the selection unit 17 may use the amount of solar radiation measured by a pyranometer installed in the solar power plant 1 as the power plant information. In areas where snowfall occurs, snow may accumulate on the pyranometer, making it impossible to measure the amount of solar radiation correctly. When the pyranometer is present within the angle of view of the imaging device 6 and can be imaged, or when the pyranometer can be photographed using the pan-tilt function of the imaging device 6, the snow accumulation determination device 2 may determine whether or not snow has accumulated on the pyranometer based on the image captured by the pyranometer. For example, similar to the above, the detection unit 13 may determine whether or not snow has accumulated on the pyranometer using the detection model 12.

[0114] The update unit 16 re-learns and updates the detection model 12 using the teaching data that teaches the subject to the image stored in the update database .

[0115] FIG. 12 is a flowchart of the snow accumulation determination process in the fourth embodiment.

[0116] In this flowchart, after the input unit 11 receives input of an image at a first time, the snow accumulation determination device 2 determines the state of snow accumulation on the solar panel 100 contained in this image. After determining the snow accumulation, the snow accumulation determination device 2 uses the power plant information to decide whether or not to adopt the image as learning data for updating. The snow accumulation determination device 2 uses the image as learning data for updating and updates the detection model 12 at a predetermined timing.

[0117] Also, in this flowchart, for the sake of simplicity, the flow is described in which the snow accumulation determination device 2 determines snow accumulation based on the first image and selects learning data using this result and power plant information, but the snow accumulation determination device 2 may also select learning data using past images.

[0118] The operations of steps S31 to S33 are the same as the operations of steps S1 to S4 described in Fig. 4, and therefore will not be described again. In step S34, the determination unit 14 outputs the snow accumulation determination result to the output unit 15 and also outputs this result to the selection unit 17. In step S35, the selection unit 17 extracts power plant information at the time when the image for which snow accumulation was determined was acquired from the power plant information database 18. For example, the selection unit 17 extracts the PR value from the power plant information database 18.

[0119] In step S36, the selection unit 17 determines whether the snow accumulation determination result matches the power plant information. For example, the power generation amount at the time the image was acquired may be used as the power plant information, or the PR value may be used, or a combination of these data may be used. If it is determined that the snow accumulation determination result matches the power plant information (YES in step S36), in step S37 the selection unit 17 adopts this image as learning data for update and stores it in the update database 28. Note that the user may create the teaching data at any time.

[0120] In step S38, the update unit 16 updates the detection model 12 using the image stored as learning data in the update database 28. The update of the detection model 12 is performed in the same manner as in step S5 described above. On the other hand, if it is determined that the snow accumulation determination result does not match the power plant information (NO in step S36), the selection unit 17 discards this image without storing it in the update database 28.

[0121] FIG. 13 is a functional block diagram of a snow accumulation determination system according to a modified example of the fourth embodiment.

[0122] In this modified example, the configuration of the second snow accumulation determination device 2'' is different from that of Figure 7. The second snow accumulation determination device 2'' includes a learning image input unit 21, a pseudo-snow accumulation image generation unit 22, a generation unit 23, a control unit 24, a correction unit 25, a pseudo-snow accumulation image database 26, a second detection model 27, an update database 28, a selection unit 17, and a power plant information database 18. Below, we will mainly explain the functions that are different from the snow accumulation determination system of Figure 7.

[0123] In this modified example, the selection unit 17 described above is realized within the control unit 24. The configurations of the first snow accumulation determination device 2' and the second snow accumulation determination device 2'' are not limited to this example. For example, the selection unit 17 may be included in the first snow accumulation determination device 2', or the function corresponding to the update unit 16 may be realized by a functional block other than the control unit 24.

[0124] The generation unit 23 creates attribute data based on the image at the first time. The control unit 24 determines whether to modify the snow-covered area based on whether the snow-covered area is assigned to a desired area among the attributes assigned by the generation unit 23. If the attribute data is to be modified, the modification unit 25 modifies the attribute data.

[0125] In this modification, the selection unit 17 determines whether the snow accumulation determination result matches the power plant information before storing the first image and attribute data in the update database 28. The snow accumulation determination is performed by the control unit 24 using a method similar to that of the determination unit 14. For example, the control unit 24 compares the calculated snow accumulation rate with a threshold value, and if the snow accumulation rate exceeds the threshold value, it can determine that there is snow on the solar panel 100. The snow accumulation determination may also be performed by the user checking the image.

[0126] The selection unit 17 uses the power plant information for that time period to determine whether or not to use the image used for the snow accumulation determination as learning data for updating. If the selection unit 17 determines that the snow accumulation determination result matches the power plant information, it selects the image used for the snow accumulation determination as learning data for updating the second detection model 27. The selection unit 17 stores the image selected as learning data for updating in the update database 28.

[0127] The generating unit 23 may be configured to generate attribute data based on past images. In this case, the selecting unit 17 determines whether to use the past image used for snow accumulation determination as learning data for updating, using the power plant information for that time period.

[0128] FIG. 14 is a flowchart of snow accumulation determination processing in a modified example of the fourth embodiment.

[0129] In this flowchart, the second snow accumulation determination device 2'' creates attribute data based on an image at a first time, modifies the attribute data as necessary, and then decides whether to adopt this image as learning data for updating. If the image is to be adopted, the selection unit 17 stores it in the update database 28, and the update unit 16 updates the second detection model 27.

[0130] The second snow accumulation determining device 2'' may be configured to create attribute data based on a past image instead of the image at the first time, and determine whether or not to use this image as learning data for updating.

[0131] The flow of steps S41 to S45 is the same as the flow of steps S21 to S25 described above, and therefore a description thereof will be omitted. If the control unit 24 determines that the snow-covered area in the attribute data is assigned to a desired area and that correction of this area is unnecessary (YES in step S45), the control unit 24 performs a snow-covered determination on the attribute data in step S46. The selection unit 17 also extracts power plant information at the time when the image for which snow coverage was determined was acquired from the power plant information database 18. For example, the selection unit 17 extracts a PR value from the power plant information database 18. On the other hand, if the control unit 24 determines that the snow-covered area in the attribute data is not assigned to a desired area and that correction of this area is necessary (NO in step S45), the correction unit 25 corrects the attribute data in step S47. In step S46, the selection unit 17 extracts power plant information at the time when the image for which snow coverage was determined was acquired from the power plant information database 18.

[0132] In step S48, the selection unit 17 determines whether the snow accumulation determination result matches the power plant information. For example, the power generation amount at the time the image was acquired may be used as the power plant information, or the PR value may be used, or a combination of these data may be used. If it is determined that the snow accumulation determination result matches the power plant information (YES in step S48), in step S49 the selection unit 17 adopts this image as learning data for update and stores it in the update database 28. Note that the user may create the teaching data at any time.

[0133] In step S50, the update unit 16 updates the second detection model 27 using the image stored as learning data in the update database 28. The update of the second detection model 27 is performed in the same manner as in step S5 described above. On the other hand, if it is determined that the snow accumulation determination result does not match the power plant information (NO in step S48), the selection unit 17 discards this image without storing it in the update database 28.

[0134] According to this embodiment and its modified example, the snow accumulation determination device 2 or the second snow accumulation determination device 2'' determines the validity of the snow accumulation determination result using power plant information. Furthermore, the snow accumulation determination device 2 or the second snow accumulation determination device 2'' stores only images for which the validity of the snow accumulation determination result is determined to be high in the update database 28 and uses them to update the detection model 12 or the second detection model 27. The snow accumulation determination device 2 or the second snow accumulation determination device 2'' determines the presence or absence of snow accumulation at the solar power plant 1 using both the snow accumulation determination result and the power plant information and selects learning data for update, so that the accuracy of snow accumulation determination can be improved as the detection model 12 or the second detection model 27 is updated.

[0135] Furthermore, according to this embodiment and its modified examples, the snow accumulation determination device 2 or the second snow accumulation determination device 2'' uses, for example, the PR value as power plant information. By using the PR value as power plant information, the snow accumulation determination device 2 or the second snow accumulation determination device 2'' can take into account not only the amount of power generation but also the influence of weather when determining the validity of the snow accumulation determination result.

[0136] Although several embodiments have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. The novel snow accumulation determination device 2 described in this specification can be embodied in various other forms. Furthermore, various omissions, substitutions, and modifications can be made to the snow accumulation determination device 2 described in this specification without departing from the spirit of the invention. The appended claims and their equivalents are intended to cover such forms and modifications that fall within the scope and spirit of the invention. [Explanation of symbols]

[0137] 1: Solar power plant, 2: Snow accumulation determination device, 2': First snow accumulation determination device, 2): Second snow accumulation determination device, 3: Monitoring system, 4: Memory unit, 5: Display device, 6: imaging device, 11: input unit, 12: detection model, 13: detection unit, 14: determination unit, 15: Output unit, 16: Update unit, 17: Selection unit, 20: Network device, 21: Learning image input unit, 22: Pseudo snow image generation unit, 23: Generation unit, 24: Control unit, 25: Correction unit, 26: Pseudo snow image database, 27: Second detection model, 29: First detection model, 30: Network, 31: First container, 32: Second container, 33: Third container, 34: fourth container, 35: fifth container, 52: processor, 53: main memory device, 54: Auxiliary storage device, 55: Network interface, 56: Device interface, 57: Bus, 58: External device 130: ground, 140: sky, 150: mounting frame, 100: solar panel, 400: building, 500: string configuration, 700: array configuration,

Claims

1. an input unit that receives an input of an image at a first reference time; a detection unit that includes an input layer and an output layer and detects an area where snow is present on the solar panel in the image at the first time based on a detection model that has been trained to be able to detect an area where snow is present on the solar panel; a determination unit that determines the state of snow accumulation on the solar panel; an update unit that updates the detection model using the image at the first time and teaching data in which an attribute of a subject is assigned to the image at the first time, Snowfall determination device.

2. The input layer receives an image at the first time as an input, The output layer outputs a detection result of an area where snow is present. The snow accumulation determination device according to claim 1 .

3. The snow accumulation determination device of claim 1, wherein the detection model is trained using an image at a second time, which is a time before the first time, and teaching data in which subject attributes are assigned to the image at the second time.

4. The snow accumulation determination device according to claim 1 , wherein the detection model is updated by updating parameters of all layers or parameters of only the output layer.

5. The snow accumulation determination device of claim 1, wherein the determination unit determines the state of snow accumulation on the solar panel based on a snow accumulation rate, which is the ratio of the area of ​​pixels corresponding to snow on the solar panel to the area of ​​pixels corresponding to the solar panel.

6. The snow accumulation determination device according to claim 5 , wherein the determination unit determines the state of snow accumulation on the solar panel by comparing the snow accumulation rate with a predetermined first threshold value.

7. A first snow accumulation determination device and a second snow accumulation determination device are included, The first snow accumulation determination device is an input unit that receives an input of an image at a first reference time; a detection unit that includes a first input layer and a first output layer and detects an area where snow is present on the solar panel in the image at the first time based on a first detection model that has been trained to be able to detect an area where snow is present on the solar panel; a determination unit that determines the state of snow accumulation on the solar panel, The second snow accumulation determination device is a generation unit that includes a second input layer and a second output layer and generates first attribute data in which attributes of a subject are assigned to the image at the first time based on a second detection model that has been trained to be able to detect an area where snow is present on a solar panel, the second detection model is updated based on the image at the first time and the first attribute data; The first detection model is updated by sharing a file between the second detection model and the first detection model. Snow accumulation determination system.

8. The snow accumulation determination system according to claim 7 , wherein the first snow accumulation determination device further comprises an update unit that updates the second detection model.

9. the second snow accumulation determination device further includes a control unit that determines whether or not the first attribute data needs to be corrected; the control unit makes a determination based on a comparison between the first attribute data and instruction data in which the attribute of the subject is assigned to an image at a second time that is a time before the first time. The snow accumulation determination system according to claim 7 .

10. a pseudo-snow-covered image generating unit that generates a pseudo-snow-covered image by converting pixels of an image at a second time that is a time before the first time into pixels corresponding to snow, and third attribute data that assigns attributes of a subject to the pseudo-snow-covered image; The snow accumulation determination system according to claim 7 , wherein the first detection model is trained using the pseudo snow accumulation image and the third attribute data.

11. Accepting input of an image at a first reference time; detecting an area where snow is present on the solar panel in the image at the first time based on a detection model that includes an input layer and an output layer and that has been trained to be able to detect an area where snow is present on the solar panel; determining the state of snow accumulation on the solar panel; updating the detection model using the image at the first time and teaching data in which the image at the first time is assigned with an attribute of the subject; Snowfall determination method.

12. The snow accumulation determination device of claim 3 further comprises a selection unit that determines whether or not to use the image at the first time or the second time to update the detection model based on a comparison of the determination result of the snow accumulation state on the solar panel determined from the image at the first time or the second time with power plant information including at least one of information regarding the power generation of the solar panel and information regarding the weather in the area where the solar panel is installed.

13. The snow accumulation determination device of claim 12, wherein the power plant information includes at least one of the power generation amount of the solar power plant where the solar panels are installed, the PR value of the solar power plant where the solar panels are installed, and weather data of the area where the solar panels are installed.

14. The selection unit For an image at the first time or the second time determined to have snow accumulation, it is determined whether the power generation amount or the PR value at the time when the image at the first time was acquired is equal to or less than a predetermined second threshold value; If the difference is equal to or less than the second threshold, it is determined that the image at the first time or the second time is used to update the detection model. The snow accumulation determination device according to claim 13.

15. The selection unit For an image at the first time or the second time determined to have snow accumulation, it is determined whether the power generation amount or the PR value at the time when the image at the first time was acquired is equal to or less than a predetermined second threshold value; If the difference is not equal to or less than the second threshold, it is determined that the image at the first time or the second time is not used to update the detection model. The snow accumulation determination device according to claim 13.

Images