Method for detecting photovoltaic modules on an image

Abstract

Method for detecting photovoltaic modules on an image The present invention relates to a method for detecting photovoltaic modules on an image, the method comprising: receiving an initial image of photovoltaic modules, coarsely detecting, by a model, at least one photovoltaic module imaged entirely on the initial image and highlighting said photovoltaic module on the initial image by an enclosing shape, determining a contour mask for each photovoltaic module detected on the initial image as a function of the enclosing shape, and applying each determined contour mask to the initial image to finely detect the corresponding photovoltaic module. Figure for abstract: 1

Description

Title of the invention: Method for detecting photovoltaic modules on an image

[0001] The present invention relates to a method for detecting photovoltaic modules on an image. The present invention also relates to an associated computer program product.

[0002] Checking the condition of photovoltaic modules in a photovoltaic power plant is a key step in detecting defects in the modules or anticipating possible maintenance actions that could affect the power plant's energy production. Indeed, the drop in production in the photovoltaic modules can result from electrical defects, shading or dirt on the photovoltaic modules.

[0003] Ideally, to diagnose these phenomena and provide decision-making support, it is necessary to be able to detect all the photovoltaic modules in the power plant and also to precisely determine their contours, because a drop in production may come from a problem located at the end of the module, such as the junction boxes, for example.

[0004] The current state of the art for the detection of photovoltaic modules on images is carried out either by threshold detection methods or by neural network detection methods.

[0005] However, threshold detection methods do not allow good detection of module contours (inaccurate module masks). In addition, they generally require providing measurements relating to the modules. However, since the module masks are not precise, the position of the module centers is also not precise, and it then happens that there is a gap between the actual contour of the module and the contour of the detected module.

[0006] Neural network detection methods are, for their part, effective for detecting elements, but are also not satisfactory in the context of precise contour research.

[0007] There is therefore a need for a means of more precisely detecting the photovoltaic modules present in an image.

[0008] To this end, the invention relates to a method for detecting photovoltaic modules on an image, the method being implemented by computer and comprising the following steps: a. receiving an initial image of photovoltaic modules, b. coarse detection, by a model, of at least one photovoltaic module imaged entirely on the initial image and highlighting said photovoltaic module on the initial image by an enclosing shape, the enclosing shape enclosing at least part of the photovoltaic module, c. determining a contour mask for each photovoltaic module detected on the initial image, the determination step comprising for each photovoltaic module detected: i. enlarging the enclosing shape by a predetermined factor so as to obtain a shape, called the enlarged shape, enclosing a single photovoltaic module in its entirety, called the central module, ii. the processing of an image, called a reduced image, corresponding to the enlarged form, the processing comprising the binarization of the reduced image and the detection of the contours of the central module on the binarized image, iii. determining a contour mask for the photovoltaic module based on the contours detected for the central module, d. applying each contour mask determined on the initial image to finely detect the corresponding photovoltaic module.

[0009] According to other advantageous aspects of the invention, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0010] - the model is a neural network that has been previously trained on a base data including images of photovoltaic modules;

[0011] - the processing of the reduced image comprises the transformation of the reduced image into grayscale and blurring of the transformed image;

[0012] - the processing of the reduced image includes the reinforcement of the white contours on the blurred image;

[0013] - the binarization of the reduced image is carried out by self-adaptive thresholding of which the decision threshold is determined by the median and the mean of the pixels of the reduced image;

[0014] - the detected contours of the central module correspond to a quadrilateral having a area between 60% and 95% of the binarized image;

[0015] - the predetermined factor is chosen so that the enlarged shape has an area in less than twice the surface area of ​​a photovoltaic module;

[0016] - the enlargement of the enclosing shape includes the enlargement of the sides of the encompassing form of the predetermined factor, preferably the predetermined factor being equal to;

[0017] - the contrast of the initial image was adjusted, before the coarse detection step, by an adaptive histogram spread of the initial image; and

[0018] - the method comprises a step of extracting an image from each photo module voltaic according to the contour mask of said photovoltaic module, and a step of applying one or more defect signature masks to the extracted images in order to identify defects on the detected photovoltaic modules.

[0019] The invention also relates to a computer program product comprising program instructions recorded on a computer-readable medium, for executing a detection method as described above, when the computer program is executed on a computer.

[0020] The present description also relates to a readable information medium on which a computer program product as previously described is stored.

[0021] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0022] [Fig. 1], [Fig. 1], a schematic view of an example of a computer allowing the implementation of a method for detecting photovoltaic modules on an image,

[0023] [Fig.2], [Fig.2], a flowchart of an example of implementation of a method for detecting photovoltaic modules on an image,

[0024] [Fig.3], [Fig.3], a schematic representation of an example of an initial image on which a photovoltaic module has been highlighted by an enclosing shape,

[0025] [Fig.4], [Fig.4], a schematic representation of an example of a reduced image of the initial image of [Fig.3], the reduced image corresponding to the encompassing shape enlarged by a predetermined factor,

[0026] [Fig.5], [Fig.5], a schematic representation of an example of a binarized image of the reduced image of [Fig.4],

[0027] [Fig.6], [Fig.6], a schematic representation of an example of the contours of the central module on the binarized image,

[0028] [Fig.7], [Fig.7], a schematic representation of an example of a contour mask obtained from the detected contours of [Fig.6], and

[0029] [Fig.8], [Fig.8], a schematic representation of an example of the application of the contour mask of [Fig.7] on the initial image, followed by the extraction of the image of the photovoltaic module.

[0030] A calculator 10 and a computer program product 12 are illustrated in [Fig.l].

[0031] The calculator 10 is preferably a computer.

[0032] More generally, the computer 10 is an electronic computer capable of manipulating and / or transforming data represented as electronic or physical quantities in computer registers 10 and / or memories into other similar data corresponding to physical data in memories, registers or other types of display, transmission or storage devices.

[0033] The calculator 10 interacts with the computer program product 12.

[0034] As illustrated by [Fig.l], the computer 10 comprises a processor 14 comprising a data processing unit 16, memories 18 and an information medium reader 20. In the example illustrated by [Fig.l], the computer 10 comprises a keyboard 22 and a display unit 24.

[0035] The computer program product 12 comprises an information medium 26.

[0036] The information medium 26 is a medium readable by the computer 10, usually by the data processing unit 16. The readable information medium 26 is a medium suitable for storing electronic instructions and capable of being coupled to a bus of a computer system.

[0037] By way of example, the information medium 26 is a floppy disk or flexible disk (also known as a "Floppy disk"), an optical disk, a CD-ROM, a magneto-optical disk, a ROM memory, a RAM memory, an EPROM memory, an EEPROM memory, a magnetic card or an optical card.

[0038] On the information medium 26 is stored the computer program 12 comprising program instructions.

[0039] The computer program 12 is loadable onto the data processing unit 16 and is adapted to cause the implementation of a method for detecting MPV photovoltaic modules on an image, when the computer program 12 is implemented on the processing unit 16 of the computer 10.

[0040] Alternatively, the calculator 10 is in the form of an electronic card comprising microcontrollers, or integrated circuits.

[0041] The operation of the calculator 10 will now be described with reference to [Fig. 2], which schematically illustrates an example of implementation of a method for detecting MPV photovoltaic modules on an image, and to FIGS. 3 to 8 which illustrate examples of steps of the method.

[0042] Each MPV photovoltaic module (or photovoltaic panel) is formed from the assembly of photovoltaic cells. MPV photovoltaic modules belong, for example, to a photovoltaic power plant.

[0043] The detection method comprises a step 100 of receiving an initial image IM of several MPV photovoltaic modules (or an image of a photovoltaic power plant). Step 100 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0044] The initial image IM is preferably an image seen from the sky. By the term "view from the sky", it is understood that the images were taken from a high point of view allowing, for example, to image the roofs of buildings.

[0045] The initial image IM has, for example, been acquired by a satellite system. Alternatively, the initial image IM has been acquired by an acquisition system, comprising one or more cameras, mounted on an aircraft or a drone.

[0046] The initial image IM is, for example, a color image (RGB from the English “Red Green Blue”, translated into French as Rouge Vert Bleu).

[0047] Alternatively, the initial image IM is a grayscale image, or an infrared image, or even an electroluminescent image.

[0048] The detection method comprises a step 200 of coarse detection, by a model, of at least one MPV photovoltaic module imaged entirely on the initial image IM and the highlighting of said MPV photovoltaic module on the initial image IM by an encompassing shape F. The coarse detection allows detection of the location of an MPV photovoltaic module, but without precise detection of the contours of the MPV photovoltaic module. Step 200 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0049] The encompassing shape F encompasses at least a portion of the MPV photovoltaic module, preferably at least three-quarters of the surface area of ​​the MPV photovoltaic module.

[0050] The enclosing shape F is, for example, a rectangle. Alternatively, the enclosing shape F is a circle or any other geometric shape.

[0051] An example of an initial image IM with a superimposed enclosing shape F (rectangle) enclosing a photovoltaic module MPV is illustrated by [Fig.3].

[0052] Preferably, the contrast of the initial image IM has been adjusted, before the coarse detection step, by an adaptive histogram spread of the initial image IM. This makes it possible to make the MPV photovoltaic modules more visible on the initial image IM.

[0053] For example, histogram spreading is performed in the following manner. The image is divided into "blocks" of fixed size, for example 16*16, and a contrast limitation is set on this area (for example less than 50). In each of these blocks, a histogram equalization is performed:

[0054] Let 255 be the number of pixel values ​​in the image.

[0055] We define nk as the number of occurrences of the pixel value xk.

[0056] To each pixel of value xk, we associate a new value ytcl that:

[0057] = 255 vk n ■■ A' number of pixels in the image

[0058] On the new histogram obtained, if a value is above the maximum contrast the number of higher occurrences is uniformly redistributed over all the values ​​of the histogram. Finally, we perform these operations on all blocks in the image.

[0059] In an exemplary embodiment, the model is a neural network which has been previously trained on a database comprising images of MPV photovoltaic modules.

[0060] The images in the database are of the same nature as the initial image IM (e.g.: infrared image if the initial image IM is an infrared image, or color images if the initial image IM is a color image, etc.).

[0061] An example of training the model will now be described.

[0062] In this example, the images in the database are infrared images imaging MPV photovoltaic modules with variable inclinations, as well as detection disturbing elements. The training database used comprises 2304 images (2048 for training and validation and 256 for testing). The database comprises the images, as well as the targets which are, in this case, rectangles circumscribed to the MPV photovoltaic modules (coordinates of the center + width + height).

[0063] In this example, the model is based on an instance segmentation algorithm, such as the YOLO algorithm (in this case YOLO V8). The training is carried out in batches with a gradient descent algorithm, in this case the SGDM algorithm (stochastic gradient descent with inertia).

[0064] In this example, the performance of the neural network is evaluated using the MAP metric (from the English "Mean Average Precision"). A detection is considered correct if it is greater than a threshold (called loU) which in our case is set at 0.5. It corresponds to the ratio between the surface at the intersection of the detected rectangle and that of the expected circumscribed rectangle on the surface of the union of these 2 rectangles. Precision is the ratio of correct detections over all detections made by the network; it is the network's ability to make good predictions. Recall is the ratio of correct detections over all photovoltaic modules present; it is the network's ability to detect photovoltaic modules. The area under the curve of precision as a function of recall provides an indicator of the average precision of the model, called AP (average precision).

[0065] In this example, the model is also evaluated using the MAP^ 5 0 95 metric which is the average of the MAPs with decision thresholds ranging from 0.5 to 0.95 with a step size of 0.05. We trained our neural network to detect only the entire modules in the image. We obtained a detection of 100% of the photovoltaic modules, a MAP = 0.971 and a MAPq^q 95 — 0.55.

[0066] The detection method comprises a step 300 of determining a contour mask Mc for each photovoltaic module MPV detected on the initial image IM. Step 300 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0067] The determination step 300 comprises a sub-step 310 of enlarging the encompassing shape F by a predetermined factor so as to obtain a shape, called enlarged shape, encompassing a single MPV photovoltaic module in its entirety, called central module MPVo. The enlarged shape therefore has a surface area greater than the surface area of ​​an MPV photovoltaic module.

[0068] Advantageously, the predetermined factor is chosen so that the enlarged shape has a surface area less than twice the surface area of ​​an MPV photovoltaic module. This allows the enlarged shape to encompass only a single MPV photovoltaic module in its entirety.

[0069] In an exemplary embodiment, enlarging the enclosing shape F comprises enlarging the sides of the enclosing shape F by the predetermined factor (e.g., when the enclosing shape F is a rectangle). The predetermined factor is, for example, equal to

[0070] Alternatively, the predetermined factor is such that the sides of the enlarged shape are enlarged by a value of between 5% and 40% of the value of one side of the enclosing shape F, preferably between 8% and 15% of the value of one side of the enclosing shape F.

[0071] The determination step 300 comprises a sub-step 320 of processing an image, called reduced image IMR, corresponding to the enlarged shape so as to detect the contours C of the central module MPV.c-

[0072] The processing comprises the binarization of the reduced image IMR and the detection of the contours C of the central module MPV c on the binarized image IMB. The binarization of an image consists of assigning a white or black color to the pixels of the image according to the value of the pixels.

[0073] For example, the binarization of the reduced IMR image is carried out by self-adaptive thresholding whose decision threshold is determined by the median and the mean of the pixels of the reduced IMR image.

[0074] The detection of the contours C of the central module MPV c is carried out by considering the white limits on the image which describe a quadrilateral. As the image contains a single photovoltaic module, the quadrilateral with an area greater than 60% and less than 95% is chosen. Thus, the detected contours C of the central module MPV c advantageously correspond to a quadrilateral having an area between 60% and 95% of the binarized image IMB.

[0075] An example of a reduced IMR image corresponding to the enlarged shape is illustrated in [Fig.4] (the starting point being [Fig.3]). An example of a binarized image IMb is illustrated in [Fig.5]. An example of the detected C-contours of the target MPV photovoltaic module is illustrated in [Fig.6].

[0076] Preferably, the processing of the reduced IMR image comprises, before binarization, the transformation of the reduced IMR image into gray levels and the blurring of the transformed image. For example, after the image has been converted into gray levels, a kernel is considered (size = 5% of the image but at least = 3). The central element of the kernel is replaced by the median value of the kernel pixels. This method makes it possible to remove snow-type noise.

[0077] Preferably, the processing of the reduced IMR image comprises, before binarization, the reinforcement of the white contours on the blurred image.

[0078] For example, to increase the continuity of the contours, a convolution is performed on the image with a white mask of dimension 3x3. The white regions bordering 1 black pixel are dilated by the convolution, this makes it possible to close and reinforce the contours of the target MPV photovoltaic module.

[0079] The determination step 300 comprises a sub-step 330 of determining a contour mask Mc for the MPV photovoltaic module as a function of the contours C detected for the central module MPV-c-

[0080] The contour mask Mc corresponds to the surface delimited by the detected contours C of the MPV photovoltaic module. An example of a contour mask Mc is illustrated in [Fig.7].

[0081] The detection method comprises a step 400 of applying each determined contour mask Mc to the initial image IM to finely detect the corresponding MPV photovoltaic module. Step 400 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0082] [Fig. 8] on the left illustrates the fine detection of the MPV photovoltaic module considered in [Fig.3].

[0083] Optionally, the detection method comprises a step 500 of extracting an IMPV image of each MPV photovoltaic module as a function of the contour mask Mc of said MPV photovoltaic module. Step 500 is implemented by the computer 10 in interaction with the computer program product 12, i.e. is implemented by computer.

[0084] Figure 9 on the right illustrates the extraction of the IMPV image of an MPV photovoltaic module.

[0085] Optionally, the detection method comprises a step 600 of applying one or more defect signature masks to the extracted IMPV images in order to identify defects on the detected MPV photovoltaic modules. Step 600 is implemented by the computer 10 in interaction with the program product computer 12, that is, is implemented by computer.

[0086] Optionally, the detection method comprises a step of maintenance of the MPV photovoltaic modules according to the defects identified during step 600.

[0087] Thus, the present method makes it possible, by means of a detection in several stages (coarse and fine), to precisely detect MPV photovoltaic modules, in particular their contours C. A key stage is the increase in the detection zone (encompassing shape F) of the MPV photovoltaic module on the original image which makes it possible to ensure that the entire MPV photovoltaic module is captured.

[0088] Such a method makes it possible to facilitate the field of electricity production by photovoltaic power plants, in particular monitoring, diagnosis and decision-making support during the operation of these photovoltaic power plants.

[0089] Those skilled in the art will understand that the previously described embodiments and variants may be combined to form new embodiments provided that they are technically compatible.

Claims

Claims

1. A method of detecting photovoltaic modules (PVMs) in an image, the method being computer-implemented and comprising the following steps: a. receiving an initial image (IM) of photovoltaic modules (PVMs), b. the rough detection, by a model, of at least one photovoltaic module (MPV) imaged entirely on the initial image (IM) and the highlighting of said photovoltaic module (MPV) on the initial image (IM) by an enclosing shape (F), the enclosing shape (F) enclosing at least part of the photovoltaic module (MPV), c. determining a contour mask (Mc) for each photovoltaic module (MPV) detected on the initial image (IM), the determination step comprising for each photovoltaic module (MPV) detected: i. enlarging the enclosing shape (F) by a predetermined factor so as to obtain a shape, called enlarged shape, enclosing a single photovoltaic module (MPV) in its entirety, called central module (MPV.c), ii. the processing of an image, called a reduced image (IMR), corresponding to the enlarged form, the processing comprising the binarization of the reduced image (IMR) and the detection of the contours (C) of the central module (M pv c) on the binarized image (IMB), iii. the determination of a contour mask (Mc) for the photovoltaic module (MPV) based on the contours (C) detected for the central module (MPV-c), d. the application of each contour mask (Mc) determined on the initial image (IM) to finely detect the corresponding photovoltaic module (MPV).

2. The method of claim 1, wherein the model is a neural network that has been previously trained on a database comprising images of photovoltaic modules (PVMs).

3. A method according to claim 1 or 2, wherein processing the reduced image (IMR) comprises transforming the reduced image (IMr) into grayscale and routing the transformed image.

4. The method of claim 3, wherein processing the reduced image (IMR) comprises enhancing white contours on the blurred image.

5. Method according to any one of claims 1 to 4, in which the binarization of the reduced image (IMR) is carried out by self-adaptive thresholding whose decision threshold is determined by the median and the mean of the pixels of the reduced image (IMR).

6. Method according to any one of claims 1 to 5, in which the detected contours (C) of the central module (MPV c) correspond to a quadrilateral having an area between 60% and 95% of the binarized image (IMB).

7. A method according to any one of claims 1 to 6, wherein the predetermined factor is chosen so that the enlarged shape has an area less than twice the area of ​​a photovoltaic module (M

8. pvj- A method according to any one of claims 1 to 7, wherein enlarging the enclosing shape (F) comprises enlarging the sides of the enclosing shape (F) by the predetermined factor, preferably the predetermined factor being equal to

9. A method according to any one of claims 1 to 8, wherein the contrast of the initial image (IM) has been adjusted, before the coarse detection step, by an adaptive histogram spreading of the initial image (IM).

10. Method according to any one of claims 1 to 9, wherein the method comprises a step of extracting an image (IMPV) of each photovoltaic module (MPV) as a function of the contour mask (MC) of said photovoltaic module (MPV), and a step of applying one or more defect signature masks to the extracted images (IMPV) in order to identify defects on the detected photovoltaic modules (MPV).

11. A computer program product comprising program instructions recorded on a computer-readable medium, for executing a detection method according to any one of claims 1 to 10 when the computer program is executed on a computer.

Images