Method for detecting photovoltaic modules in an image
A multi-stage detection method using a neural network enhances photovoltaic module contour detection, addressing inaccuracies in existing methods, enabling precise positioning and maintenance support in photovoltaic power plants.
Patent Information
- Application Number
- FR2023013181
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2043-11-28
AI Technical Summary
Existing methods for detecting photovoltaic modules in images, whether threshold detection or neural networks, fail to accurately identify the contours of these modules, leading to inaccurate positioning and detection of their centers.
A multi-stage detection method involving coarse and fine detection stages, using a neural network trained on photovoltaic module images, to enhance contour detection by enlarging an encompassing shape, processing the image to enhance visibility, and applying contour masks for precise module identification.
The method achieves precise detection of photovoltaic module contours, ensuring complete capture and accurate positioning, facilitating monitoring and maintenance in photovoltaic power plants.
Smart Images
Figure 00000013_0000 
Figure 00000014_0000 
Figure 00000015_0000
Abstract
Description
Title of the invention: Method for detecting photovoltaic modules in an image
[0001] The present invention relates to a method for detecting photovoltaic modules in an image. The present invention also relates to an associated computer program product.
[0002] Monitoring the condition of photovoltaic modules in a photovoltaic power plant is a key step in detecting defects in the modules or anticipating potential maintenance actions that could affect the plant's energy production. Indeed, a decrease in production at the photovoltaic module level can result from electrical faults, shading, or soiling of the photovoltaic modules.
[0003] Ideally, in order to diagnose these phenomena and offer decision support, it is necessary to be able to detect all the photovoltaic modules of the power plant and also to determine precisely their outlines, because a drop in production may come from a problem located at the end of the module such as the junction boxes.
[0004] The current state of the art for the detection of photovoltaic modules on images is achieved either by threshold detection methods or by neural network detection methods.
[0005] However, threshold detection methods do not allow for accurate detection of module contours (inaccurate module masks). Furthermore, they generally require inputting measurements related to the modules. Since the module masks are not accurate, the position of the module centers is also inaccurate, and a discrepancy may therefore occur between the actual module contour and the detected module contour.
[0006] Neural network detection methods are, for their part, effective for detecting elements, but are also not satisfactory in the context of a precise search for contours.
[0007] There is therefore a need for a means of detecting more precisely the photovoltaic modules present in an image.
[0008] To this end, the invention relates to a method for detecting photovoltaic modules in an image, the method being implemented by computer and comprising the following steps: a. the reception of an initial image from photovoltaic modules, b. the coarse detection, by a model, of at least one photovoltaic module fully imaged on the initial image and the highlighting of said photovoltaic module on the initial image by an encompassing shape, the encompassing shape encompassing at least a part of the photovoltaic module, c. the determination of a contour mask for each photovoltaic module detected in the initial image, the determination step comprising for each detected photovoltaic module: i. the enlargement of the encompassing shape by a predetermined factor so as to obtain a shape, called the enlarged shape, encompassing a single photovoltaic module in its entirety, called the central module, ii. the processing of an image, called a reduced image, corresponding to the enlarged form, the processing including the binarization of the reduced image and the detection of the contours of the central module on the binarized image, iii. the determination of a contour mask for the photovoltaic module based on the contours detected for the central module, d. the application of each determined contour mask on the initial image to precisely detect the corresponding photovoltaic module.
[0009] According to other advantageous aspects of the invention, the method comprises one or more of the following features, taken individually or in all technically possible combinations:
[0010] - the model is a neural network that has been previously trained on a basis data including images of photovoltaic modules;
[0011] - the processing of the reduced image includes the transformation of the reduced image into grey levels and blurring of the transformed image;
[0012] - the processing of the reduced image includes strengthening the white contours on the blurred image;
[0013] - the binarization of the reduced image is performed by a self-adaptive thresholding of which the decision threshold is determined by the median and mean of the pixels of the reduced image;
[0014] - the detected contours of the central module correspond to a quadrilateral having a area between 60% and 95% of the binarized image;
[0015] - the predetermined factor is chosen so that the enlarged shape has a surface less than twice the surface area of a photovoltaic module;
[0016] - the enlargement of the encompassing shape includes the enlargement of the sides of the encompassing form of the predetermined factor, preferably the predetermined factor being equal to;
[0017] - the contrast of the initial image was adjusted, before the coarse detection step, by an adaptive histogram spreading of the initial image; and
[0018] - the method includes a step of extracting an image from each module photovoltaic depending on the contour mask of said photovoltaic module, and a step of applying one or more signature defect masks to the extracted images in order to identify defects on the detected photovoltaic modules.
[0019] The invention also relates to a computer program product comprising program instructions recorded on a computer-readable medium, for the execution of a detection method as described above, when the computer program is executed on a computer.
[0020] This description also relates to a readable information medium on which a computer program product as previously described is stored.
[0021] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:
[0022] [Fig. 1], [Fig. 1], a schematic view of an example computer enabling the implementation of a method for detecting photovoltaic modules on an image,
[0023] [Fig.2], [Fig.2], a flowchart of an example of the implementation of a method for detecting photovoltaic modules on an image,
[0024] [Fig.3], [Fig.3], a schematic representation of an example of an initial image in which a photovoltaic module has been highlighted by an encompassing shape,
[0025] [Fig.4], [Fig.4], a schematic representation of an example of a reduced image of the initial image of [Fig.3], the reduced image corresponding to the enlarged encompassing shape by a predetermined factor,
[0026] [Fig.5], [Fig.5], a schematic representation of an example of a binarized image of the reduced image of [Fig.4],
[0027] [Fig.6], [Fig.6], a schematic representation of an example of the contours of the central module on the binarized image,
[0028] [Fig.7], [Fig.7], a schematic representation of an example of a contour mask obtained from the detected contours of [Fig.6], and
[0029] [Fig.8], [Fig.8], a schematic representation of an example of the application of the contour mask of [Fig.7] on the initial image, followed by the extraction of the image of the photovoltaic module.
[0030] A calculator 10 and a computer program product 12 are illustrated by [Fig.1].
[0031] Calculator 10 is preferably a computer.
[0032] More generally, the calculator 10 is an electronic calculator designed to manipulate and / or transform data represented as electronic or physical quantities in registers of calculator 10 and / or memories into other similar data corresponding to physical data in memories, registers or other types of display, transmission or storage devices.
[0033] The calculator 10 interacts with the computer program product 12.
[0034] As illustrated in [Fig. 1], the computer 10 comprises a processor 14 including a data processing unit 16, memories 18, and a data storage reader 20. In the example illustrated in [Fig. 1], the computer 10 includes a keyboard 22 and a display unit 24.
[0035] The computer program product 12 includes an information carrier 26.
[0036] The information support 26 is a support readable by the computer 10, usually by the data processing unit 16. The readable information support 26 is a medium suitable for storing electronic instructions and capable of being coupled to a bus of a computer system.
[0037] By way of example, the information medium 26 is a floppy disk or flexible disk (from the English name "Floppy disk"), an optical disk, a CD-ROM, a magneto-optical disk, a ROM memory, a RAM memory, an EPROM memory, an EEPROM memory, a magnetic card or an optical card.
[0038] The computer program 12, comprising program instructions, is stored on the information support 26.
[0039] The computer program 12 can be loaded onto the data processing unit 16 and is adapted to drive the implementation of a method for detecting MPV photovoltaic modules on an image, when the computer program 12 is implemented on the processing unit 16 of the computer 10.
[0040] Alternatively, the calculator 10 is in the form of an electronic card comprising microcontrollers, or integrated circuits.
[0041] The operation of the calculator 10 will now be described with reference to [Fig.2], which schematically illustrates an example of the implementation of a method for detecting MPV photovoltaic modules on an image, and to Figures 3 to 8, which illustrate examples of steps in the method.
[0042] Each MPV photovoltaic module (or photovoltaic panel) is formed from an assembly of photovoltaic cells. MPV photovoltaic modules belong, for example, to a photovoltaic power plant.
[0043] The detection method includes a step 100 of receiving an initial image IM of several MPV photovoltaic modules (or an image of a photovoltaic power plant). Step 100 is implemented by the computer 10 in interaction with the computer program 12, i.e., is implemented by computer.
[0044] The initial image IM is preferably an image seen from above. By the term "view from above", it is understood that the images were taken from a high vantage point allowing, for example, the imagery of building roofs.
[0045] The initial IM image was, for example, acquired by a satellite system. Alternatively, the initial IM image was acquired by an acquisition system, comprising one or more cameras, mounted on an aircraft or drone.
[0046] The initial image IM is, for example, a color image (RGB from the English "Red Green Blue", translated into French as Rouge Vert Bleu).
[0047] Alternatively, the initial image IM is a greyscale image, or an infrared image, or even an electroluminescent image.
[0048] The detection method includes a step 200 of coarse detection, by a model, of at least one MPV photovoltaic module fully imaged on the initial image IM and highlighting said MPV photovoltaic module on the initial image IM by an encompassing shape F. The coarse detection allows for the detection of the location of an MPV photovoltaic module, but without precise detection of the contours of the MPV photovoltaic module. Step 200 is implemented by the computer 10 in interaction with the computer program 12, i.e., is implemented by computer.
[0049] The encompassing shape F encompasses at least a part of the MPV photovoltaic module, preferably at least three-quarters of the surface of the MPV photovoltaic module.
[0050] The encompassing shape F is, for example, a rectangle. Alternatively, the encompassing shape F is a circle or any other geometric shape.
[0051] An example of an initial image IM with superimposed an encompassing shape F (rectangle) encompassing an MPV photovoltaic module is illustrated by [Fig.3].
[0052] Preferably, the contrast of the initial IM image was adjusted, before the coarse detection step, by adaptive histogram spreading of the initial IM image. This makes the MPV photovoltaic modules more visible in the initial IM image.
[0053] For example, histogram spreading is performed as follows. The image is divided into fixed-size "blocks," for example 16x16, and a contrast limit is set on this area (for example, less than 50). In each of these blocks, histogram equalization is performed:
[0054] Let 255 be the number of pixel values in the image.
[0055] We define nk as the number of occurrences of the pixel value x k.
[0056] To each pixel with value xk, a new value 5 del is associated that:
[0057] = ________255____:____yk ^k number of pixels in the image j=(>J
[0058] On the new histogram obtained, if a value is above the maximum contrast, the number of occurrences above it is uniformly redistributed across all the values in the histogram. Finally, these operations are performed on all blocks of the image.
[0059] In one embodiment, the model is a neural network that has been previously trained on a database comprising images of MPV photovoltaic modules.
[0060] The images in the database are of the same nature as the initial image IM (e.g., infrared image if the initial image IM is an infrared image, or color images if the initial image IM is a color image, etc.).
[0061] An example of training the model will now be described.
[0062] In this example, the images in the database are infrared images of MPV photovoltaic modules with varying tilts, as well as elements that interfere with detection. The training database used comprises 2304 images (2048 for training and validation and 256 for testing). The database includes the images, as well as the targets, which in this case are rectangles circumscribed around the MPV photovoltaic modules (center coordinates + width + height).
[0063] In this example, the model is based on an instance segmentation algorithm, such as the YOLO algorithm (in this case YOLO V8). Training is performed in batches with a gradient descent algorithm, in this case the SGDM algorithm (stochastic gradient descent with inertia).
[0064] In this example, the performance of the neural network is evaluated using the MAP metric (Mean Average Precision). A detection is considered correct if it exceeds a threshold (called loU), which we set to 0.5 in this case. It corresponds to the ratio between the area at the intersection of the detected rectangle and that of the expected circumscribed rectangle divided by the area of the union of these two rectangles. Precision is the ratio of correct detections to all detections made by the network; it is the The network's ability to make accurate predictions. Recall is the ratio of correct detections across all present photovoltaic modules; it represents the network's capacity to detect photovoltaic modules. The area under the curve representing accuracy versus recall provides an indicator of the model's average accuracy, denoted AP (average accuracy).
[0065] In this example, the model is also evaluated using the metric MAPq 5 g 95 which is the average of the MAPs with decision thresholds ranging from 0.5 at 0.95 with a step of 0.05. We trained our neural network to detect only the entire modules in the image. We obtained 100% detection of the photovoltaic modules, a MAP = 0.971 and a MAPq 5 q 95 = 0.55.
[0066] The detection method includes a step 300 of determining a contour mask Mc for each photovoltaic module MPV detected on the initial image IM. Step 300 is implemented by the computer 10 in interaction with the product computer program 12, i.e., is implemented by computer.
[0067] The determination step 300 includes a substep 310 of enlarging the encompassing shape F by a predetermined factor so as to obtain a shape, called the enlarged shape, encompassing a single MPV photovoltaic module in its entirety, called the central MPV module. c- The enlarged shape therefore has a surface area greater than the surface area of an MPV photovoltaic module.
[0068] Advantageously, the predetermined factor is chosen so that the enlarged shape has a surface area less than twice the surface area of an MPV photovoltaic module. This allows the enlarged shape to encompass only a single MPV photovoltaic module in its entirety.
[0069] In one embodiment, the enlargement of the encompassing shape F includes the enlargement of the sides of the bounding shape F by a predetermined factor (for example, when the bounding shape F is a rectangle). The predetermined factor is, for example, equal to
[0070] Alternatively, the predetermined factor is such that the sides of the enlarged shape are enlarged by a value between 5% and 40% of the value of one side of the encompassing shape F, preferably between 8% and 15% of the value of one side of the encompassing shape F.
[0071] The determination step 300 includes a substep 320 of image processing, called the reduced image MRI, corresponding to the enlarged shape so as to detect the contours C of the central module MPV_c-
[0072] The processing includes binarizing the reduced IMR image and detecting the C contours of the central MPV module c on the binarized IMB image. The binarization of a image consists of assigning a white or black color to the pixels of the image based on the value of the pixels.
[0073] For example, the binarization of the reduced IMR image is carried out by a self-adaptive thresholding whose decision threshold is determined by the median and the mean of the pixels of the reduced IMR image.
[0074] The detection of the C contours of the central MPV module c is performed by considering the white boundaries on the image that describe a quadrilateral. Since the image contains a single photovoltaic module, the quadrilateral with an area greater than 60% and less than 95% is chosen. Thus, the detected C contours of the central MPV module c advantageously correspond to a quadrilateral having an area between 60% and 95% of the binarized IMB image.
[0075] An example of a reduced IMR image corresponding to the enlarged shape is illustrated by [Fig. 4] (the starting point being [Fig. 3]). An example of a binarized IMb image is illustrated by [Fig. 5]. An example of the detected C contours of the target MPV photovoltaic module is illustrated by [Fig. 6].
[0076] Preferably, the processing of the reduced IMR image includes, before binarization, transforming the reduced IMR image into grayscale and blurring the transformed image. For example, after converting the image to grayscale, a kernel is considered (size = 5% of the image but at least = 3). The central element of the kernel is replaced by the median value of the kernel's pixels. This method makes it possible to remove snow-like noise.
[0077] Preferably, the processing of the reduced IMR image includes, before binarization, the strengthening of the white contours on the blurred image.
[0078] For example, to increase the continuity of the contours, a convolution is performed on the image with a white mask of dimension 3x3. The white regions bordering 1 black pixel are dilated by the convolution, this makes it possible to close and strengthen the contours of the target MPV photovoltaic module.
[0079] The determination step 300 includes a substep 330 of determining a contour mask Mc for the MPV photovoltaic module based on the contours C detected for the MPV-c central module
[0080] The contour mask Mc corresponds to the surface delimited by the detected contours C of the MPV photovoltaic module. An example of a contour mask Mc is illustrated in [Fig.7].
[0081] The detection method includes a step 400 of applying each contour mask Mc determined on the initial image IM to precisely detect the corresponding MPV photovoltaic module. Step 400 is implemented by the computer 10 interacting with the computer program product 12, that is, is implemented by computer.
[0082] The [Fig. 8] on the left illustrates the fine detection of the MPV photovoltaic module considered in [Fig.3].
[0083] Optionally, the detection method includes a step 500 of extracting an IMPV image of each MPV photovoltaic module based on the contour mask Mc of said MPV photovoltaic module. Step 500 is implemented by the computer 10 in interaction with the computer program 12, i.e., is implemented by computer.
[0084] Figure 9 on the right illustrates the extraction of the IMPV image of an MPy photovoltaic module.
[0085] Optionally, the detection method includes a step 600 of applying one or more signature defect masks to the extracted IMPV images in order to identify defects on the detected MPV photovoltaic modules. Step 600 is implemented by the computer 10 in interaction with the product computer program 12, i.e., is computer-implemented.
[0086] Optionally, the detection method includes a step of maintenance of the MPV photovoltaic modules according to the defects identified during step 600.
[0087] Thus, the present method allows, through a multi-stage detection (coarse and fine), the precise detection of MPV photovoltaic modules, in particular their contours C. A key step is the increase of the detection area (enclosing shape F) of the MPV photovoltaic module on the original image, which ensures that the MPV photovoltaic module is captured in its entirety.
[0088] Such a process makes it possible to facilitate the field of electricity production by photovoltaic power plants, in particular the monitoring, diagnosis and decision support during the operation of these photovoltaic power plants.
[0089] A person skilled in the art will understand that the embodiments and variants described above can be combined to form new embodiments provided that they are technically compatible.
Claims
1. Demands Method for detecting photovoltaic modules (PVMs) in an image, the method being implemented by computer and comprising the following steps: a. the reception of an initial image (IM) of photovoltaic modules (PV), b. the coarse detection, by a model, of at least one photovoltaic module (PVM) imaged entirely on the initial image (IM) and the highlighting of said photovoltaic module (PVM) on the initial image (IM) by an encompassing shape (F), the encompassing shape (F) encompassing at least a part of the photovoltaic module (PVM), c. the determination of a contour mask (Mc) for each photovoltaic module (PVM) detected on the initial image (IM), the determination step comprising for each detected photovoltaic module (PVM): i. the enlargement of the encompassing shape (F) by a predetermined factor so as to obtain a shape, called the enlarged shape, encompassing a single photovoltaic module (PVM) in its entirety, called the central module (PVM-c), the predetermined factor being chosen so that the enlarged shape has a surface area less than twice the surface area of a photovoltaic module (PVM), ii. the processing of an image, called reduced image (IMR), corresponding to the enlarged form, the processing including the binarization of the reduced image (IMr) and the detection of the contours (C) of the central module (MPV c) on the binarized image (IMB), iii. the determination of a contour mask (Mc) for the photovoltaic module (MPV) as a function of the contours (C) detected for the central module (MPV-c), the contour mask (Mc) corresponding to the surface delimited by the contours (C) detected of the photovoltaic module (MPV), d. the application of each contour mask (Mc) determined on the initial image (IM) to finely detect the corresponding photovoltaic module (MPV).
2. A method according to claim 1, wherein the model is a neural network that has been previously trained on a database comprising images of photovoltaic modules (PVM).
3. A method according to claim 1 or 2, wherein the reduced image processing (RIP) comprises the transformation of the reduced image (RIP) into greyscale and the blurring of the transformed image.
4. A method according to claim 3, wherein the reduced image processing (IMR) includes strengthening the white outlines on the blurred image.
5. A method according to any one of claims 1 to 4, wherein the binarization of the reduced image (RIM) is carried out by a self-adaptive thresholding whose decision threshold is determined by the median and mean of the pixels of the reduced image (RIM).
6. A method according to any one of claims 1 to 5, wherein the detected contours (C) of the central module (MPV c) correspond to a quadrilateral having an area between 60% and 95% of the binarized image (IMB).
7. A method according to any one of claims 1 to 6, wherein the predetermined factor is chosen so that the enlarged shape has an area less than twice the area of a photovoltaic module (PVM).
8. A method according to any one of claims 1 to 7, wherein the enlargement of the encompassing shape (F) comprises enlarging the sides of the encompassing shape (F) by a predetermined factor, preferably the predetermined factor being equal to
9. A method according to any one of claims 1 to 8, wherein the contrast of the initial image (IM) has been adjusted, before the coarse detection step, by an adaptive histogram spread of the initial image (IM).
10. A method according to any one of claims 1 to 9, wherein the method comprises a step of extracting an image (IMPV) of each photovoltaic module (MPV) based on the contour mask (MC) of said photovoltaic module (MPV), and a step of applying one or more defect signature masks to the 12 images (IMPV) extracted in order to identify defects on the detected photovoltaic modules (MPV).
11. Product computer program comprising program instructions recorded on a computer-readable medium, for the execution of a detection method according to any one of claims 1 to 10 when the computer program is executed on a computer.