Foreign matter detection method and device for photovoltaic panel, storage medium and terminal

Abstract

The invention discloses a foreign matter detection method and device for a photovoltaic panel, a storage medium and a terminal. The foreign matter detection method comprises the following steps: acquiring an initial image of the photovoltaic panel; preprocessing the initial image to obtain a to-be-detected image; detecting the to-be-detected image based on a Reti naNet model to obtain a plurality of initial foreign matter detection frames; and screening the initial foreign matter detection frame to obtain a final foreign matter detection result. According to the method, an attention mechanism module is added into a backbone network of the RetinaNet model to refine features, and the improved GIoU Loss is used as a loss function to solve the problem of class imbalance in target detection, so that the model can realize high-accuracy detection under a complex background or different illumination conditions; moreover, the method can achieve the quick detection while guaranteeing the high detection accuracy, improves the cleaning efficiency and accuracy, and also reduces the operation and maintenance cost.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic panels, and in particular to a foreign matter detection method, device, storage medium and terminal for photovoltaic panels. Background Art

[0002] As the core component of the photovoltaic power generation system, the performance of photovoltaic panels is directly related to the power generation efficiency and stability of the entire system. However, in actual applications, foreign objects often adhere to the surface of photovoltaic panels due to various factors (such as dust, bird droppings, leaves, weeds, etc.). These foreign objects not only affect the lighting effect of photovoltaic panels, but also increase the thermal stress of photovoltaic panels and accelerate material aging, thereby reducing power generation efficiency and service life.

[0003] Traditional photovoltaic panel foreign body detection methods mainly rely on manual inspections, which is not only time-consuming and laborious, but also difficult to achieve comprehensive and timely detection. Especially in large-scale photovoltaic power stations, the efficiency and accuracy of manual inspections are even more difficult to guarantee.

[0004] Although the automatic cleaning equipment on the market has reduced labor costs to a certain extent, its ability to identify dust and foreign matter is limited. In complex and changeable natural environments, these devices often find it difficult to accurately determine the type and distribution of pollutants, resulting in unsatisfactory cleaning results, and may even result in incomplete or excessive cleaning, which in turn reduces cleaning efficiency and increases equipment wear. In addition, there are also monitoring devices installed to monitor the cleanliness of photovoltaic panels, but they lack intelligent processing capabilities and cannot effectively identify and classify dust and foreign matter on the surface of photovoltaic panels in real time, let alone automatically adjust cleaning strategies according to pollution conditions.

[0005] Therefore, how to solve the deficiencies and shortcomings of existing technologies through effective photovoltaic panel foreign body detection methods has become an important issue that researchers in this field need to solve urgently. Summary of the invention

[0006] The purpose of the present invention is to address the above-mentioned problems and to provide a method, device, storage medium and terminal for detecting foreign matter on the surface of an energy storage box.

[0007] The technical solution of the present invention is: a foreign body detection method for photovoltaic panels, comprising the following steps: obtaining an initial image of the photovoltaic panel, the initial image being an RGB image; preprocessing the initial image to obtain an image to be detected, the image to be detected being an HSV image; creating a RetinaNet model, detecting the image to be detected based on the RetinaNet model to obtain a number of initial foreign body detection frames and foreign body detection frame information corresponding to the several initial foreign body frames; screening the initial foreign body detection frames to obtain a final foreign body detection result.

[0008] As an improvement of an embodiment of the present invention, the preprocessing specifically includes: using the Canny edge detection algorithm to obtain edge information of the photovoltaic panel in the initial image, performing contour detection on the position of the photovoltaic panel based on the edge information to obtain the photovoltaic panel area image in the initial image, scaling and image enhancement processing on the photovoltaic panel area image, and denoising and color space conversion on the enhanced photovoltaic panel area image to obtain the image to be detected.

[0009] As an improvement of an embodiment of the present invention, the RetinaNet model specifically includes: a backbone network, a feature pyramid and a prediction subnetwork; the backbone network is a deep residual network ResNet-50, which is used to extract the features of the image to be detected; the feature pyramid is used to construct a multi-scale feature pyramid through a top-down path and lateral connections to obtain an anchor box using the features obtained by the backbone network; the prediction subnetwork is used to predict the category, position and size of the anchor box and obtain an initial foreign body detection box.

[0010] As an improvement of an embodiment of the present invention, the RetinaNet model also includes: adding an attention mechanism module to the backbone network of the RetinaNet model and using an improved GIoU Loss as a loss function; the attention mechanism module includes a channel attention module and a spatial attention module, and the attention mechanism module is used to refine the features extracted by the backbone network to obtain refined features.

[0011] As an improvement of an embodiment of the present invention, the prediction subnetwork includes a classification subnetwork and a regression subnetwork, the classification subnetwork is used to predict the category of the anchor box, and the regression subnetwork is used to calculate the position and size of the anchor box and obtain an initial foreign body detection box.

[0012] As an improvement of an embodiment of the present invention, the foreign object detection frame information includes the positions, sizes and confidences of the several initial foreign object detection frames in one-to-one correspondence; the final prediction result is several initial foreign object detection frames whose confidences are greater than a preset threshold.

[0013] As an improvement to the embodiment of the present invention, the preset threshold is 0.8.

[0014] To achieve one of the above-mentioned purposes, an embodiment of the present invention provides a foreign body detection device for a photovoltaic panel, comprising the following modules:

[0015] An information acquisition module, used to acquire an initial image of the photovoltaic panel, wherein the initial image is an RGB image;

[0016] A processing module, used for preprocessing the initial image to obtain an image to be detected, wherein the image to be detected is an HSV image;

[0017] The foreign body detection module is used to create a RetinaNet model, detect the image to be detected based on the RetinaNet model to obtain a number of initial foreign body detection frames and foreign body detection frame information corresponding to the initial foreign body frames; and screen the initial foreign body detection frames to obtain a final foreign body detection result.

[0018] To achieve one of the above objects of the invention, an embodiment of the present invention provides a storage medium storing program instructions, wherein the program instructions, when executed by a processor, implement the foreign matter detection method as described in any one of the above items.

[0019] To achieve one of the above objects of the invention, an embodiment of the present invention provides a terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the foreign body detection method as described in any one of the above items.

[0020] The foreign object detection method, device, storage medium and terminal for photovoltaic panels provided by the embodiments of the present invention have the following advantages: the present invention uses the RetinaNet model to detect foreign objects in images of photovoltaic panels, adds an attention mechanism module to its backbone network to refine the features, and uses the improved GIoU Loss as the loss function to solve the problem of class imbalance in target detection, so that the model can achieve high-accuracy detection even under complex backgrounds or different lighting conditions, ensuring that the operating efficiency and power generation performance of the photovoltaic system will not be reduced due to pollutants that are not removed in time.

[0021] In addition, the RetinaNet model can process the entire image in a single forward propagation, avoiding the need to scan the image multiple times in traditional detection systems. Therefore, the present invention can achieve fast detection while ensuring high detection accuracy, which not only improves cleaning efficiency and accuracy, but also reduces operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the process of the foreign matter detection method for photovoltaic panels according to the present invention;

[0023] Figure 2 It is a schematic diagram of the structure of the RetinaNet model of the present invention;

[0024] Figure 3 It is a schematic diagram of the structure of the deep residual network ResNet-50 described in the present invention;

[0025] Figure 4 It is a schematic diagram of the characteristic pyramid structure of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0027] If the present invention involves directions (for example, up, down, left, right, front, back, outside, inside, etc.) when describing, it is necessary to define the directions involved.

[0028] The scope of the embodiments herein includes the entire scope of the claims, and all available equivalents of the claims.Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements.In fact, the first element can also be called the second element, and vice versa.Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements not only include those elements, but also include other elements not clearly listed, or also include elements inherent to such structure, device or equipment.In the absence of more restrictions, the elements limited by the statement "include one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements.Each embodiment herein is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same similar parts between each embodiment can be referred to each other.

[0029] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0030] Embodiment 1 of the present invention provides a foreign body detection method for photovoltaic panels, such as Figure 1 As shown, the foreign body detection method comprises the following steps:

[0031] Step 101: Acquire an initial image of the photovoltaic panel, wherein the initial image is an RGB image;

[0032] In practice, according to the characteristics, size and detection requirements of the environment in which the photovoltaic panels are located, high-resolution and wide-angle cameras can be selected and installed in appropriate locations to ensure that the entire surface or key areas of the photovoltaic panels can be captured clearly and unobstructed. For large-scale photovoltaic panel arrays, high-definition cameras carried by drones can be used for inspection. For fixed cameras, it is necessary to ensure that they are stable and not easily disturbed by external factors. After that, the resolution, white balance and other parameters of the camera are adjusted according to the requirements of detection accuracy to ensure accurate image color and reduce the impact of extreme lighting conditions on image quality. In addition, the camera's shooting frame rate can be set according to the periodicity of the detection task and the shooting can be scheduled. At the same time, tag information such as acquisition time, location, photovoltaic panel number, etc. can be added to each initial image to facilitate subsequent data processing and analysis, realize automatic acquisition, reduce manual intervention, and regularly maintain and repair the image acquisition equipment to ensure its continuous and stable operation, and pay close attention to the status of network connections and storage devices to ensure the reliability of data transmission and storage.

[0033] Step 102: preprocessing the initial image to obtain an image to be detected, wherein the image to be detected is an HSV image;

[0034] Here, the HSV (Hue, Saturation, Value) color model is a color space established based on the intuitive characteristics of color. The color parameters in the HSV color model are: hue, saturation and brightness. In practice, the color type of the initial image can be converted to the HSV color model by the following method: Get the R, G, B values ​​of the initial image, R ′ =R / 255, G ′ =G / 255,B ′ =B / 255,C max =max(R ′ ,G ′ ,B ′ ), C min =min(R ′ ,G ′ ,B ′ ), ΔC=C max -C min ; V=C ma Compared with RGB color space, HSV color space is more intuitive and convenient in describing color information. In HSV space, the three components of hue (H), saturation (S) and brightness (V) are separated, which helps to better separate the color information and brightness information in the image.

[0035] Step 103: Create a RetinaNet model, detect the image to be detected based on the RetinaNet model to obtain a number of initial foreign object detection frames and foreign object detection frame information corresponding to the initial foreign object frames; filter the initial foreign object detection frames to obtain a final foreign object detection result.

[0036] Here, RetinaNet is a single-stage object detection model that introduces the Focal Loss loss function to solve the problem that the traditional single-stage detector is not as accurate as the two-stage detector. Focal Loss dynamically adjusts the weights of easy-to-classify samples and difficult-to-classify samples, so that the model pays more attention to difficult-to-distinguish samples during training, thereby significantly improving the detection performance. RetinaNet combines the feature pyramid network as a feature extractor and the anchor box mechanism to predict objects of different scales, enabling it to effectively detect foreign objects of various sizes and shapes in images.

[0037] In practice, it is necessary to collect annotated images containing photovoltaic panels and various foreign objects that may exist on them as training sets. Subsequently, the RetinaNet model is trained using these annotated data to continuously optimize the Focal Loss and positioning loss. After the training is completed, the photovoltaic panel image to be detected is input into the trained RetinaNet model. The model extracts multi-scale features of the image, and predicts the possible locations of foreign objects in the image through the anchor frame mechanism, generating several initial foreign object detection frames and their corresponding confidence scores and category information. Preferably, Figure 2 As shown, the present invention adds an attention mechanism module to the backbone network of the RetinaNet model and uses the improved GIoU Loss as the loss function, and classifies and quantifies the final foreign body detection results. By counting the number and coverage area of ​​different targets, the degree of influence of foreign objects on the performance of photovoltaic panels is evaluated, the urgency and necessity of cleaning are determined, and finally a maintenance report is generated according to the analysis results. Among them, the improved GIo U Loss solves the problem that the traditional IoU Loss cannot provide effective gradient information in some cases by introducing the minimum enclosing box. The calculation steps of GIoU Loss are as follows: first, calculate the intersection and union IoU of two bounding boxes A and B; find the minimum bounding box C that can contain A and B at the same time; use the area of ​​C to subtract the union area of ​​A and B to calculate the area difference between the minimum enclosing box C and the union U, and then use the area difference to adjust the IoU value to obtain GIoU, GIoU Loss = 1-GIoU. It can be understood that the improved GIoU Loss can more accurately measure the regression quality of the bounding box, thereby improving the accuracy of target detection.

[0038] In this embodiment, the preprocessing specifically includes: using the Canny edge detection algorithm to obtain edge information of the photovoltaic panel in the initial image, performing contour detection on the position of the photovoltaic panel based on the edge information to obtain the photovoltaic panel area image in the initial image, scaling and image enhancement processing on the photovoltaic panel area image, and denoising and color space conversion on the enhanced photovoltaic panel area image to obtain the image to be detected.

[0039] In practice, the Canny algorithm is first used to extract the edge information of the photovoltaic panel. The Canny algorithm uses a Gaussian filter to smooth the image, calculates the gradient strength and direction of the smoothed image, and identifies potential edge points. By comparing the gradient strength of each pixel with the adjacent pixels in its gradient direction, the point with the largest local gradient strength is retained as the edge point. Set a high threshold and a low threshold, the high threshold is used to determine the strong edge, and the low threshold is used to determine the weak edge. Then, by connecting the strong edge and the edge extended from the weak edge, a complete edge contour is formed. Based on the edge information obtained by Canny edge detection, the present invention uses the findContours function in the contour detection algorithm OpenCV library to identify the contour of the photovoltaic panel, and cuts out the photovoltaic panel area image from the initial image according to the identified contour of the photovoltaic panel. The extracted photovoltaic panel area image is scaled to 640×480 pixels to meet the input requirements of the RetinaNet model. At the same time, image enhancement techniques such as histogram equalization and contrast stretching are used to improve image quality and improve the visibility of surface details of the photovoltaic panel. Noise may be introduced during the image enhancement process, so it is necessary to use median filtering, Gaussian filtering, etc. to denoise the enhanced image. Finally, the denoised photovoltaic panel area image is converted from the RGB color space to the HSV color space. The HSV color space separates the three components of hue, saturation, and brightness, which is convenient for subsequent color segmentation and foreign body detection.

[0040] In this embodiment, the RetinaNet model specifically includes: a backbone network, a feature pyramid and a prediction subnetwork; the backbone network is a deep residual network ResNet-50, which is used to extract the features of the image to be detected; the feature pyramid is used to construct a multi-scale feature pyramid through a top-down path and lateral connections to obtain an anchor box using the features obtained by the backbone network; the prediction subnetwork is used to predict the category, position and size of the anchor box and obtain an initial foreign body detection box.

[0041] Here, if Figure 3As shown in the figure, the deep residual network ResNet-50 has 50 layers, including 16 residual blocks. Each residual block has several layers of neurons, which are connected by identity connections, making it easier for the gradient to cross the barriers between layers during back propagation, thereby effectively alleviating the problem of gradient vanishing or exploding during deep network training. In addition, ResNet-50 introduces the concept of residual learning. The residual block allows the network to learn the difference between input and output instead of directly learning the output, so that the network can maintain good performance while deepening, solving the problem of difficult training of traditional deep networks. Each residual block contains two convolutional layers and an identity connection. The identity connection passes the input directly to the output, avoiding the problem of gradient vanishing. ResNet-50 uses a global average pooling layer at the very end of the model, taking the average value of all pixels of each feature map as the output of the feature map, which helps to reduce the number of parameters of the model and reduce the risk of overfitting.

[0042] In this embodiment, the prediction subnetwork includes a classification subnetwork and a regression subnetwork. The classification subnetwork is used to predict the category of the anchor box, and the regression subnetwork is used to calculate the position and size of the anchor box and obtain an initial foreign object detection box.

[0043] In this embodiment, the foreign object detection frame information includes the corresponding positions, sizes and confidences of the several initial foreign object detection frames; the final prediction result is several initial foreign object detection frames whose confidences are greater than a preset threshold. Preferably, the preset threshold is 0.8. It can be understood that by setting a higher preset threshold to filter the initial foreign object detection frames and retaining only these high-confidence detection frames in the final prediction result, the accuracy of foreign object detection can be significantly improved.

[0044] Embodiment 2 of the present invention provides a foreign body detection device for photovoltaic panels, comprising the following modules:

[0045] An information acquisition module, used to acquire an initial image of the photovoltaic panel, wherein the initial image is an RGB image;

[0046] A processing module, used for preprocessing the initial image to obtain an image to be detected, wherein the image to be detected is an HSV image;

[0047] The foreign body detection module is used to create a RetinaNet model, detect the image to be detected based on the RetinaNet model to obtain a number of initial foreign body detection frames and foreign body detection frame information corresponding to the initial foreign body frames; and screen the initial foreign body detection frames to obtain a final foreign body detection result.

[0048] Embodiment 3 of the present invention provides a storage medium storing program instructions, which, when executed by a processor, implement the foreign matter detection method as described in Embodiment 1 and Embodiment 2 of the present invention.

[0049] Embodiment 4 of the present invention provides a terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the foreign body detection method as described in Embodiment 1 and Embodiment 2 of the present invention.

[0050] Storage medium can be a tangible device that keeps and stores the instructions used by the instruction execution device. Storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0051] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0052] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A foreign body detection method for photovoltaic panels, characterized in that: The following steps are involved: Acquire an initial image of the photovoltaic panel, wherein the initial image is an RGB image; Preprocessing the initial image to obtain an image to be detected, wherein the image to be detected is an HSV image; A RetinaNet model is created, and based on the RetinaNet model, the image to be detected is detected to obtain a plurality of initial foreign object detection frames and foreign object detection frame information corresponding to the plurality of initial foreign object frames; and the initial foreign object detection frames are screened to obtain a final foreign object detection result.

2. The foreign body detection method according to claim 1, characterized in that: The preprocessing specifically includes: using the Canny edge detection algorithm to obtain edge information of the photovoltaic panel in the initial image, performing contour detection on the position of the photovoltaic panel based on the edge information to obtain the photovoltaic panel area image in the initial image, scaling and image enhancement processing on the photovoltaic panel area image, and denoising and converting the color space of the enhanced photovoltaic panel area image to obtain the image to be detected.

3. The foreign body detection method according to claim 1, characterized in that: The RetinaNet model specifically includes: a backbone network, a feature pyramid and a prediction subnetwork; the backbone network is a deep residual network ResNet-50, which is used to extract the features of the image to be detected; the feature pyramid is used to construct a multi-scale feature pyramid through a top-down path and lateral connections to obtain an anchor box using the features obtained by the backbone network; the prediction subnetwork is used to predict the category, position and size of the anchor box and obtain an initial foreign object detection box.

4. The foreign matter detection method according to claim 3, characterized in that: The RetinaNet model also includes: adding an attention mechanism module to the backbone network of the RetinaNet model and using an improved GIoU Loss as a loss function; the attention mechanism module includes a channel attention module and a spatial attention module, and the attention mechanism module is used to refine the features extracted by the backbone network to obtain refined features.

5. The foreign body detection method according to claim 3, characterized in that: The prediction subnetwork includes a classification subnetwork and a regression subnetwork. The classification subnetwork is used to predict the category of the anchor box, and the regression subnetwork is used to calculate the position and size of the anchor box and obtain an initial foreign body detection box.

6. The foreign body detection method according to claim 1, characterized in that: The foreign object detection frame information includes the one-to-one corresponding positions, sizes and confidences of the several initial foreign object detection frames; the final prediction result is several initial foreign object detection frames whose confidences are greater than a preset threshold.

7. The foreign matter detection method according to claim 6, characterized in that: The preset threshold is 0.

8.

8. A foreign body detection device for photovoltaic panels, characterized in that: Includes the following modules: An information acquisition module, used to acquire an initial image of the photovoltaic panel, wherein the initial image is an RGB image; A processing module, used for preprocessing the initial image to obtain an image to be detected, wherein the image to be detected is an HSV image; The foreign body detection module is used to create a RetinaNet model, detect the image to be detected based on the RetinaNet model to obtain a number of initial foreign body detection frames and foreign body detection frame information corresponding to the initial foreign body frames; and screen the initial foreign body detection frames to obtain a final foreign body detection result.

9. A storage medium storing program instructions, characterized in that: When the program instructions are executed by a processor, the foreign matter detection method according to any one of claims 1 to 7 is implemented.

10. A terminal, characterized in that: The method comprises a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the foreign matter detection method according to any one of claims 1 to 7.

Images