A photovoltaic module fault detection method
Through the RAM-YOLOv8 network model and cross-board walking robot, combined with deep learning technology, photovoltaic module failures are identified, and the problems of inefficiency and high cost of traditional detection methods are solved, efficient and accurate fault detection is achieved, and the operation of photovoltaic power stations is improved. Peacekeeping power generation efficiency.
Patent Information
- Application Number
- CN202410313450.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-19
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-03-19
AI Technical Summary
Traditional photovoltaic module fault detection methods are inefficient and costly, and are particularly difficult to detect internal defects, which affects the power generation efficiency and operation and maintenance efficiency of photovoltaic power stations.
Using the RAM-YOLOv8 network model combined with cross-board walking robot and video imager, the fault types and locations in the surface images of photovoltaic modules are identified through binary processing and deep learning technology, and the feature extraction is improved using residual aggregation module and Swin Transformer network. capability, improve detection accuracy and speed.
It improves the efficiency and accuracy of photovoltaic module fault detection, reduces the cost of manual inspection, can effectively process large-scale data and multi-objective detection, and improves the operation and maintenance efficiency of photovoltaic power stations.
Smart Images

Figure CN118100797B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent operation and maintenance of photovoltaic power stations, and more specifically to a photovoltaic component fault detection method. Background Art
[0002] In recent years, due to my country's support for the construction of photovoltaic power stations, the installed capacity of photovoltaic power generation has developed extremely rapidly. With the increase in utilization rate, the demand for the number of equipment has also increased. In the era of rapid advancement of photovoltaic power generation technology, the life of equipment has become a factor restricting its development. Factors such as the material of the material, the angle between the equipment installation location and the ground may affect the conversion efficiency of the equipment. Considering that photovoltaic modules are mostly installed outdoors, the solar panels are prone to various failures due to the operating environment.
[0003] Faced with such a huge market scale, the operation and maintenance of photovoltaic power stations and distributed power generation has become the focus of research. How to effectively reduce the cost of operation and maintenance and improve the efficiency of operation and maintenance will directly affect the economic benefits of photovoltaic power stations and whether they can operate stably. As a key component of photovoltaic power stations, the photoelectric conversion efficiency of photovoltaic modules will directly affect the power generation efficiency of the power station. However, there have always been many problems in the inspection process of photovoltaic modules. A large amount of manpower is required in traditional operation and maintenance, which will lead to higher maintenance costs in long-term operation. At the same time, some defects of photovoltaic modules, such as microcracks or internal circuit problems, may not be easy to detect through appearance inspection, which indirectly leads to lower efficiency in large-scale photovoltaic power stations, especially when thousands or tens of thousands of photovoltaic panels need to be inspected.
[0004] In the field of detection methods, there are two main types of mature photovoltaic module fault detection methods on the market: the first is based on traditional image processing technology, tending to use machine learning algorithms to extract the edge information of photovoltaic module hot spots, and then use classifiers for effective classification, detection and identification; the second is based on deep learning. Among them, with the popularization of artificial intelligence and its wide application in many fields, artificial intelligence detection algorithms based on deep learning have attracted the attention of scholars in various fields.
[0005] Therefore, how to introduce deep learning technology into traditional photovoltaic power stations to solve the current problems of low detection efficiency, high inspection costs, and difficulty in detecting internal defects in traditional operation and maintenance is an issue that technical personnel in this field urgently need to solve. Summary of the invention
[0006] In view of the above problems, the present invention provides a photovoltaic module fault detection method to at least solve some of the technical problems mentioned in the above background technology.
[0007] In order to achieve the above object, the present invention adopts the following technical solution:
[0008] The present invention provides a photovoltaic module fault detection method, comprising the following steps:
[0009] S1, obtaining a surface image of a photovoltaic module;
[0010] S2, performing binarization processing on the surface image of the photovoltaic module;
[0011] S3. Use the trained RAM-YOLOv8 network model to identify and detect the binary processed surface image of the photovoltaic module to obtain the fault type and fault location.
[0012] Furthermore, the step S1 specifically includes:
[0013] A cross-panel walking robot equipped with a video imager and an infrared detection sensor is used to inspect the photovoltaic module panels and collect images of the photovoltaic module surface.
[0014] Furthermore, the RAM-YOLOv8 network model replaces the C2f modules of the 2nd, 6th and 8th layers in the backbone network with residual aggregation modules based on the YOLOv8 network; and uses the MST module for upsampling in the neck-end network.
[0015] Furthermore, the RAM-YOLOv8 network model, based on the neck-end network, performs convolution on the 4th layer of the backbone network, and splices the convolution result with the MST module; performs convolution on the 6th layer of the backbone network, and splices the convolution result with the output of the pyramid pooling layer.
[0016] Furthermore, in the residual aggregation module, after each passing through the bottleneck network layer, concat splicing and convolution channel adjustment are performed.
[0017] Furthermore, the MST module includes a first channel and a second channel; the first channel is only convolved; the second channel passes through a Swin Transformer network after convolution; and the results of the two channels are concatenated and then the convolution channel is adjusted.
[0018] Furthermore, the Swin Transformer network includes: a window-based multi-head self-attention module and a moving window multi-head self-attention module; the window-based multi-head self-attention module and the moving window multi-head self-attention module are both composed of a multi-head self-attention module.
[0019] Furthermore, the training process of the RAM-YOLOv8 network model includes:
[0020] P1. Obtain a data set containing photovoltaic module images and corresponding annotation information; the annotation information includes a fault type label and a fault location label;
[0021] P2. Preprocess the images in the data set;
[0022] P3. Train the built RAM-YOLOv8 network model using the preprocessed dataset; the training process includes forward propagation of the input image and back propagation to optimize the model parameters.
[0023] Furthermore, the preprocessing includes reducing noise, enhancing contrast, adjusting brightness and color balance.
[0024] Furthermore, during the training process, a positive and negative sample allocation strategy and a loss function group are used in loss calculation; the loss function group includes classification loss and regression loss; the classification loss uses Varifocal Loss; the regression loss uses CIOU Loss and DFL Loss.
[0025] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a photovoltaic module fault detection method, which has the following beneficial effects:
[0026] The present invention calls the RAM-YOLOv8 model to improve the efficiency and accuracy of photovoltaic module fault detection, and solves the problems of insufficient processing of large-scale data and multi-target detection in photovoltaic module defect detection.
[0027] The present invention adopts a residual aggregation module to replace the C2F module in the original traditional YOLOv8 network to enhance the feature extraction capability; introduces the network structure idea of Swin Transformer into the feature fusion network, improves the shortcomings of the original model's poor extraction capability and weak receptive field, and improves the detection speed and accuracy.
[0028] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the present invention.
[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0031] Figure 1 A schematic diagram of a photovoltaic module fault detection method provided by an embodiment of the present invention.
[0032] Figure 2 A schematic diagram of the framework of a photovoltaic module fault detection method provided in an embodiment of the present invention.
[0033] Figure 3 A schematic diagram of the YOLOv8 network structure provided for related prior art.
[0034] Figure 4 A schematic diagram of the RAM-YOLOv8 network model structure provided in an embodiment of the present invention.
[0035] Figure 5 A schematic diagram of the RAM module structure provided by an embodiment of the present invention.
[0036] Figure 6 A schematic diagram of the MST structure provided in an embodiment of the present invention.
[0037] Figure 7 A schematic diagram of a photovoltaic module fault detection method according to an embodiment of the present invention.
[0038] Figure 8 A schematic diagram of the real-time fault detection effect of the cloud platform provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] See also Figure 1 and Figure 2 As shown, an embodiment of the present invention discloses a photovoltaic module fault detection method, comprising the following steps:
[0041] S1, obtaining a surface image of a photovoltaic module;
[0042] S2, performing binarization processing on the surface image of the photovoltaic module;
[0043] S3. Use the trained RAM-YOLOv8 network model to identify and detect the binary processed surface image of the photovoltaic module to obtain the fault type and fault location.
[0044] Next, each of the above steps will be described respectively.
[0045] In the above step S1, a cross-panel walking robot equipped with a video imager and an infrared detection sensor is used to inspect the photovoltaic module panel and collect the surface image of the photovoltaic module;
[0046] The cross-board walking robot comprises a chassis, a shell, a walking component, an adsorption component and a scanning component; wherein the shell is located on the top of the chassis and connected to the chassis; the walking component is connected to the chassis; the adsorption component comprises a turbofan and a supporting structure, the supporting structure surrounds and forms a negative pressure cavity, and the turbofan is arranged in the negative pressure cavity; and the supporting structure is connected to the chassis; the scanning component is arranged on the top of the shell; the video imager is located inside the shell; and the infrared detection sensor is arranged at the bottom of the chassis;
[0047] The present invention uses a cross-panel walking robot equipped with a video imager and an infrared detection sensor to conduct inspections on photovoltaic component panels, thereby reducing labor costs to achieve efficient operation and maintenance, save resources, and reduce power station maintenance costs; the fault detection method is no longer affected by factors such as low efficiency and high labor costs.
[0048] In the above step S2, the acquired surface image of the photovoltaic module is binarized, mainly by dividing the image into two parts, light (255) and dark (0), according to certain rules; that is, the binarized image has only two pixel values: 0 and 255; it is a black-and-white image; commonly used methods include: (1) fixed threshold binarization, that is, given a range, the pixels with pixel values in the given range are set to 255, and the rest are set to 0; (2) dynamic threshold binarization, that is, first smoothing the image, then comparing the original image with the smoothed image pixel by pixel, and setting the pixel values brighter than the original image to 255; the pixel values darker than the original image can also be set to 255; (3) binarization based on grayscale histogram, that is, automatically selecting a suitable threshold for segmentation according to the double peaks of the grayscale histogram;
[0049] After the surface image of the photovoltaic module is binarized, morphological operations are performed. Common morphological operations are mainly image expansion, erosion, opening operation, and closing operation. Among them, the opening operation is to erode first and then expand; while the closing operation is the opposite, which is to expand first and then erode.
[0050] In the above step S3, the RAM-YOLOv8 network model is obtained by improving the traditional YOLOv8 network in the present invention; the schematic diagram of the traditional YOLOv8 network can be found in Figure 3As shown in the figure, it is mainly divided into three parts: backbone network (Backbone), neck network (Neck) and detection head (Head); after the image is input to the Backbone through the input end (Input), it passes through two convolutional layers, mainly to expand the image channel, reduce the feature map, and extract deep features; then the image enters the c2f module, after the spilt operation (i.e., chunk operation), the number of channels is divided, and it is concatenated (concat) through n bottleneck networks, and the convolution channels are adjusted; the c2f module mainly enhances the ability of feature expression through dense residual structure, changes the number of channels through splitting and concatenation operations according to the scaling factor, and reduces the computational complexity and model capacity; the bottleneck is repeated 6, 6, and 3 times in the c2f module in the 4th, 6th, and 8th layers (the starting layer number is 0) respectively; finally, the fast spatial pyramid pooling layer (SPPF) is used at the tail to increase the receptive field and capture the feature information of different levels in the scene; the path aggregation network (Path Aggregation Network) is used in the Neck network The feature maps of the three stages in Backbone are fused to help the shallow information to aggregate to the deep features. Finally, the feature maps are output to the detection head for recognition.
[0051] The improved RAM-YOLOv8 network model of the present invention can be found in Figure 4 As shown in the figure, the C2f modules of the 2nd, 6th and 8th layers in the backbone network of the traditional YOLOv8 network are replaced with residual aggregation modules (RAM modules); the swing Transformers fusion module (MST) is used in the neck network for upsampling; the head part is replaced with a decoupled head structure to separate the classification and detection heads, so that classification and regression are divided into two branches to reduce the conflict between tasks; and the head part adopts Anchor-Free, which plays a huge role in detecting irregular targets, so as to achieve more accurate and robust target detection;
[0052] In addition, unlike the traditional YOLOv8 network, the present invention is based on the neck network, performs convolution on the 4th layer of the backbone network, and splices the convolution result with the MST module; performs convolution on the 6th layer of the backbone network, and splices the convolution result with the output of the pyramid pooling layer; since the Backbone part is mainly for feature extraction and the Neck part is mainly for feature fusion, the final result required by the model is the result recognized by the detection head, and the position where the detection head is connected is biased to the back, that is, the tail of the Neck structure; and because the Backbone part obtains the most original feature information of the image and the Neck part obtains the processed feature information, therefore, the present invention can obtain both the processed feature information and the original feature information by changing the feature fusion route, that is, the deep semantics and the shallow semantics are spliced, and the fused feature information is richer, making the model more sensitive to fault recognition of small targets of multiple sizes.
[0053] The residual aggregation module structure diagram can be found in Figure 5 As shown; unlike the traditional C2f module, after the image is input into the RAM module, it needs to be concat spliced and convolution channel adjusted each time after passing through the bottleneck network layer; in this process, the original feature map of the image can be passed to the next layer for processing after each concat splicing (the traditional C2f module does not pass the original image to the next layer); since the number of channels will change after each concat splicing, in order to keep the number of channels unchanged, a convolution is performed again after each splicing to adjust the number of channels; compared with the traditional C2f module, the RAM module of the present invention integrates the idea of residual network and processes the input information through two parallel branches; each branch contains multiple layers of convolution and residual connection, so that the feature extraction is richer and deeper; the feature information is merged after the number of channels is adjusted by the convolution layer, which effectively combines the features of the deep and shallow layers in the network, and reduces the gradient disappearance or explosion problems that may occur during the information transmission process; it has better effects on the model recognition of small targets and processing of complex backgrounds.
[0054] The schematic diagram of the MST module structure can be found in Figure 6As shown, the MST module includes a first channel and a second channel; the first channel is only convolved; the second channel passes through the Swin Transformer network after convolution; the results of the two channels are spliced and then the convolution channel is adjusted, and finally output; in the MST module, the lower branch can extract the overall features of the feature map by introducing the Swin Transformer, and the upper branch convolution extracts local features. The combination of global information and local information can view the feature map from different angles, that is, more abundant feature information is obtained; that is, the MST structure after integrating the Swin Transformer enriches the receptive field of the model, can extract the global feature information and local feature information of the feature map, and combine the two to observe the feature information from different angles;
[0055] Among them, the Swin Transformer network first divides the input image into a series of non-overlapping small blocks, which are usually square, and each small block is regarded as a separate input unit. A hierarchical structure is introduced, in which each layer is divided into two parts: cross-level connection and cross-patch connection; this structure allows information to be freely transferred between different levels and small blocks; the model structure usually contains multiple levels of Transformer layers, each layer consists of multiple Transformer blocks, which are composed of self-attention mechanisms and feedforward neural networks; these blocks are responsible for feature extraction and transformation of the input. This makes the model deep and can process large-scale images, so that cross-level connections allow information exchange between different levels; each layer has a set of Transformer blocks that can transfer information between layers. This helps the model capture features of different levels of abstraction from different levels, so as to achieve better feature extraction and information transfer;
[0056] In the embodiment of the present invention, the Swin Transformer network mainly includes two submodules: a windowed multi-head self-attention module (W-MSA) and a moving window multi-head self-attention module (SW-MSA); both submodules are composed of a multi-head self-attention (MSA) module. The processing flow is that the feature map is processed by the LayerNorm (LN) layer, and then passed to the W-MSA for residual connection; the connected feature map passes through the LayerNorm layer again, and then passed to the multi-layer perceptron (MLP), and another residual connection is performed; SW-MSA follows the same processing flow and finally outputs the processing result. The introduction of the W-MSA module is intended to reduce the computational overhead and achieve efficient model modeling by implementing the self-attention mechanism in the local window. In this design, the image is evenly divided by non-overlapping windows, each window includes M×M pixel blocks, and compared with calculating the global multi-head self-attention (MSA) over the entire image range, this window-based self-attention method significantly reduces the computational complexity. Specifically, for an image of h×w pixel blocks, the computational complexity of the global MSA module and the window-based MSA module can be described by the following formulas:
[0057] Ω(MSA)=4hwC 2 +2(hw) 2 C
[0058] Ω(W-MSA)=4hwC 2 +2M 2 HkDJ
[0059] The computational complexity of global multi-head self-attention (MSA) increases quadratically with the number of patches h×w, making it difficult to process high-resolution images. In contrast, the complexity of window-based MSA is linear, especially when the window size M is fixed (usually set to 7). Since window-based self-attention can be flexibly adjusted, it greatly reduces the amount of computation required compared to global MSA, making it more suitable for processing high-resolution images, thereby improving the performance of the model while maintaining efficiency. Since W-MSA can only focus on the content of the window itself and does not allow cross-window connections, information cannot be transferred between windows. SW-MSA introduces cross-window connections by shifting windows while maintaining efficient computation of non-overlapping windows.
[0060] The training process of the RAM-YOLOv8 network model includes:
[0061] P1. Obtain a data set containing PV module images and corresponding annotation information; the annotation information includes a fault type label and a fault location label;
[0062] P2. Preprocess the images in the dataset, including reducing noise, enhancing contrast, adjusting brightness and color balance, etc., to improve the robustness and performance of the model;
[0063] P3. Train the built RAM-YOLOv8 network model using the preprocessed data set. The training process includes forward propagation of the input image and back propagation to optimize the model parameters.
[0064] P4. Model tuning: During the training process, the model is tuned according to the actual situation, such as adjusting the learning rate, data enhancement strategy, regularization, etc., to improve the detection accuracy and robustness; specifically, the image is input into the pre-trained neural network to obtain the corresponding feature map; a predetermined region of interest is set for each point in the feature map to obtain multiple candidate regions of interest; these candidate regions of interest are sent to the feature fusion network, and SwinTransformer is introduced into the feature network to enrich the model receptive field; then the candidate regions of interest are clustered; finally, these regions of interest are classified and identified
[0065] Use the trained RAM-YOLOv8 model to perform defect detection on new PV module images. That is, input the binary PV module surface image into the RAM-YOLOv8 model, and obtain the target box and category prediction results through forward propagation. The detection results can be visualized, and the bounding box can be drawn on the original image or the defect area can be marked for further analysis and interpretation.
[0066] Also included: P5, post-processing the detection results; this can include non-maximum suppression (NMS) to eliminate overlapping bounding boxes, setting thresholds to filter out low-confidence detections, etc.
[0067] In the embodiment of the present invention, during the training process, a positive sample matching strategy (TaskAligned Assigner) is adopted in the loss calculation, and a distribution focal loss (DistributionFocal Loss) is introduced, which mainly includes two parts: a positive and negative sample allocation strategy and a loss calculation; the loss calculation part mainly includes a classification loss Varifocal Loss and a regression loss CIOU Loss+DFL Loss. Varifocal Loss is a loss function for target detection tasks, which not only focuses on samples that are difficult to classify, but also considers the accuracy of the predicted probability to improve the performance of the target detection model; CIOU Loss is a loss function for target detection tasks, which aims to help train neural networks to better locate and classify target objects. It is an improved IoU loss, which aims to solve the problems of center point deviation and aspect ratio difference in traditional IoU loss; DFL Loss aims to improve the performance of deep learning models with a large number of negative samples and a small number of positive samples, which helps the model to better focus on and learn positive samples, and reduces the performance degradation caused by the dominance of negative samples.
[0068] After the robot collects and pre-processes the image, it transmits it to the cloud platform, and the PV module detection cloud platform calls the RAM-YOLOv8 algorithm to determine whether a fault is found in the PV module image. If no fault is found, it jumps to "Collecting the original image of the PV module" and continues to execute; if a fault is found, it calculates the weight of all the target pixels and classifies and identifies the fault; then the location of the faulty PV module is promptly reported to the PV power station staff for maintenance. For details, see Figure 7 As shown; the fault detection diagram can be found in Figure 8 shown.
[0069] In another embodiment, the photovoltaic component fault detection method provided by the present invention can be integrated with a traditional photovoltaic panel power generation efficiency detection method; specifically, the power generation efficiency of the photovoltaic panel is first tracked in real time using traditional technology. When the power generation efficiency fluctuates, it indicates that the photovoltaic panel may have defects or dust accumulation. At this time, the photovoltaic component fault detection method provided by the present invention is used to perform real-time detection of the photovoltaic panel, thereby greatly improving the inspection efficiency and inspection accuracy, and completing many complex tasks, such as cracks, stains, snail patterns, vegetation obstruction, damage, welding strip failure and other problems.
[0070] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0071] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A photovoltaic module fault detection method, It is characterized in that The steps include: S1, obtaining a surface image of a photovoltaic module; S2, performing binarization processing on the surface image of the photovoltaic module; S3, using the trained RAM-YOLOv8 network model to identify and detect the binary processed surface image of the photovoltaic module to obtain the fault type and fault location; The RAM-YOLOv8 network model replaces the C2f modules of the 2nd, 6th and 8th layers in the backbone network with residual aggregation modules based on the YOLOv8 network; and uses the MST module for upsampling in the neck network; The RAM-YOLOv8 network model, based on the neck network, performs convolution on the 4th layer of the backbone network, and splices the convolution result with the MST module; performs convolution on the 6th layer of the backbone network, and splices the convolution result with the output of the pyramid pooling layer; In the residual aggregation module, after each passing through the bottleneck network layer, concat splicing and convolution channel adjustment are performed; The MST module includes a first channel and a second channel; the first channel is only convolved; the second channel passes through a Swin Transformer network after convolution; the results of the two channels are spliced and then the convolution channel is adjusted; The Swin Transformer network includes: a windowed multi-head self-attention module and a moving window multi-head self-attention module; The window-based multi-head self-attention module and the moving window multi-head self-attention module are both composed of a multi-head self-attention module.
2. A photovoltaic module fault detection method according to claim 1, It is characterized in that The step S1 specifically includes: A cross-panel walking robot equipped with a video imager and an infrared detection sensor is used to inspect the photovoltaic module panels and collect images of the photovoltaic module surface.
3. A photovoltaic module fault detection method according to claim 1, It is characterized in that The training process of the RAM-YOLOv8 network model includes: P1. Obtain a data set containing photovoltaic module images and corresponding annotation information; the annotation information includes a fault type label and a fault location label; P2. Preprocess the images in the data set; P3. Train the built RAM-YOLOv8 network model using the preprocessed dataset; the training process includes forward propagation of the input image and back propagation to optimize the model parameters.
4. A photovoltaic module fault detection method according to claim 3, It is characterized in that The pre-processing includes reducing noise, enhancing contrast, adjusting brightness and color balance.
5. A photovoltaic module fault detection method according to claim 3, It is characterized in that During the training process, a positive and negative sample allocation strategy and a loss function group are used for loss calculation; the loss function group includes classification loss and regression loss; the classification loss uses Varifocal Loss; the regression loss uses CIOU Loss and DFL Loss.
