Photovoltaic module fault detection method and device, computer device and storage medium

Abstract

The embodiment of the specification discloses a photovoltaic module fault detection method, device, computer equipment and storage medium, through target detection on a photovoltaic module image, a preliminary fault detection result is obtained, and in the case that a preliminary fault category included in the preliminary fault detection result belongs to a first easy misjudgment category, image classification is performed on a fault area image included in the photovoltaic module image, a fault classification result corresponding to the fault area image is obtained, and the preliminary fault detection result and the fault classification result are further compared, so that the target fault category is determined based on the comparison result, in this way, through the cooperation of target detection and classification detection, when a fault category misjudged due to the similar image features of the fault area image is identified, the accuracy of fault category identification is improved.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of photovoltaic power generation technology, and in particular to a method, apparatus, computer equipment, and storage medium for detecting photovoltaic module faults. Background Technology

[0002] With the development of photovoltaic power generation technology, photovoltaic power plants have emerged. During the long-term operation of photovoltaic power plants, factors such as dust accumulation, tree shading, and component damage can lead to reduced power generation efficiency and equipment failure. Therefore, it is necessary to inspect the photovoltaic modules in photovoltaic power plants to identify faults in a timely manner.

[0003] In related technologies, drones are beginning to be used to inspect photovoltaic modules in photovoltaic power plants. Drones can be equipped with image acquisition modules to acquire images of the photovoltaic modules. Furthermore, deep learning algorithms are used to detect faults in the photovoltaic module images.

[0004] However, the accuracy of identifying the fault categories of photovoltaic modules in related technologies needs to be improved. Summary of the Invention

[0005] This specification provides a photovoltaic module fault detection method, apparatus, computer equipment, and storage medium to improve the accuracy of identifying the fault type of photovoltaic modules in related technologies.

[0006] This specification provides a photovoltaic module fault detection method, the method comprising: performing target detection on a photovoltaic module image to obtain a preliminary fault detection result; if the preliminary fault category included in the preliminary fault detection result belongs to a first easily misdetected category, performing image classification on the fault area image corresponding to the preliminary fault category to obtain a fault classification result corresponding to the fault area image; wherein, the first easily misdetected category is a fault category that is misdetected during the target detection process due to the similarity of image features of the fault area images; and determining the target fault category corresponding to the fault area image based on the comparison result between the preliminary fault detection result and the fault classification result.

[0007] This specification provides a photovoltaic module fault detection device, comprising: a module fault detection module for performing target detection on a photovoltaic module image to obtain a preliminary fault detection result; a fault image classification module for classifying the fault area image corresponding to the preliminary fault category when the preliminary fault category included in the preliminary fault detection result belongs to a first falsely detected category, to obtain a fault classification result corresponding to the fault area image; wherein, the first falsely detected category is a fault category that is falsely detected due to the similarity of image features of the fault area images during the target detection process; and a fault category determination module for determining the target fault category corresponding to the fault area image based on the comparison result between the preliminary fault detection result and the fault classification result.

[0008] One embodiment of this specification provides a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the above embodiments.

[0009] One embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0010] One embodiment of this specification provides a computer program product including instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0011] In the above-described embodiment, target detection is performed on the photovoltaic module image to obtain a preliminary fault detection result. If the preliminary fault category included in the preliminary fault detection result belongs to the first easily misdetected category, image classification is performed on the fault area image included in the photovoltaic module image to obtain the fault classification result corresponding to the fault area image. The preliminary fault detection result is further compared with the fault classification result, thereby determining the target fault category based on the comparison result. In this way, by using target detection and classification detection in combination, the accuracy of fault category identification is improved when identifying fault categories that are misdetected due to similar image features of fault area images. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a photovoltaic module fault detection system provided in the embodiments of this specification.

[0013] Figure 2 This is a flowchart illustrating the photovoltaic module fault detection method provided in the embodiments of this specification.

[0014] Figure 3 This is a flowchart illustrating the photovoltaic module fault detection method provided in the embodiments of this specification.

[0015] Figure 4a This is a flowchart illustrating the photovoltaic module fault detection method provided in the embodiments of this specification.

[0016] Figure 4b A schematic diagram of the target detection model provided for the implementation of this specification.

[0017] Figure 4c A schematic diagram of the target detection model provided for the implementation of this specification.

[0018] Figure 4d A schematic diagram of the CBAM attention mechanism module provided for the implementation of this specification.

[0019] Figure 4e A schematic diagram of the channel attention module provided in the embodiments of this specification.

[0020] Figure 4f A schematic diagram of the spatial attention module provided for the embodiments of this specification.

[0021] Figure 5 This is a flowchart illustrating the photovoltaic module fault detection method provided in the embodiments of this specification.

[0022] Figure 6 This is a schematic diagram of the structure of the photovoltaic module fault detection device provided in the embodiments of this specification.

[0023] Figure 7 A schematic diagram of the structure of a computer device provided for embodiments of this specification. Detailed Implementation

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] With the development of photovoltaic (PV) power generation technology, PV power plants have emerged, including distributed and centralized PV power plants. As the proportion of newly installed capacity continues to increase, the operation and maintenance of PV power plants faces increasingly severe challenges. During operation, PV modules often experience reduced power generation efficiency due to dust accumulation, tree shading, and module damage, leading to frequent equipment failures and severely impacting project investment returns. Manual inspection requires significant time and effort, suffers from inconsistent standards, low efficiency, high costs, and substantial risks. With the rapid development of drone technology, the application of drones' patrol capabilities for inspecting PV modules holds great promise.

[0026] During the long-term operation of a photovoltaic (PV) power plant, PV modules are subject to obstructions such as bird droppings, loose soil, and fallen leaves. Additionally, PV panels may experience cracks or missing panels, which can be visually detected. Other defects, such as strings not generating power or junction box problems, are internal and not easily detected visually. However, when a PV module malfunctions, the current and voltage of certain individual cells within the module change. This not only reduces the rated voltage of the PV module but also increases the product of local current and voltage, causing a rise in localized temperature. This reduces the lifespan, reliability, and photoelectric conversion efficiency of the PV module, and may even lead to localized cell burnout, melted solder joints, and shattered cover glass. Therefore, accurately identifying the type of PV module malfunction is crucial for the efficient and economical operation of a PV power plant.

[0027] By capturing infrared and / or visible light images of photovoltaic power plants under various environments using drones, and classifying and analyzing the collected images, common faults in photovoltaic power plants can be categorized into six types: hot spots, shading hot spots, cracks, junction box problems, missing modules, and non-generating strings. These faults have a serious impact on current photovoltaic power plants, reducing power generation and shortening the lifespan of photovoltaic modules, thus posing a hidden danger to the safe and stable operation of the power plant.

[0028] Therefore, in order to quickly and accurately detect and identify component faults in photovoltaic power plants, firstly, drone inspections are used to replace traditional manual inspections. Then, deep learning algorithms are used to automatically detect and identify images captured by drones, automatically identifying whether each image contains component faults and the type of fault, thereby improving the operation and maintenance efficiency of photovoltaic power plants and ensuring the power generation efficiency of components.

[0029] In this embodiment, a photovoltaic module image is acquired, target detection is performed on the photovoltaic module image, and the location of the fault area image included in the photovoltaic module image, the preliminary fault category corresponding to the fault area image, and the detection confidence level corresponding to the preliminary fault category are determined. The fault area image is then extracted from the photovoltaic module image based on the location. Next, to accurately identify the fault category of the photovoltaic module, a secondary identification is performed on the preliminary fault category, with different secondary identification processing methods applied to different preliminary fault categories. Specifically, it is determined whether the preliminary fault category belongs to the first falsely detected category. If the preliminary fault category is determined to belong to the first falsely detected category, since images of faults belonging to the first falsely detected category have a certain similarity, to reduce the probability of false detection, the extracted fault area image corresponding to the first falsely detected category is image classified to obtain the fault classification result corresponding to the fault area image. The fault classification result includes the fault category to be determined and the classification confidence level corresponding to the fault category to be determined. Finally, the fault category to be determined is compared with the preliminary fault category. If the preliminary fault category is consistent with the fault category to be determined, then the preliminary fault category and the fault category to be determined can be determined as the target fault category. If the preliminary fault category does not match the fault category to be determined, the classification confidence level corresponding to the fault category to be determined is further compared with the detection confidence level corresponding to the preliminary fault category. If the detection confidence level is greater than the classification confidence level, the preliminary fault category is determined as the target fault category. If the detection confidence level is less than the classification confidence level, the fault category to be determined is determined as the target fault category.

[0030] This specification provides a scenario example of a photovoltaic module fault detection method, which is applied to... Figure 1 The photovoltaic module fault detection system shown is illustrated here. It should be noted that this scenario example is intended to illustrate the application scenario of the photovoltaic module fault detection method, helping readers understand one implementation of the method, and is not intended to limit the application scenario of the photovoltaic module fault detection method.

[0031] Please see Figure 1 The photovoltaic module fault detection system includes a drone 102, a client 104, and a server 106. The server is used to train YOLOv5 and ResNet-50 algorithm models. In this scenario example, YOLOv5 and ResNet-50 algorithm models can be built, and the CBAM (Convolutional Block Attention Module) attention mechanism can be introduced to improve both.

[0032] In this scenario example, a drone inspects a photovoltaic power station. During the inspection, infrared cameras capture and save infrared images. To train the YOLOv5 and ResNet-50 algorithms, these infrared images need to be classified to build training samples. Specifically, images of photovoltaic modules without component faults are extracted from the captured infrared images to identify faulty infrared images. Since six fault categories are pre-defined—hot spots, shading hot spots, cracks, junction box problems, missing modules, and non-generating strings—the faulty infrared images are categorized according to these six categories, with images of the same type of fault stored in the same folder. Annotation tools (such as LabelImg) are used to label the faults on the infrared images. For example, after acquiring the faulty infrared images and their corresponding fault categories, the annotation tools are used to label the defects in the faulty infrared images, accurately marking the location and fault category of each defect. Furthermore, the annotation information is saved as XML files with names corresponding one-to-one with the faulty infrared images.

[0033] In this example scenario, after data annotation, the annotation boxes might be marked outside the boundaries of the infrared image. Therefore, these annotation boxes that exceed the infrared image boundaries are deleted; otherwise, it will affect subsequent model training. This completes the preprocessing of the annotated fault infrared images. Since the number of images corresponding to each fault category is limited, a limited number of fault infrared images are needed for data augmentation. Data augmentation operations such as random rotation, random scaling, and color distortion can be used to ensure that the number of samples for each fault category reaches a certain level. Furthermore, the label data format needs to be converted. The generated labels are in XML format, which needs to be converted to the format required by the YOLO algorithm model. Finally, the dataset is divided into training, validation, and test sets in a 7:1:2 ratio, thus obtaining the fault detection dataset for training the improved YOLOv5 algorithm.

[0034] In this scenario example, analysis revealed that the improved YOLOv5 algorithm did not perform well in detecting hot spots and fragmentation, two fault categories with similar characteristics. Therefore, further fault classification is needed to refine the detection results of the improved YOLOv5 algorithm, thereby enhancing the accuracy of fault identification.

[0035] In this example scenario, the ResNet-50 image classification algorithm is used to classify the fault area image extracted by the YOLOv5 algorithm. Because hot spots and fragmentation have similar features, a CBAM attention mechanism can be added to improve the ResNet-50 algorithm, allowing it to better learn the differences in features between the two and improve the classification performance for hot spots and fragmentation.

[0036] In this example scenario, the improved YOLOv5 and ResNet-50 algorithms are trained using a dataset. Once the model training stops, a photovoltaic module fault detection model and a fault classification model are obtained. These trained models can then be deployed on a client-side application.

[0037] In this example scenario, an image acquisition device is installed on the drone. This device can be an infrared camera. A trained photovoltaic module fault detection model and fault classification model are deployed on the client. The client can connect to the drone's communication equipment to receive infrared images of the photovoltaic module captured by the infrared camera. These infrared images of the photovoltaic module can then be used as input to the photovoltaic module fault detection model.

[0038] In this scenario example, the client receives an infrared image of a photovoltaic module and inputs it into the photovoltaic module fault detection model. The photovoltaic module fault detection model detects and identifies the photovoltaic module's infrared image and outputs the detected fault category, the corresponding detection confidence level, and the detection location of the photovoltaic module's defect in the photovoltaic module's infrared image.

[0039] In this scenario example, different secondary identification methods need to be executed for different detected fault categories output by the photovoltaic module fault detection model. Therefore, it is necessary to judge the detected fault categories output by the photovoltaic module fault detection model.

[0040] In this scenario example, when determining whether the detected fault category is hot spot or fragmentation, the defect area image from the photovoltaic module's infrared image is input into the fault classification model. The fault classification model performs secondary recognition on the defect area image, identifying the corresponding fault category and outputting the corresponding classification confidence score. The detected fault category output by the photovoltaic module fault detection model is compared with the fault category identified by the fault classification model. If they match, for example, both are hot spots, then the defect in the photovoltaic module's infrared image is determined to include hot spots; if both are fragmentation, then the defect in the photovoltaic module's infrared image is determined to include fragmentation. If they do not match, the detection confidence score and classification confidence score are compared, and the fault category corresponding to the higher confidence score is determined as the final fault category. For example, if the photovoltaic module fault detection model outputs a hot spot with a detection confidence score of 0.85, and the fault classification model outputs fragmentation with a classification confidence score of 0.6, then since 0.85 is greater than 0.6, the defect in the photovoltaic module's infrared image is a hot spot.

[0041] In this scenario example, because the reflection from the photovoltaic module can lead to false detections of non-power-generating strings as a fault category, a secondary verification is performed when the fault category is determined to be a non-power-generating string. Specifically, a specific number of pixels are randomly selected from the defect area image in the infrared image of the photovoltaic module, and the average pixel value of these pixels is calculated. The calculated average pixel value is compared with an initial threshold. If the average pixel value is greater than or equal to the initial threshold, the defect in the infrared image of the photovoltaic module is determined to include non-power-generating strings, and the output fault category remains unchanged. If the average pixel value is less than the initial threshold, the defect area image is considered to contain no non-power-generating strings and is a fault-free area, and the output fault category is modified.

[0042] In this scenario example, since the characteristics of the three fault categories of junction box failure, missing components, and blocked hot spots are relatively obvious and they do not have similar characteristics to other fault categories, if the detected fault category is determined to be any one of junction box failure, missing components, or blocked hot spots, no secondary identification is required, and it can be directly used as the final detection result output.

[0043] In this scenario example, the fault location and fault type can be further marked on the infrared image of the photovoltaic module.

[0044] In the above scenario example, the photovoltaic module fault detection model and the fault classification model are used to detect hot spots or cracks that are prone to false positives twice. This reduces the false positive rate caused by the similarity of features between hot spots and cracks, and improves the accuracy of fault detection results. By extracting the pixel mean, the probability of false positives for non-power-generating strings caused by photovoltaic module reflection is reduced. Finally, the fault detection results are presented intuitively by outputting a result image marked with the fault location and fault category.

[0045] This specification provides a method for detecting photovoltaic module faults. Please refer to [link / reference]. Figure 2 The photovoltaic module fault detection method may include the following steps:

[0046] S210. Perform target detection on the photovoltaic module image to obtain preliminary fault detection results.

[0047] The photovoltaic module image can be an infrared or visible light image obtained by photographing the photovoltaic module. The preliminary fault detection result can be obtained by defect detection on the photovoltaic module image, and may also require preliminary detection results for secondary identification.

[0048] In some cases, faults in photovoltaic (PV) modules can reduce their lifespan, reliability, and photoelectric conversion efficiency, and may even cause localized cell burnout, melted solder joints, or shattered cover glass. Therefore, target detection can be used to accurately identify PV module faults. Specifically, a pre-trained target detection model can be deployed. The PV module image is input into the target detection model, which detects faults in the image, identifying defects or their target locations within the image. Based on these target locations, a fault region image can be determined within the PV module image. This fault region image indicates the presence of a PV module fault. The fault region image can be a partial image of the PV module defect in an infrared or visible light image. The target detection model can also output a preliminary fault category corresponding to the fault region image and a detection confidence score for that category. For example, the target detection model can be a YOLOv5 network model incorporating the CBAM attention mechanism.

[0049] S220. If the preliminary fault category included in the preliminary fault detection results belongs to the first easily misdetected category, perform image classification on the fault area image corresponding to the preliminary fault category to obtain the fault classification result corresponding to the fault area image.

[0050] The first category of faults prone to false detection is the category of faults that are falsely detected during the target detection process due to the similarity of image features in the images of the fault areas.

[0051] In some cases, the initial fault category can be any one of six fault categories: hot spot, obstructed hot spot, fracture, junction box problem, missing component, and non-generating string. However, among these six fault categories, some with similar characteristics are prone to false positives. Fault categories that are easily misidentified due to similar image features of the fault area can be designated as the first easily misidentified category. The first easily misidentified category can include hot spots and fractures. For example, hot spots and fractures may be identified as fractures due to their similar features, or vice versa. Therefore, it is necessary to judge the initial fault category included in the initial fault detection results to improve the reliability of the fault detection results.

[0052] Specifically, the preliminary fault detection results include a preliminary fault category, and a determination of whether the preliminary fault category belongs to the first category most prone to false detection. If the preliminary fault category is determined to belong to the first category most prone to false detection, it indicates that a secondary confirmation of the preliminary fault category is required. The fault region image obtained from target detection is input into a classification model, which classifies the fault region image to obtain the fault classification result. The fault classification result may include at least one of the fault category corresponding to the fault region image and the corresponding classification confidence score.

[0053] For example, the classification model can employ a ResNet network with an incorporated CBAM attention mechanism. Since hot spots and fragmentation have similar features, adding the CBAM attention mechanism can better learn the differences between their features, thus improving the classification performance for hot spots and fragmentation.

[0054] S230. Based on the comparison between the preliminary fault detection results and the fault classification results, determine the target fault category corresponding to the fault area image.

[0055] The preliminary fault detection results may include at least one of a preliminary fault category and a corresponding detection confidence score. The fault classification results may include at least one of a fault category corresponding to the fault region image and a corresponding classification confidence score.

[0056] In some cases, to accurately identify photovoltaic module faults, preliminary fault detection results are compared with fault classification results to determine the final fault detection result. Specifically, the preliminary fault categories included in the preliminary fault detection results are compared with the fault categories included in the fault classification results to obtain a fault category comparison result. Based on the fault category comparison result, the target fault category is determined. If they match, either the preliminary fault category included in the preliminary fault detection results or the fault category included in the fault classification results can be determined as the target fault category. If they do not match, the fault category included in the fault classification results can be directly determined as the target fault category. If they still do not match, the detection confidence level included in the preliminary fault detection results and the classification confidence level included in the fault classification results can be compared, and the one with the higher confidence level can be determined as the target fault category.

[0057] In the above embodiments, target detection is performed on the photovoltaic module image to obtain preliminary fault detection results. If the preliminary fault category included in the preliminary fault detection results belongs to the first easily misdetected category, the fault area image included in the photovoltaic module image is classified to obtain the fault classification result corresponding to the fault area image. The preliminary fault detection results and the fault classification results are further compared to determine the target fault category based on the comparison results. In this way, by using target detection and classification detection in combination, the accuracy of fault category identification is improved when identifying fault categories that are misdetected due to similar image features of fault area images. This reduces the impact of photovoltaic module faults on the operation of photovoltaic power plants and is conducive to the stable operation of photovoltaic power plants.

[0058] In some implementations, please refer to Figure 3 The photovoltaic module fault detection method may also include the following steps:

[0059] S310. If the preliminary fault category included in the preliminary fault detection results belongs to the second easily misdetectable category, determine several discontinuous target pixels in the fault area image.

[0060] S320. Based on the comparison result between the average pixel value of the target pixel and the preset pixel threshold, determine the fault status of the photovoltaic module.

[0061] The second category of false detections refers to the category of faults that are falsely detected during the target detection process because the image features of the fault area image are similar to the image features of a specific scene image.

[0062] In some cases, the initial fault category can be any one of six fault categories: hot spot, obstructed hot spot, fracture, junction box problem, missing component, and non-generating string. However, some of these six fault categories may be falsely identified due to similarity in image features to images of specific scenes. These falsely identified fault categories due to similarity in image features between the fault area image and the specific scene image can be designated as the second falsely identified category. The second falsely identified category can include non-generating strings. The specific scene image can be a component reflection image. For example, non-generating strings and component reflections may be identified as either non-generating strings or vice versa due to their similar features. Therefore, it is necessary to judge the initial fault category included in the initial fault detection results to improve the reliability of the fault detection results. Furthermore, there are differences in pixel values ​​between the image of a non-generating string and the image of component reflection; therefore, the initial fault category can be re-identified based on the pixel values ​​in the fault area image.

[0063] Specifically, the initial fault category in the preliminary fault detection results is determined to belong to the second category prone to false detection. If it does, a specific number of pixels need to be extracted from the fault area image, and the average pixel value of these pixels is calculated. In some cases, the temperature distribution in the fault area is uneven, and the pixel values ​​of the pixels in the fault area image are also uneven, while the pixel values ​​of the pixels in the specific scene image are evenly distributed. Therefore, several discontinuous target pixels can be identified in the fault area image. The average pixel value of the target pixels is determined based on their pixel values. The average pixel value of the target pixels is compared with a preset pixel threshold. Based on the comparison result, it is verified whether the initial fault category belongs to the second category prone to false detection, or whether a component in the specific scene image is mistakenly detected as a faulty component.

[0064] For example, the temperature distribution in the non-generating string area is uneven, and the pixel values ​​of the pixels in the fault area image corresponding to the non-generating string are also uneven, while the pixel values ​​of the pixels in the component reflective area image are evenly distributed. Therefore, several discontinuous target pixels can be identified in the fault area image. The average pixel value of the target pixels is determined based on their pixel values. The average pixel value of the target pixels is compared with a preset pixel threshold. Based on the comparison result, it is verified whether the preliminary fault category belongs to the non-generating string or the reflective component.

[0065] In the above implementation, by determining whether the preliminary fault category included in the preliminary fault detection result belongs to the second easily misdetected category, if it does, several discontinuous target pixels are determined in the fault area image. Based on the comparison result of the pixel mean of the target pixels and the preset pixel threshold, the fault condition of the photovoltaic module is determined, reducing the probability of false detection events of faulty modules caused by characteristic scenes and improving the accuracy of fault detection results.

[0066] In some implementations, the fault status of the photovoltaic module is determined based on the comparison between the average pixel value of the target pixel and a preset pixel threshold, including at least one of the following: if the average pixel value is less than the preset pixel threshold, the preliminary fault category included in the preliminary fault detection result is discarded, and the photovoltaic module is determined to be in a fault-free state; or, if the average pixel value is not less than the preset pixel threshold, the target fault category corresponding to the photovoltaic module is determined to include the preliminary fault category.

[0067] Specifically, if the average pixel value is less than a preset pixel threshold, it indicates that the image of the fault area is an image captured for the feature scene, not an image captured for the fault area. Therefore, the preliminary fault category included in the preliminary fault detection results is discarded, and the photovoltaic module is determined to be in a fault-free state.

[0068] When the average pixel value is not less than the preset pixel threshold, it indicates that the image of the fault area is not an image captured for the feature scene, but an image captured for the fault area. There is no need to modify the preliminary fault category included in the preliminary fault detection results, and the target fault category corresponding to the photovoltaic module is determined to include the preliminary fault category.

[0069] In the above embodiments, by comparing the average pixel value to a preset pixel threshold, a secondary judgment is made on the preliminary fault category included in the initial fault detection results, thereby reducing the probability of false detection events of faulty components caused by characteristic scenarios and improving the accuracy of fault detection results.

[0070] In some implementations, the first category prone to false detection is the hot spot fault category or the fragmentation fault category.

[0071] Specifically, the initial fault category can be any one of six fault categories: hot spot, obstructed hot spot, fracture, junction box problem, missing component, and non-generating string. Among these six fault categories, some fault categories with similar characteristics are prone to false detection. Fault categories that are prone to false detection due to similar image features of the fault area can be designated as the first false detection category. The first false detection category is either the hot spot fault category or the fracture fault category.

[0072] In some implementations, the second category of easily misdetected components is the non-power-generating component failure category, and the specific scene image is the photovoltaic module reflection image.

[0073] Specifically, among the six fault categories—hot spots, obstructed hot spots, cracks, junction box problems, missing modules, and non-generating strings—some fault categories exhibit image features similar to those of specific scene images, leading to false detections. These fault categories, where false detections occur due to the similarity between the image features of the fault area and those of the specific scene image, can be categorized as the second easily misdetected category. The second easily misdetected category is the non-generating module fault category, and the specific scene image is the reflective image of the photovoltaic module.

[0074] In the above embodiments, by performing two inspections on hot spots or cracks that are prone to false detection, the false detection rate caused by the similarity of the characteristics of hot spots and cracks is reduced, thereby improving the accuracy of fault detection results. By re-confirming non-power-generating strings, the probability of false detection events of non-power-generating strings caused by photovoltaic module reflection is reduced.

[0075] In some implementations, determining the target fault category corresponding to the fault area image based on the comparison between the preliminary fault detection result and the fault classification result may include: if the preliminary fault category is consistent with the fault category to be determined included in the fault classification result, determining the preliminary fault category or the fault category to be determined as the target fault category.

[0076] Specifically, when the initial fault category is determined to belong to the first easily misdetected category, it indicates that a secondary confirmation of the initial fault category is required. The fault area image obtained from target detection is input into the classification model, which classifies the fault area image to obtain the fault classification result. The fault classification result can include the fault category corresponding to the fault area image. The initial fault category included in the initial fault detection result is compared with the fault category included in the fault classification result. If they match, the initial fault category included in the initial fault detection result or the fault category included in the fault classification result can be determined as the target fault category.

[0077] In some implementations, the preliminary fault detection result includes the detection confidence level corresponding to the preliminary fault category, and the fault classification result includes the fault category to be determined and the classification confidence level corresponding to the fault category to be determined. Determining the target fault category corresponding to the fault region image based on the comparison between the preliminary fault detection result and the fault classification result may include: if the preliminary fault category and the fault category to be determined included in the fault classification result are inconsistent, determining the fault category corresponding to the higher confidence level between the detection confidence level and the classification confidence level as the target fault category.

[0078] Specifically, the preliminary fault categories included in the preliminary fault detection results are compared with the undetermined fault categories included in the fault classification results. If the two are inconsistent, the detection confidence levels included in the preliminary fault detection results and the classification confidence levels included in the fault classification results can be further compared. The category with the higher confidence level can be determined as the target fault category. For example, if the detection confidence level is greater than the classification confidence level, the preliminary fault category is determined as the target fault category. If the detection confidence level is less than the classification confidence level, the undetermined fault category is determined as the target fault category.

[0079] In the above embodiments, by combining target detection and classification detection, the accuracy of fault category identification is improved when identifying fault categories that are falsely detected due to similar image features of fault area images. This reduces the impact of photovoltaic module faults on the operation of photovoltaic power plants and is conducive to the stable operation of photovoltaic power plants.

[0080] In some embodiments, the photovoltaic module fault detection method may further include: determining the preliminary fault category as the target fault category when the preliminary fault category included in the preliminary fault detection results does not belong to the first falsely detected category and does not belong to the second falsely detected category.

[0081] In some cases, the initial fault category can be any one of six fault categories: hot spot, obstructed hot spot, fracture, junction box problem, missing component, and non-generating string. The first false detection category is the fault category that is falsely detected during target detection due to similar image features of the fault area image. The second false detection category is the fault category that is falsely detected during target detection due to similar image features of the fault area image and image features of a specific scene image. Among the six fault categories, there exist fault categories that belong neither to the first nor the second false detection category. For this type of fault category, its features are relatively obvious and it does not have similar features to other fault categories. Therefore, the output of the target detection process is reliable and does not require secondary identification. Specifically, if the initial fault category included in the initial fault detection result does not belong to either the first or the second false detection category, the initial fault category is determined as the target fault category.

[0082] For example, since the characteristics of the three fault categories—junction box failure, missing components, and blocked hot spots—are relatively obvious and do not have similar characteristics to other fault categories, if the initial fault category is determined to be any one of these three, no secondary identification is required, and the initial fault category can be directly output as the final detection result.

[0083] In the above embodiments, different processing methods are adopted for different objects in the six fault categories of hot spots, blocked hot spots, cracks, junction box problems, missing components and non-generating strings. On the basis of improving the accuracy of fault detection, unnecessary calculation processes are reduced and fault detection efficiency is improved.

[0084] In some implementations, the preliminary fault detection results also include the location information of the fault area image corresponding to the target fault category within the photovoltaic module image. The photovoltaic module fault detection method may further include: marking the photovoltaic module image based on the location information and the target fault category corresponding to the fault area image, so as to indicate the fault location and fault category within the photovoltaic module image.

[0085] Specifically, to visually present the fault detection results, the fault area image corresponding to the target fault category is bounded within the photovoltaic module image based on its location information within the photovoltaic module image, and the target fault category is displayed at an appropriate location, thereby marking the fault location and fault category in the photovoltaic module image. In some implementations, the marked photovoltaic module image can be saved.

[0086] In some implementations, please refer to Figure 4a To perform target detection on photovoltaic module images and obtain preliminary fault detection results, the following steps may be included:

[0087] S410. Input the photovoltaic module image into the target detection model for feature extraction to obtain image features of different sizes.

[0088] S420. The image features of at least some different sizes are weighted in the channel dimension and spatial dimension respectively by the convolutional block attention mechanism module to obtain the final image features.

[0089] S430. Determine the preliminary results of fault detection based on the final image features.

[0090] The object detection model comprises several network branches. Some of these branches have convolutional block attention mechanisms at their ends.

[0091] The convolutional block attention mechanism module includes a channel attention module for channel-weighted processing and a spatial attention module for spatial-weighted processing. This not only saves parameters and computing power, but also ensures that it can be integrated into existing network architectures as a plug-and-play module.

[0092] In some cases, photovoltaic module images may contain regions with multiple fault categories. Different fault categories have different sizes and occupy different areas within the photovoltaic module image. Therefore, the target detection model is designed with several network branches to extract image features of different sizes. Furthermore, during module fault detection and identification, there are instances of missed and false identifications, resulting in the detection accuracy falling short of expectations. Therefore, a CBAM (Convolutional Block Attention Module) attention mechanism module is introduced, specifically as follows: Figure 4b As shown, by adding the CBAM attention mechanism module, the effect of component fault detection and identification is improved, the detection accuracy is increased, and the false identification rate of component faults is reduced, thus reducing the probability of misidentifying non-faulty areas as component faults.

[0093] Specifically, the structure of the object detection model is as follows: Figure 4b As shown, an image of a photovoltaic module is input into the object detection model. Feature extraction is performed on the photovoltaic module image through each network branch, resulting in image features of several different sizes. These different sized image features can be used to determine faults of different sizes. For network branches with convolutional block attention mechanisms at their ends, different sized image features can be output. For example, the object detection model can use a YOLOv5 network. Figure 4c As shown, Figure 4c A portion of the YOLOv5 network is shown, with CBAM attention mechanism modules introduced at the ends of some branches of the YOLOv5 network.

[0094] In this embodiment, image features of different sizes are input into the convolutional block attention mechanism module. The convolutional block attention mechanism module includes a channel attention module and a spatial attention module. The channel attention module performs channel-weighted image feature processing on the image features along the channels, resulting in channel-weighted image features. The spatial attention module then performs spatial weighted processing on the channel-weighted image features, resulting in the final image features. Based on the final image features, a preliminary fault detection result is determined. For example, as shown... Figure 4dAs shown, the convolutional block attention mechanism module can adopt the CBAM attention mechanism module. The CBAM attention mechanism module includes a Channel Attention Module (CAM) and a Spartial Attention Module (SAM).

[0095] The Channel Attention Module (CAM) structure is as follows: Figure 4e As shown, the specific process is as follows: The input feature map F (H×W×C) is processed by global max pooling based on width and global average pooling based on height, resulting in two 1×1×C feature maps. Next, the two 1×1×C feature maps are fed into a two-layer neural network (MLP). The first layer has C / r neurons (r is the reduction rate) and uses ReLU activation, while the second layer has C neurons. These two layers are shared. Then, the features output by the MLP are summed element-wise, followed by sigmoid activation to generate the final channel-weighted image feature (Mc). Finally, Mc is multiplied element-wise with the input feature map F to obtain the input features required by the spatial attention module.

[0096] The structure of the Spartial Attention Module (SAM) is as follows: Figure 4f As shown. The specific process is as follows: The feature map output by the channel attention module is used as the input feature map of this module. First, a channel-based global max pooling and global average pooling are performed to obtain two H×W×1 feature maps. Then, the two H×W×1 feature maps are concatenated based on the channel (concat operation). Then, a 7×7 convolution operation is performed to reduce the dimensionality to 1 channel, i.e., H×W×1. Then, a sigmoid function is used to generate a spatially weighted image feature (Ms). Finally, this spatially weighted image feature is multiplied by the input feature of this module to obtain the final image feature.

[0097] In the above embodiments, by setting a convolutional block attention mechanism module at the end of some network branches of the target detection model, the convolutional block attention mechanism module is used to perform weighted processing on at least some image features of different sizes in the channel dimension and spatial dimension respectively to obtain the final image features, so that the target detection model pays more attention to recognizing target objects, thereby improving the detection and recognition effect of component faults.

[0098] This specification provides a photovoltaic module fault detection method in its embodiments. Please refer to [link / reference]. Figure 5 The photovoltaic module fault detection method includes the following steps:

[0099] S502. Input the photovoltaic module image into the target detection model for feature extraction to obtain image features of different sizes.

[0100] The object detection model includes several network branches; some network branches have convolutional block attention mechanism modules at their ends.

[0101] S504. The image features of at least some different sizes are weighted in the channel dimension and spatial dimension respectively by the convolutional block attention mechanism module to obtain the final image features.

[0102] S506. Determine the preliminary results of fault detection based on the final image features.

[0103] The preliminary fault detection results include the preliminary fault category, the detection confidence level corresponding to the preliminary fault category, the fault area image corresponding to the target fault category, and the location information of the fault area image in the photovoltaic module image.

[0104] S508. If the preliminary fault category included in the preliminary fault detection results belongs to the first easily misdetected category, perform image classification on the fault area image corresponding to the preliminary fault category to obtain the fault classification result corresponding to the fault area image.

[0105] The first category of faults prone to false detection refers to fault categories that are falsely detected during target detection due to the similarity of image features in the fault area images. For example, the first category of faults prone to false detection is either a hot spot fault or a fragmentation fault.

[0106] The fault classification results include the fault category to be determined and the classification confidence level corresponding to the fault category to be determined.

[0107] S510. If the preliminary fault category is consistent with the fault category to be determined included in the fault classification result, determine the preliminary fault category or the fault category to be determined as the target fault category.

[0108] S512. If the preliminary fault category is inconsistent with the fault category to be determined included in the fault classification result, the fault category corresponding to the higher confidence level between the detection confidence level and the classification confidence level shall be determined as the target fault category.

[0109] S514. If the preliminary fault category included in the preliminary fault detection results belongs to the second easily misdetectable category, determine several discontinuous target pixels in the fault area image.

[0110] The second category of faults prone to false detection refers to fault categories that are falsely detected during target detection due to the similarity between the image features of the fault area image and the image features of a specific scene image. For example, the second category of faults prone to false detection is the non-power-generating component fault category, and the specific scene image is a reflective image of a photovoltaic module.

[0111] S516. Based on the comparison result between the average pixel value of the target pixel and the preset pixel threshold, determine the fault status of the photovoltaic module.

[0112] Specifically, if the average pixel value is less than a preset pixel threshold, the preliminary fault categories included in the preliminary fault detection results are discarded, and the photovoltaic module is determined to be in a fault-free state; or, if the average pixel value is not less than a preset pixel threshold, the target fault category corresponding to the photovoltaic module is determined to include the preliminary fault category.

[0113] S518. If the preliminary fault category included in the preliminary fault detection results does not belong to the first false detection category and does not belong to the second false detection category, the preliminary fault category shall be determined as the target fault category.

[0114] Furthermore, the photovoltaic module image can be marked according to the location information and the target fault category corresponding to the fault area image, so as to indicate the fault location and fault category in the photovoltaic module image.

[0115] This specification provides a photovoltaic module fault detection device in its embodiments. Please refer to [link / reference]. Figure 6 The photovoltaic module fault detection device 600 includes: a module fault detection module 610, a fault image classification module 620, and a fault category determination module 630.

[0116] The component fault detection module 610 is used to perform target detection on the photovoltaic module image and obtain preliminary fault detection results.

[0117] The fault image classification module 620 is used to classify the fault area image corresponding to the preliminary fault category when the preliminary fault category included in the preliminary fault detection result belongs to the first easily misdetected category, and obtain the fault classification result corresponding to the fault area image; wherein, the first easily misdetected category is the fault category that is misdetected due to the similarity of image features of the fault area images during the target detection process.

[0118] The fault category determination module 630 is used to determine the target fault category corresponding to the fault area image based on the comparison result between the preliminary fault detection result and the fault classification result.

[0119] Specific limitations regarding the photovoltaic module fault detection device can be found in the limitations of the photovoltaic module fault detection method described above, and will not be repeated here. Each module in the aforementioned photovoltaic module fault detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0120] This specification provides a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the above embodiments.

[0121] This specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0122] This specification provides a computer program product that includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0123] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a photovoltaic module fault detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0124] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0126] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] In the description of this specification, references to terms such as "an embodiment," "some implementations," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0129] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0130] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A photovoltaic module fault detection method, characterized by, The method comprises: target detection is performed on a photovoltaic module image to obtain a preliminary fault detection result; in a case where a preliminary fault category included in the preliminary fault detection result belongs to a first false detection-prone category, image classification is performed on a fault region image corresponding to the preliminary fault category to obtain a fault classification result corresponding to the fault region image; the first false detection-prone category is a fault category that causes false detection in the process of target detection due to similar image features of the fault region image; a target fault category corresponding to the fault region image is determined based on a comparison result of the preliminary fault detection result and the fault classification result; The method further comprises: in a case where a preliminary fault category included in the preliminary fault detection result belongs to a second false detection-prone category, a plurality of discontinuous target pixels are determined in the fault region image; the second false detection-prone category is a fault category that causes false detection in the process of target detection due to similar image features of the fault region image and image features of a specific scene image; a fault condition of the photovoltaic module is determined based on a comparison result of a pixel mean value of the target pixels and a preset pixel threshold.

2. The method of claim 1, wherein, The determination of the fault condition of the photovoltaic module based on the comparison result of the pixel mean value of the target pixels and the preset pixel threshold comprises at least one of the following: in a case where the pixel mean value is less than the preset pixel threshold, the preliminary fault category included in the preliminary fault detection result is discarded, and it is determined that the photovoltaic module is in a fault-free state; in a case where the pixel mean value is not less than the preset pixel threshold, it is determined that the target fault category corresponding to the photovoltaic module includes the preliminary fault category.

3. The method of claim 1, wherein, The second false detection-prone category is a non-power generation component fault category, and the specific scene image is a photovoltaic module reflection image.

4. The method of claim 1, wherein, The first false detection-prone category is a hot spot fault category or a fragmentation fault category.

5. The method of claim 1, wherein, The determination of the target fault category corresponding to the fault region image based on the comparison result of the preliminary fault detection result and the fault classification result comprises: in a case where the preliminary fault category is consistent with a to-be-determined fault category included in the fault classification result, the preliminary fault category or the to-be-determined fault category is determined as the target fault category.

6. The method of claim 1, wherein, The preliminary fault detection result includes a detection confidence corresponding to the preliminary fault category, and the fault classification result includes a to-be-determined fault category and a classification confidence corresponding to the to-be-determined fault category; in a case where the preliminary fault category is inconsistent with the to-be-determined fault category included in the fault classification result, a fault category corresponding to a higher confidence between the detection confidence and the classification confidence is determined as the target fault category. The method further comprises:

7. The method of claim 1, wherein, in a case where the preliminary fault category included in the preliminary fault detection result does not belong to the first false detection-prone category and does not belong to the second false detection-prone category, the preliminary fault category is determined as the target fault category. ​ 8. The method according to any one of claims 1 to 7, characterized in that, The method comprises the following steps: The photovoltaic module image is input into a target detection model for feature extraction, to obtain a plurality of image features of different sizes; wherein the target detection model comprises a plurality of network branches; a convolution block attention mechanism module is arranged at the end of part of the network branches; The convolution block attention mechanism module is used to perform weighted processing on at least part of the image features of different sizes in the channel dimension and the spatial dimension, to obtain final image features; The final image features are used to determine the preliminary fault detection result.

9. The method according to any one of claims 1 to 7, characterized in that, The preliminary fault detection result further comprises position information of a fault region image corresponding to the target fault category in the photovoltaic module image; the method further comprises: The photovoltaic module image is labeled according to the position information and the target fault category corresponding to the fault region image, to mark the fault position and the fault category in the photovoltaic module image.

10. A photovoltaic module fault detection apparatus for implementing the method of any one of claims 1 to 9, characterized by, The device comprises: A component fault detection module, configured to perform target detection on a photovoltaic module image, to obtain a preliminary fault detection result; A fault image classification module, configured to, in a case where a preliminary fault category included in the preliminary fault detection result belongs to a first false detection prone category, perform image classification on a fault region image corresponding to the preliminary fault category, to obtain a fault classification result corresponding to the fault region image; wherein the first false detection prone category is a fault category that is prone to false detection due to similar image features of the fault region image in the target detection process; A fault category determination module, configured to determine a target fault category corresponding to the fault region image based on a comparison result of the preliminary fault detection result and the fault classification result.

11. A computer device, comprising: A computer program product, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein the computer readable storage medium controls a device in which the computer readable storage medium is located to execute the method according to any one of claims 1 to 9 when the computer program is running.

Images