Defect detection method and device for photovoltaic module of power station, equipment and medium

Abstract

The invention provides a power station photovoltaic module defect detection method and device, equipment and a medium, and the method comprises the steps: employing a photovoltaic segmentation model comprising a lightweight decoder to detect a region where a photovoltaic module is located in a visible light image and a thermal imaging image, and precisely segmenting a visible light photovoltaic module image and a thermal imaging photovoltaic module image; afterwards, according to the invention, the multi-task defect detection model obtained by improving the Faster Net model is used for detecting the photovoltaic defects in the thermal imaging photovoltaic module image and the visible light photovoltaic module image, so that not only can the temperature-sensitive photovoltaic defects be detected, but also the temperature-insensitive photovoltaic defects can be detected at the same time; and the excellent detection precision is ensured, the parameter quantity is small, and the nearly real-time detection speed is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of clean energy, and in particular to the field of defect detection technology, and discloses a method, device, equipment, and medium for detecting defects in photovoltaic modules of a power station. Background Art

[0002] Photovoltaic power generation uses renewable resources such as solar energy to achieve large-scale power generation through clean photoelectric conversion technology, which can significantly reduce dependence on fossil energy and significantly reduce greenhouse gas emissions. In addition, photovoltaic power generation has cost advantages, increases energy diversity, and ensures energy security, which is of great significance to sustainable development and renewable energy strategies.

[0003] Photovoltaic modules in power stations are prone to failure due to long-term exposure to the natural environment. Typical failures such as hot spots will cause the electrical efficiency of the area corresponding to the hot spots to drop significantly, seriously affecting the power generation of the entire photovoltaic module, and may even cause problems with the entire module. Hot spots are temperature-sensitive defects. In addition, photovoltaic modules are also subject to other damages due to long-term exposure to the natural environment, resulting in other temperature-insensitive defects, such as VA shedding, foreign body obstruction, cracks, yellowing, bubbles, etc. These defects also have a great impact on power generation efficiency.

[0004] At present, the detection of the above defects has the following defects: low accuracy, poor robustness, susceptibility to environmental factors, and low detection efficiency. Summary of the invention

[0005] The present disclosure at least provides a method and device, equipment, and medium for detecting defects in photovoltaic modules of a power station to solve at least one of the above-mentioned technical problems.

[0006] According to one aspect of the present disclosure, a method for detecting defects of photovoltaic modules in a power station is provided, comprising:

[0007] Collect thermal imaging images and visible light images to be detected in the target area where the photovoltaic components are set;

[0008] The thermal imaging image and the visible light image are segmented using the trained photovoltaic segmentation model to obtain a first mask image of the area where the photovoltaic components are located in the thermal imaging image and a second mask image of the area where the photovoltaic components are located in the visible light image; the image of the area where the photovoltaic components are located is cropped from the thermal imaging image using the first mask image to obtain a thermal imaging photovoltaic component image; the image of the area where the photovoltaic components are located is cropped from the visible light image using the second mask image to obtain a visible light photovoltaic component image;

[0009] The thermal imaging photovoltaic module image and the visible light photovoltaic module image are processed respectively by using the trained multi-task defect detection model to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results at least include visible light defect detection results; the temperature-sensitive defect detection results at least include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results at least include the center point position, height and width information, confidence, and defect type of visible light type defects;

[0010] The temperature-sensitive defect detection results and the temperature-insensitive defect detection results with confidence levels higher than a set threshold are used as target defect detection results;

[0011] The photovoltaic segmentation model includes an image coding network and a lightweight decoder in sequence; the lightweight decoder includes a convolution layer, an LN, a convolution layer, an LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence;

[0012] The multi-task defect detection model includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; the FasterNet model main structure includes the first FasterNet Block, the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block after the FasterNet model removes Global Pool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNetBlock and the fourth FasterNetBlock are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head; the PANet structure includes a convolution layer, a network layer corresponding to the upsampling operation, and a network layer corresponding to the splicing operation; the loss function of the multi-task defect detection model includes: a category prediction loss function, a bounding box regression loss function and a confidence loss function.

[0013] In a possible implementation, the image coding network is an image coding network in a SegmentAnything Model network, and when the photovoltaic segmentation model is trained, weight information of the image coding network is frozen.

[0014] In a possible implementation, the step of generating a data set for training the photovoltaic segmentation model is also included:

[0015] Collect multiple visible light sample images to obtain a visible light original image set; collect multiple thermal imaging sample images to obtain a thermal imaging original image set;

[0016] For each sample image in the visible light original image set, a visible light defect area is annotated to obtain a first visible light annotated data set; for each sample image in the visible light original image set, a photovoltaic module area is annotated to obtain a second visible light annotated data set; the visible light original image set and the second visible light annotated data set are used as original visible light segmentation data sets; the original visible light segmentation data set is divided into a visible light segmentation training set and a visible light segmentation test set;

[0017] For each sample image in the thermal imaging original image set, hot spot area annotation is performed to obtain a first thermal imaging annotation data set; for each sample image in the thermal imaging original image set, photovoltaic module area annotation is performed to obtain a second thermal imaging annotation data set; the thermal imaging original image set and the second thermal imaging annotation data set are used as original thermal imaging segmentation data sets; the original thermal imaging segmentation data set is divided into a thermal imaging segmentation training set and a thermal imaging segmentation test set;

[0018] The visible light segmentation training set is subjected to online random data enhancement processing to obtain a target visible light segmentation training set; and the thermal imaging segmentation training set is subjected to online random data enhancement processing to obtain a target thermal imaging segmentation training set; the target visible light segmentation training set, the target thermal imaging segmentation training set, the visible light segmentation test set and the thermal imaging segmentation test set are used as data sets for training the photovoltaic segmentation model.

[0019] In a possible implementation, the step of generating a data set for training the multi-task defect detection model is also included:

[0020] For each visible light sample image in the original visible light segmentation data set, according to the annotation information corresponding to the visible light sample image in the second visible light annotation data set, an image of the area where the photovoltaic component is located is cropped from the visible light sample image, and the annotation information corresponding to the visible light sample image in the first visible light annotation data set is updated to the cropped image to obtain a visible light photovoltaic annotated image;

[0021] Copying at least once the image area of ​​at least one type of visible light defect in all visible light photovoltaic annotated images, and randomly pasting the copied image area into a visible light photovoltaic annotated image to obtain an original visible light photovoltaic data set; dividing the original visible light photovoltaic data set into a visible light photovoltaic training set and a visible light photovoltaic test set;

[0022] For each thermal imaging sample image in the original thermal imaging segmentation data set, according to the annotation information corresponding to the thermal imaging sample image in the second thermal imaging annotation data set, an image of the area where the photovoltaic component is located is cropped from the thermal imaging sample image, and the annotation information corresponding to the thermal imaging sample image in the first thermal imaging annotation data set is updated to the cropped image to obtain a thermal imaging photovoltaic annotation image;

[0023] Copying at least one image area of ​​a hot spot defect in all thermal imaging photovoltaic annotated images at least once, and randomly pasting the copied image area into a thermal imaging photovoltaic annotated image to obtain an original thermal imaging photovoltaic data set; dividing the original thermal imaging photovoltaic data set into a thermal imaging photovoltaic training set and a thermal imaging photovoltaic test set;

[0024] The visible light photovoltaic training set, the visible light photovoltaic test set, the thermal imaging photovoltaic training set and the thermal imaging photovoltaic test set are used as data sets for training the multi-task defect detection model.

[0025] In a possible implementation, the step of training the multi-task defect detection model is further included:

[0026] Using the visible light photovoltaic training set and the thermal imaging photovoltaic training set, the multi-task defect detection model to be trained is trained to obtain the multi-task defect detection model with initial training completed;

[0027] Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the visible light defect detection head, use the thermal imaging photovoltaic training set to train the multi-task defect detection model completed in the initial state training, adjust the weight information of the thermal imaging defect detection head, and obtain the multi-task defect detection model completed in the intermediate state training;

[0028] Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the thermal imaging defect detection head, use the visible light photovoltaic training set to train the multi-task defect detection model completed in the intermediate state training, adjust the weight information of the visible light defect detection head, and obtain the multi-task defect detection model completed in the final state training;

[0029] The visible light photovoltaic test set and the thermal imaging photovoltaic test set are used to screen the weight information of the multi-task defect detection model completed in the final state training. The weight information with the highest model detection accuracy is used as the weight information of the multi-task defect detection model to obtain the multi-task defect detection model that has been finally trained.

[0030] In a possible implementation manner, the step of collecting a thermal imaging image and a visible light image to be detected in a target area where photovoltaic components are arranged includes:

[0031] Using an inspection robot to scan an image of a target area where photovoltaic modules are set to obtain video data; wherein the inspection robot moves based on a positioning and navigation algorithm;

[0032] A thermal imaging image and a visible light image are extracted from the video data.

[0033] In a possible implementation, the step of using the first mask image to cut out an image of a region where the photovoltaic assembly is located from the thermal imaging image to obtain a thermal imaging photovoltaic assembly image includes:

[0034] Expanding the boundary of the first mask image to obtain a first clipping area;

[0035] Clipping the thermal imaging image according to the first clipping area to obtain a thermal imaging photovoltaic assembly image;

[0036] The method of using the second mask image to cut out the image of the region where the photovoltaic assembly is located from the visible light image to obtain the visible light photovoltaic assembly image includes:

[0037] Extending the boundary of the second mask image to obtain a second clipping area;

[0038] The visible light image is clipped according to the second clipping area to obtain a visible light photovoltaic component image.

[0039] According to another aspect of the present disclosure, there is provided a photovoltaic module defect detection device for a power station, comprising:

[0040] An image acquisition and processing module, used for acquiring thermal imaging images and visible light images to be detected in a target area where photovoltaic modules are arranged;

[0041] A photovoltaic component segmentation module is used to segment the thermal imaging image and the visible light image using a trained photovoltaic segmentation model to obtain a first mask image of the region where the photovoltaic components are located in the thermal imaging image and a second mask image of the region where the photovoltaic components are located in the visible light image; using the first mask image to crop the image of the region where the photovoltaic components are located from the thermal imaging image to obtain a thermal imaging photovoltaic component image; using the second mask image to crop the image of the region where the photovoltaic components are located from the visible light image to obtain a visible light photovoltaic component image;

[0042] A photovoltaic defect detection module, used to process the thermal imaging photovoltaic module image and the visible light photovoltaic module image respectively using a trained multi-task defect detection model to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results at least include visible light defect detection results; the temperature-sensitive defect detection results at least include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results at least include the center point position, height and width information, confidence, and defect type of visible light type defects; the temperature-sensitive defect detection results and temperature-insensitive defect detection results with confidence levels higher than a set threshold are used as target defect detection results;

[0043] The photovoltaic segmentation model includes an image coding network and a lightweight decoder in sequence; the lightweight decoder includes a convolution layer, an LN, a convolution layer, an LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence;

[0044] The multi-task defect detection model includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; the FasterNet model main structure includes the first FasterNet Block, the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block after the FasterNet model removes Global Pool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head; the PANet structure includes a convolution layer, a network layer corresponding to the upsampling operation, and a network layer corresponding to the splicing operation; the loss function of the multi-task defect detection model includes: a category prediction loss function, a bounding box regression loss function and a confidence loss function.

[0045] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0046] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0047] The power station photovoltaic module defect detection method disclosed in the present invention is different from other photovoltaic module defect detection algorithms based on deep learning. It not only uses the information of visible light images, but also uses the temperature information of thermal imaging images. This enables the present invention to detect not only temperature-sensitive photovoltaic defects, such as hot spots, but also temperature-insensitive photovoltaic defects, such as EVA shedding, foreign body occlusion, cracks, yellowing, bubbles, etc. Such a comprehensive defect detection method can detect photovoltaic defects more comprehensively and improve detection stability.

[0048] Based on the purpose of photovoltaic component segmentation, the present invention improves the lightweight design of the decoding module of the SAM model to obtain a lightweight decoder. The lightweight decoder can be fine-tuned using a small amount of visible light sample data set and thermal imaging sample data set to obtain a photovoltaic segmentation model; the photovoltaic segmentation model can segment photovoltaic components in visible light images and thermal imaging images, and has excellent segmentation performance, with an mIoU of 97%.

[0049] After using a deep learning algorithm to segment visible light photovoltaic component images or thermal imaging photovoltaic component images, the present invention uses a multi-task defect detection model obtained by improving the FasterNet model to realize the detection of photovoltaic defects in thermal imaging photovoltaic component images and visible light photovoltaic component images. Different from using two separate deep learning models to complete defect detection of visible light photovoltaic component images and thermal imaging photovoltaic component images, the method of the present invention can ensure excellent detection accuracy while achieving a small number of parameters and a near real-time detection speed.

[0050] Compared with the method of detecting photovoltaic defects based on EL images with higher detection accuracy, the present invention does not need to apply voltage to the photovoltaic module, and can perform defect detection when the photovoltaic module is in normal working condition, thereby not affecting the service life of the photovoltaic module. The defect detection rate of the present invention can reach 96%, and various types of photovoltaic module defects can be effectively detected.

[0051] In short, by calling the algorithm disclosed in the present invention, it is possible to identify defect anomalies on photovoltaic modules in visible light images and thermal imaging images captured by the camera, and return the location information of defective photovoltaic modules in real time, so that power station staff can maintain defective photovoltaic modules in time, ensure the safe and stable operation of the power station, and extend the service life of photovoltaic modules.

[0052] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0054] Figure 1 is a flow chart of a method for detecting defects in photovoltaic modules of a power station disclosed herein;

[0055] Figure 2A is a schematic diagram of the structure of a photovoltaic segmentation model according to the present disclosure;

[0056] Figure 2B is a schematic diagram of the structure of a SAM according to the present disclosure;

[0057] Figure 2C is a schematic diagram of the structure of a lightweight decoder according to the present disclosure;

[0058] Figure 3A is a schematic diagram of the structure of FasterNet according to the present disclosure;

[0059] Figure 3B is a schematic diagram of the structure of a multi-task defect detection model according to the present disclosure;

[0060] Figure 3C is a schematic diagram of the structure of PANet according to the present disclosure;

[0061] Figure 4 is a schematic diagram of the structure of a system for detecting defects in photovoltaic modules of a power station according to an embodiment of the present disclosure;

[0062] Figure 5 It is a schematic diagram of the structure of a photovoltaic module defect detection device for a power station according to the present disclosure;

[0063] Figure 6 is a schematic structural diagram of an electronic device according to the present disclosure. DETAILED DESCRIPTION

[0064] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0065] In view of the defects of current abnormality detection of photovoltaic components, such as low accuracy, poor robustness, susceptibility to environmental factors, and low detection efficiency, the present invention proposes a method and device for detecting defects in photovoltaic components of a power station. By calling the algorithm of the present invention, it is possible to identify defect anomalies on photovoltaic components in visible light images and thermal imaging images captured by a camera, and return the location information of defective photovoltaic components in real time, effectively solving at least one of the above defects, and enabling power station staff to promptly maintain defective photovoltaic components, thereby ensuring safe and stable operation of the power station and extending the service life of photovoltaic components.

[0066] The technical solution of the present disclosure is described below through specific embodiments.

[0067] like Figure 1 FIG. 1 is a flowchart of a method for detecting defects of photovoltaic modules in a power plant according to an embodiment of the present invention. The execution subject of the present invention is a computing device or component with data processing capability. Specifically, the method of the present invention may include the following steps:

[0068] S110 , collecting thermal imaging images and visible light images to be detected in a target area where photovoltaic modules are installed.

[0069] Specifically, the surrounding environment in the target area can be scanned by a patrol robot. The patrol robot reaches the target point based on the positioning and navigation algorithm, collects video data, and then extracts the thermal imaging image and visible light image in the video frame. The thermal imaging image and visible light image are input into the photovoltaic segmentation model for detection and identification.

[0070] S120, photovoltaic component segmentation step: use the trained photovoltaic segmentation model to segment the thermal imaging image and the visible light image to obtain a first mask image of the area where the photovoltaic components are located in the thermal imaging image and a second mask image of the area where the photovoltaic components are located in the visible light image; use the first mask image to crop the image of the area where the photovoltaic components are located from the thermal imaging image to obtain a thermal imaging photovoltaic component image; use the second mask image to crop the image of the area where the photovoltaic components are located from the visible light image to obtain a visible light photovoltaic component image.

[0071] like Figure 2A As shown, the photovoltaic segmentation model includes an image encoding network Image Encoder and a simple decoder (simpledecoder) in sequence. The photovoltaic segmentation model is denoted as PVS (Photovoltaic segmentation), the input is a thermal imaging image and a visible light image, and the output is a photovoltaic panel position mask map, namely the first mask map and the second mask map mentioned above.

[0072] The above image encoding network is selected from the image encoding network (ImageEncoder) of SAM (Segment Anything Model). SAM is a model that can perform interactive segmentation and automatic segmentation, such as Figure 2B As shown in the figure, it consists of three parts: image encoding network (Image Encoder), prompt encoding network (Prompt Encoder) and mask decoding network (MaskDecoder). Image is the image input to the model, valid masks are the mask map output by the model and the score corresponding to the mask map. The image encoding network (Image Encoder) of this model uses the VisionTransformer (ViT) structure and has extremely excellent feature extraction capabilities. PVS uses the image encoding network of the SAM model as the backbone network to extract the features of the input image (1024×1024×3), and the weight information of the image encoding network remains unchanged, that is, when training the photovoltaic segmentation model, the weight information of the image encoding network is frozen, which ensures that the PVS model can inherit the excellent feature extraction capabilities of SAM.

[0073] PVS adds a lightweight decoder (simpledecoder) behind the image coding network, through which the recognition and segmentation of photovoltaic components in thermal imaging images or visible light images are realized. The process of the original SAM algorithm requires the input of some location information prompts of the photovoltaic components into the decoded information in order to segment all the photovoltaic components in the input image, which is extremely labor-intensive in industrial applications; if the original SAM is used to directly and automatically segment all the objects that can be segmented in the image, and then the required photovoltaic component areas are screened out, the errors in the fully automatic segmentation and screening steps will reduce the accuracy of the final photovoltaic segmentation results. The PVS model disclosed in the present invention does not require manual input prompts in the task of photovoltaic segmentation, and directly outputs the segmented photovoltaic area quickly end-to-end.

[0074] The lightweight decoder includes a convolution layer, LN, a convolution layer, LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence. Specifically, Figure 2CAs shown in the figure, the lightweight decoder (simpledecoder) consists of a 1×1 convolution conv and LayerNorm (LN), a 3×3 convolution conv and LayerNorm (LN), two 4x up-sampled transposed convolutions and GELU activation functions, and a 1×1 convolution conv. After the image coding network extracts the image coding information, the image coding information is decoded and output by the lightweight decoder (simpledecoder) to obtain a photovoltaic position mask map of 1024×1024×3 scale.

[0075] The above method of using the first mask image to cut out the image of the area where the photovoltaic module is located from the thermal imaging image to obtain a thermal imaging photovoltaic module image can be specifically implemented by the following steps:

[0076] The first mask image is extended at the boundary to obtain a first clipping area; the thermal imaging image is clipped according to the first clipping area to obtain a thermal imaging photovoltaic module image. The purpose of the boundary extension here is to retain the adjacent background outside the area where the photovoltaic module is located, for example, to expand the background of 3 pixels outward on the basis of the first mask image to ensure the integrity of the boundary of the area where the photovoltaic module is located.

[0077] The above-mentioned method of using the second mask image to cut out the image of the region where the photovoltaic assembly is located from the visible light image to obtain the visible light photovoltaic assembly image can be specifically implemented by the following steps:

[0078] The second mask image is extended at the boundary to obtain a second clipping area; the visible light image is clipped according to the second clipping area to obtain a visible light photovoltaic component image. The purpose of the boundary extension here is to retain the adjacent background outside the area where the photovoltaic component is located, for example, the background of 3 pixels is extended outward on the basis of the first mask image to ensure the integrity of the boundary of the area where the photovoltaic component is located.

[0079] S130, photovoltaic defect detection step: use the trained multi-task defect detection model to process the thermal imaging photovoltaic component image and the visible light photovoltaic component image respectively to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results include visible light defect detection results; the temperature-sensitive defect detection results include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results include the center point position, height and width information, confidence, and defect type of visible light type defects.

[0080] The multi-task defect detection model is denoted as Mutilhead-FasterNet, which is designed based on the FasterNet model. Figure 3AFigure 1 is a schematic diagram of the structure of FasterNet. FasterNet is a model built based on partial convolution (PConv), which can reduce memory access and computational redundancy and has excellent recognition accuracy. Among them, the Embedding layer is the embedding layer, c1, c2, c3, c4 represent the number of channels of the feature map; h, w represent the height and width of the feature map; l1, l2, l3, l4 represent the number of stacking times of FasterNet Block; k represents the height and width of the convolution kernel of the partial convolution operation; identity represents "itself", which means no transformation; cp represents the number of channels for partial convolution operation, for example, the number of input channels is c (cp is generally 1 / 4 of c), the cp channel part applies partial convolution operation, and the remaining part (c-cp) does not perform any operation (ie Identity); Filters represents the number of convolution kernels for partial convolution operation; Global Pool is the global pooling layer, Merging is the merging layer, Conv is the convolutional neural network layer, FC is the fully connected layer, and PConv is the partial convolution layer, such as Figure 3A As shown, FasterNetBlock includes PConv 3×3, Conv1×1, and Conv1×1 connected in sequence.

[0081] This paper removes the last three structures of the FasterNet model, Global Pool, Conv, and FC, and uses the remaining parts as the backbone network of the multi-task defect detection model Mutilhead-FasterNet model to extract the feature information of the input image data. On this basis, branches are added after the outputs of Stage2, Stage3, and Stage4, and the PANet structure is used as the neck network to fuse the feature information extracted by branches at different levels. This part of the design can effectively integrate the semantic information of different levels of the backbone network and improve its performance in target detection tasks. In addition, two task heads (visible light defect detection head and thermal imaging defect detection head) are used to detect all categories of defects in visible light photovoltaic module images and hot spot defects in thermal imaging photovoltaic module images.

[0082] In summary, if Figure 3BThe multi-task defect detection model shown includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; wherein the FasterNet model main structure includes the first FasterNet Block, the second FasterNetBlock, the third FasterNetBlock and the fourth FasterNet Block after the FasterNet model removes the Global Pool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head.

[0083] like Figure 3C As shown, the PANet structure includes a convolutional layer, a network layer corresponding to an upsampling operation, a network layer corresponding to a splicing operation, etc.; the loss function of the multi-task defect detection model includes: a category prediction loss function (binary cross entropy loss), a bounding box regression loss function (IoU loss), and a confidence loss function (binary cross entropy loss).

[0084] Specifically, Figure 3C The quadrilaterals in the figure represent intermediate feature maps. They have different numbers of channels, heights, and widths, and have no special meaning. The downward arrow represents a "downsampling operation", the upward arrow represents an "upsampling operation", and the intersection of two arrows represents a "concatenation operation". The only meaning expressed by this picture is the fusion of feature information, and each intermediate feature map has no special meaning.

[0085] S140, taking the temperature-sensitive defect detection results and the temperature-insensitive defect detection results with confidence levels higher than a set threshold as target defect detection results.

[0086] The threshold here is set according to the actual scene, for example, it is set to 0.75.

[0087] After determining the target defect detection result, the defect type and the location information of the PV module corresponding to the defect are returned according to the location information of the inspection robot, so that subsequent maintenance personnel can locate the defective PV module and perform relevant maintenance processing.

[0088] The technical solution designed in the present disclosure is suitable for intelligently capturing video data of photovoltaic power stations through inspection robots, then extracting visible light images and thermal imaging images, and then using deep learning technology based on visible light images and thermal imaging images to quickly detect defects and return the defect type and photovoltaic component location.

[0089] In some embodiments, the step of generating a data set for training the photovoltaic segmentation model is further included:

[0090] Step 1: Collect multiple visible light sample images to obtain a visible light original image set; collect multiple thermal imaging sample images to obtain a thermal imaging original image set.

[0091] In weather with good sunlight, the inspection robot equipped with visible light cameras and thermal imaging cameras captures and collects video data of the photovoltaic components of the power station, and then extracts frames from the collected video data to obtain a set of visible light original images and a set of thermal imaging original images.

[0092] Step 2: For each sample image in the visible light original image set, annotate the visible light defect area to obtain a first visible light annotated data set. For each sample image in the visible light original image set, annotate the photovoltaic module area to obtain a second visible light annotated data set. The visible light original image set and the second visible light annotated data set are used as the original visible light segmentation data set; the original visible light segmentation data set is divided into a visible light segmentation training set and a visible light segmentation test set.

[0093] Specifically, the open source tool labelme was used to annotate the EVA shedding, foreign body occlusion, cracks, yellowing, bubble areas, etc. on the photovoltaic modules to obtain the first visible light annotation dataset. The open source tool labelme was used to annotate the photovoltaic module area (the collected video not only captured the photovoltaic modules, but also other backgrounds) to obtain the second visible light annotation dataset.

[0094] Step 3: For each sample image in the original thermal imaging image set, perform hot spot area annotation to obtain a first thermal imaging annotation data set; for each sample image in the original thermal imaging image set, perform photovoltaic module area annotation to obtain a second thermal imaging annotation data set. The original thermal imaging image set and the second thermal imaging annotation data set are used as the original thermal imaging segmentation data set; the original thermal imaging segmentation data set is divided into a thermal imaging segmentation training set and a thermal imaging segmentation test set.

[0095] Specifically, the open source tool labelme was used to annotate the hot spot area on the photovoltaic module to obtain the first thermal imaging annotation dataset. The open source tool labelme was used to annotate the photovoltaic module area (the collected video not only captured the photovoltaic module, but also other backgrounds) to obtain the second thermal imaging annotation dataset.

[0096] Step 4: Perform online random data enhancement processing on the visible light segmentation training set to obtain a target visible light segmentation training set; and perform online random data enhancement processing on the thermal imaging segmentation training set to obtain a target thermal imaging segmentation training set; use the target visible light segmentation training set, the target thermal imaging segmentation training set, the visible light segmentation test set and the thermal imaging segmentation test set as data sets for training the photovoltaic segmentation model.

[0097] Online random data enhancement processing can adopt random cropping, random increase of image noise, random Gaussian blur, etc. to enhance the robustness of the photovoltaic segmentation model.

[0098] After generating the data set used to train the photovoltaic segmentation model, the data set is used to train the photovoltaic segmentation model, and Focal Loss is used as the loss function during the training process. Compared with the BinaryCross Entropy loss function, the Focal Loss loss function can better distinguish difficult-to-identify samples and converge faster. Focal Loss calculation formula:

[0099] L fl =-(1-p t ) γ log(p t )

[0100]

[0101] Among them, the values ​​of y are 1 and 0, representing the foreground and background respectively; the value range of p is 0 to 1, which is the probability of predicting that it belongs to the foreground (i.e., the photovoltaic module); p t It reflects the closeness between the predicted result p and the category y, p t The larger the value, the closer p is to category y, that is, the more accurate the classification is; γ>0 is the adjustment factor,

[0102] (1-p t ) γ It can reduce the loss contribution of easy-to-classify samples, thereby increasing the loss proportion of hard-to-classify samples. For example, when p t When it approaches 1, it means that the sample is easy to distinguish. t ) γ It tends to 0, indicating that the contribution to the loss is small, that is, the loss ratio of easy-to-distinguish samples is reduced. t When it approaches 0, it means that the sample is difficult to distinguish. At this time (1-p t ) γ It tends to 1, indicating that the contribution to the loss is large, that is, the loss ratio of difficult-to-distinguish samples is increased.

[0103] In the process of training the frozen photovoltaic segmentation model, the weight information of the image encoder is combined to fine-tune the weight information of the lightweight decoder simpledecoder. After 100 rounds of iteration, the weight information with the best performance on the visible light segmentation test set and the thermal imaging segmentation test set is selected and saved to obtain the trained photovoltaic segmentation model.

[0104] In some embodiments, the step of generating a data set for training the multi-task defect detection model is further included:

[0105] Step 1: For each visible light sample image in the original visible light segmentation data set, according to the annotation information corresponding to the visible light sample image in the second visible light annotation data set, the image of the area where the photovoltaic component is located is cropped from the visible light sample image, and the annotation information corresponding to the visible light sample image in the first visible light annotation data set is updated to the cropped image to obtain a visible light photovoltaic labeled image.

[0106] Step 2: copy at least one type of visible light defect image area in all visible light photovoltaic annotated images at least once, and randomly paste the copied image area into a visible light photovoltaic annotated image to obtain an original visible light photovoltaic data set; divide the original visible light photovoltaic data set into a visible light photovoltaic training set and a visible light photovoltaic test set.

[0107] Step three: for each thermal imaging sample image in the original thermal imaging segmentation data set, according to the annotation information corresponding to the thermal imaging sample image in the second thermal imaging annotation data set, the image of the area where the photovoltaic component is located is cropped from the thermal imaging sample image, and the annotation information corresponding to the thermal imaging sample image in the first thermal imaging annotation data set is updated to the cropped image to obtain a thermal imaging photovoltaic annotation image.

[0108] Step 4: copy at least one image area of ​​a hot spot defect in all thermal imaging photovoltaic annotated images at least once, and randomly paste the copied image area into a thermal imaging photovoltaic annotated image to obtain an original thermal imaging photovoltaic data set; divide the original thermal imaging photovoltaic data set into a thermal imaging photovoltaic training set and a thermal imaging photovoltaic test set.

[0109] Step 5: Use the visible light photovoltaic training set, the visible light photovoltaic test set, the thermal imaging photovoltaic training set and the thermal imaging photovoltaic test set as data sets for training the multi-task defect detection model.

[0110] After obtaining the data set used for training the multi-task defect detection model, the multi-task defect detection model is trained using the data set, and the steps are as follows:

[0111] Step 1: Use the visible light photovoltaic training set and the thermal imaging photovoltaic training set to train the multi-task defect detection model to be trained, and obtain the multi-task defect detection model with initial training.

[0112] The purpose of this step is to train a backbone + neck network (i.e., FasterNet model main structure + PANet) that is suitable for both thermal imaging photovoltaic defect detection tasks and visible light photovoltaic defect detection tasks.

[0113] Step 2: Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the visible light defect detection head, use the thermal imaging photovoltaic training set to train the multi-task defect detection model completed in the initial state training, adjust the weight information of the thermal imaging defect detection head, and obtain the multi-task defect detection model completed in the intermediate state training.

[0114] The purpose of this step is to train a thermal imaging defect detection head with high detection accuracy based on the backbone + neck network.

[0115] Step 3: Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the thermal imaging defect detection head, use the visible light photovoltaic training set to train the multi-task defect detection model completed in the intermediate state training, adjust the weight information of the visible light defect detection head, and obtain the multi-task defect detection model completed in the final state training.

[0116] The purpose of this step is to train a visible light defect detection head with high detection accuracy based on the backbone + neck network.

[0117] Step 4: Use the visible light photovoltaic test set and the thermal imaging photovoltaic test set to screen the weight information of the multi-task defect detection model completed in the final state training, and use the weight information with the highest model detection accuracy as the weight information of the multi-task defect detection model to obtain the multi-task defect detection model that has been finally trained.

[0118] During the above training process, the stochastic gradient descent method is used to continuously adjust the weight information of the multi-task defect detection model, so that the accuracy of thermal imaging defect detection and visible light defect detection results is gradually improved.

[0119] like Figure 4As shown, in the technical solution of the present disclosure, the inspection robot is equipped with a visible light camera and a thermal imaging camera. When working, it inspects row by row along the arrangement direction of the photovoltaic components, collects videos, and separates thermal imaging images and visible light images from the collected videos; then, the photovoltaic segmentation model is used to process the thermal imaging image and the visible light image, and according to the mask image obtained by the processing, the thermal imaging photovoltaic component image is cropped from the thermal imaging image, and the visible light photovoltaic component image is cropped from the visible light image; then, the visible light photovoltaic component image and the thermal imaging photovoltaic component image are input into the multi-task defect detection model, and the input image is processed by the multi-task defect detection model to obtain the abnormal information of the photovoltaic component, where the abnormal information of the photovoltaic component at least includes the detection results of temperature-insensitive defects and / or the detection results of temperature-sensitive defects. In addition, the present disclosure records and returns the location information of defective photovoltaic components in real time, so that the power station staff can timely maintain the defective photovoltaic components, ensure the safe and stable operation of the power station, and extend the service life of the photovoltaic components.

[0120] Before inspecting the image to be inspected, it is necessary to collect samples of visible light images and thermal imaging images and create a data set for training PVS. At the same time, it is necessary to create a data set for training the multi-task defect detection model. Figure 4 The process in the dotted line and dotted box is step 0 of the entire algorithm process. After the model is trained, there will be no dotted process when the inspection robot performs inspection tasks.

[0121] Based on the same inventive concept, the present disclosure provides a photovoltaic module defect detection device for a power station, and the steps performed by the components of the device are the same or similar to the above method, so similar parts are not repeated. Figure 5 As shown, the photovoltaic module defect detection device for a power station of this embodiment includes:

[0122] The image acquisition and processing module 510 is used to acquire thermal imaging images and visible light images to be detected in the target area where the photovoltaic components are set.

[0123] The photovoltaic component segmentation module 520 is used to use the trained photovoltaic segmentation model to segment the thermal imaging image and the visible light image to obtain a first mask image of the area where the photovoltaic components are located in the thermal imaging image and a second mask image of the area where the photovoltaic components are located in the visible light image; use the first mask image to crop the image of the area where the photovoltaic components are located from the thermal imaging image to obtain a thermal imaging photovoltaic component image; use the second mask image to crop the image of the area where the photovoltaic components are located from the visible light image to obtain a visible light photovoltaic component image.

[0124] The photovoltaic defect detection module 530 is used to use the trained multi-task defect detection model to process the thermal imaging photovoltaic module image and the visible light photovoltaic module image respectively to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results at least include visible light defect detection results; the temperature-sensitive defect detection results at least include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results at least include the center point position, height and width information, confidence, and defect type of visible light type defects; the temperature-sensitive defect detection results and temperature-insensitive defect detection results with confidence levels higher than a set threshold are used as target defect detection results.

[0125] Among them, the photovoltaic segmentation model includes an image coding network and a lightweight decoder in sequence; the lightweight decoder includes a convolution layer, LN, a convolution layer, LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence.

[0126] The multi-task defect detection model includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; the FasterNet model main structure includes the first FasterNet Block, the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block after the FasterNet model removes Global Pool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head; the PANet structure includes a convolution layer, a network layer corresponding to the upsampling operation, and a network layer corresponding to the splicing operation; the loss function of the multi-task defect detection model includes: a category prediction loss function, a bounding box regression loss function and a confidence loss function.

[0127] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a computer-readable storage medium.

[0128] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0129] like Figure 6 As shown, the device 600 includes a computing unit 610, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 620 or a computer program loaded from a storage unit 680 into a random access memory (RAM) 630. In the RAM 630, various programs and data required for the operation of the device 600 can also be stored. The computing unit 610, the ROM 620, and the RAM 630 are connected to each other via a bus 640. An input / output (I / O) interface 650 is also connected to the bus 640.

[0130] A number of components in the device 600 are connected to the I / O interface 650, including: an input unit 660, such as a keyboard, a mouse, etc.; an output unit 670, such as various types of displays, speakers, etc.; a storage unit 680, such as a disk, an optical disk, etc.; and a communication unit 690, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 690 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0131] The computing unit 610 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 610 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 610 performs the various methods and processes described above. For example, in some embodiments, any of the above methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 680. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 620 and / or a communication unit 690. When the computer program is loaded into RAM 630 and executed by the computing unit 610, one or more steps of any of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 610 may be configured to perform any of the methods described above in any other appropriate manner (e.g., by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0134] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0137] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0138] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0139] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting defects in photovoltaic modules of a power station, characterized in that: include: Collect thermal imaging images and visible light images to be detected in the target area where the photovoltaic components are set; The thermal imaging image and the visible light image are segmented using the trained photovoltaic segmentation model to obtain a first mask image of the area where the photovoltaic components are located in the thermal imaging image and a second mask image of the area where the photovoltaic components are located in the visible light image; the image of the area where the photovoltaic components are located is cropped from the thermal imaging image using the first mask image to obtain a thermal imaging photovoltaic component image; the image of the area where the photovoltaic components are located is cropped from the visible light image using the second mask image to obtain a visible light photovoltaic component image; The thermal imaging photovoltaic module image and the visible light photovoltaic module image are processed respectively by using the trained multi-task defect detection model to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results at least include visible light defect detection results; the temperature-sensitive defect detection results at least include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results at least include the center point position, height and width information, confidence, and defect type of visible light type defects; The temperature-sensitive defect detection results and the temperature-insensitive defect detection results with confidence levels higher than a set threshold are used as target defect detection results; The photovoltaic segmentation model includes an image coding network and a lightweight decoder in sequence; the lightweight decoder includes a convolution layer, an LN, a convolution layer, an LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence; The multi-task defect detection model includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; the FasterNet model main structure includes the first FasterNet Block, the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block after the FasterNet model removes GlobalPool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNetBlock and the fourth FasterNetBlock are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head; the PANet structure includes a convolution layer, a network layer corresponding to the upsampling operation, and a network layer corresponding to the splicing operation; the loss function of the multi-task defect detection model includes: a category prediction loss function, a bounding box regression loss function and a confidence loss function.

2. The method according to claim 1, characterized in that The image coding network is an image coding network in a Segment Anything Model network, and when the photovoltaic segmentation model is trained, weight information of the image coding network is frozen.

3. The method according to claim 1, characterized in that The step of generating a data set for training the photovoltaic segmentation model is also included: Collect multiple visible light sample images to obtain a visible light original image set; collect multiple thermal imaging sample images to obtain a thermal imaging original image set; For each sample image in the visible light original image set, a visible light defect area is annotated to obtain a first visible light annotated data set; for each sample image in the visible light original image set, a photovoltaic module area is annotated to obtain a second visible light annotated data set; the visible light original image set and the second visible light annotated data set are used as original visible light segmentation data sets; the original visible light segmentation data set is divided into a visible light segmentation training set and a visible light segmentation test set; For each sample image in the thermal imaging original image set, hot spot area annotation is performed to obtain a first thermal imaging annotation data set; For each sample image in the thermal imaging original image set, the photovoltaic module area is annotated to obtain a second thermal imaging annotated data set; the thermal imaging original image set and the second thermal imaging annotated data set are used as the original thermal imaging segmentation data set; the original thermal imaging segmentation data set is divided into a thermal imaging segmentation training set and a thermal imaging segmentation test set; Performing online random data enhancement processing on the visible light segmentation training set to obtain a target visible light segmentation training set; and performing online random data enhancement processing on the thermal imaging segmentation training set to obtain a target thermal imaging segmentation training set; The target visible light segmentation training set, the target thermal imaging segmentation training set, the visible light segmentation test set and the thermal imaging segmentation test set are used as data sets for training the photovoltaic segmentation model.

4. The method according to claim 3, characterized in that The step of generating a data set for training the multi-task defect detection model is also included: For each visible light sample image in the original visible light segmentation data set, according to the annotation information corresponding to the visible light sample image in the second visible light annotation data set, an image of the area where the photovoltaic component is located is cropped from the visible light sample image, and the annotation information corresponding to the visible light sample image in the first visible light annotation data set is updated to the cropped image to obtain a visible light photovoltaic annotated image; Copying at least once the image area of ​​at least one type of visible light defect in all visible light photovoltaic annotated images, and randomly pasting the copied image area into a visible light photovoltaic annotated image to obtain an original visible light photovoltaic data set; dividing the original visible light photovoltaic data set into a visible light photovoltaic training set and a visible light photovoltaic test set; For each thermal imaging sample image in the original thermal imaging segmentation data set, according to the annotation information corresponding to the thermal imaging sample image in the second thermal imaging annotation data set, an image of the area where the photovoltaic component is located is cropped from the thermal imaging sample image, and the annotation information corresponding to the thermal imaging sample image in the first thermal imaging annotation data set is updated to the cropped image to obtain a thermal imaging photovoltaic annotation image; Copying at least one image area of ​​a hot spot defect in all thermal imaging photovoltaic annotated images at least once, and randomly pasting the copied image area into a thermal imaging photovoltaic annotated image to obtain an original thermal imaging photovoltaic data set; dividing the original thermal imaging photovoltaic data set into a thermal imaging photovoltaic training set and a thermal imaging photovoltaic test set; The visible light photovoltaic training set, the visible light photovoltaic test set, the thermal imaging photovoltaic training set and the thermal imaging photovoltaic test set are used as data sets for training the multi-task defect detection model.

5. The method according to claim 4, characterized in that Also included is the step of training the multi-task defect detection model: Using the visible light photovoltaic training set and the thermal imaging photovoltaic training set, the multi-task defect detection model to be trained is trained to obtain the multi-task defect detection model with initial training completed; Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the visible light defect detection head, use the thermal imaging photovoltaic training set to train the multi-task defect detection model completed in the initial state training, adjust the weight information of the thermal imaging defect detection head, and obtain the multi-task defect detection model completed in the intermediate state training; Freeze the main structure of the FasterNet model, the PANet structure, and the weight information of the thermal imaging defect detection head, use the visible light photovoltaic training set to train the multi-task defect detection model completed in the intermediate state training, adjust the weight information of the visible light defect detection head, and obtain the multi-task defect detection model completed in the final state training; The visible light photovoltaic test set and the thermal imaging photovoltaic test set are used to screen the weight information of the multi-task defect detection model completed in the final state training. The weight information with the highest model detection accuracy is used as the weight information of the multi-task defect detection model to obtain the multi-task defect detection model that has been finally trained.

6. The method according to claim 1, characterized in that The method of collecting the thermal imaging image and the visible light image to be detected in the target area where the photovoltaic components are set includes: Using an inspection robot to scan an image of a target area where photovoltaic modules are set to obtain video data; wherein the inspection robot moves based on a positioning and navigation algorithm; A thermal imaging image and a visible light image are extracted from the video data.

7. The method according to claim 1, characterized in that The method of using the first mask image to cut out the image of the area where the photovoltaic assembly is located from the thermal imaging image to obtain a thermal imaging photovoltaic assembly image includes: Expanding the boundary of the first mask image to obtain a first clipping area; Clipping the thermal imaging image according to the first clipping area to obtain a thermal imaging photovoltaic assembly image; The method of using the second mask image to cut out the image of the region where the photovoltaic assembly is located from the visible light image to obtain the visible light photovoltaic assembly image includes: Extending the boundary of the second mask image to obtain a second clipping area; The visible light image is clipped according to the second clipping area to obtain a visible light photovoltaic component image.

8. A photovoltaic module defect detection device for a power station, characterized in that: include: An image acquisition and processing module, used for acquiring thermal imaging images and visible light images to be detected in a target area where photovoltaic modules are arranged; A photovoltaic component segmentation module is used to segment the thermal imaging image and the visible light image using a trained photovoltaic segmentation model to obtain a first mask image of the region where the photovoltaic components are located in the thermal imaging image and a second mask image of the region where the photovoltaic components are located in the visible light image; using the first mask image to crop the image of the region where the photovoltaic components are located from the thermal imaging image to obtain a thermal imaging photovoltaic component image; using the second mask image to crop the image of the region where the photovoltaic components are located from the visible light image to obtain a visible light photovoltaic component image; A photovoltaic defect detection module, used to process the thermal imaging photovoltaic module image and the visible light photovoltaic module image respectively using a trained multi-task defect detection model to obtain temperature-sensitive defect detection results and temperature-insensitive defect detection results; wherein the temperature-insensitive defect detection results at least include visible light defect detection results; the temperature-sensitive defect detection results at least include the hot spot center point position, hot spot height and width information, and confidence; the visible light defect detection results at least include the center point position, height and width information, confidence, and defect type of visible light type defects; the temperature-sensitive defect detection results and temperature-insensitive defect detection results with confidence levels higher than a set threshold are used as target defect detection results; The photovoltaic segmentation model includes an image coding network and a lightweight decoder in sequence; the lightweight decoder includes a convolution layer, an LN, a convolution layer, an LN, a multiple upsampling transposed convolution layer and a GELU activation function, a multiple upsampling transposed convolution layer and a GELU activation function, and a convolution layer connected in sequence; The multi-task defect detection model includes: a FasterNet model main structure, a PANet structure, a visible light defect detection head and a thermal imaging defect detection head; the FasterNet model main structure includes the first FasterNet Block, the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block after the FasterNet model removes Global Pool, Conv and FC, and the outputs of the second FasterNet Block, the third FasterNet Block and the fourth FasterNet Block are all connected to the PANet structure; the output of the PANet structure is connected to the visible light defect detection head and the thermal imaging defect detection head; the PANet structure includes a convolution layer, a network layer corresponding to the upsampling operation, and a network layer corresponding to the splicing operation; the loss function of the multi-task defect detection model includes: a category prediction loss function, a bounding box regression loss function and a confidence loss function.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images