Photovoltaic module defect diagnosis method, device and equipment

Abstract

The invention discloses a photovoltaic module defect diagnosis method, device and equipment, and the method comprises the steps: carrying out the slicing of an aerial image, and obtaining a slice image; the aerial image is acquired by aerial photography of the target photovoltaic power station by the unmanned aerial vehicle at a high altitude not lower than 60 meters; using a pre-trained first deep learning model to identify the photovoltaic modules in the slice images, and marking the photovoltaic modules in each slice image; splicing the slice images to obtain a photovoltaic module image with a photovoltaic mark; according to a photovoltaic mark in the photovoltaic module image, a photovoltaic imaging area is cut in the photovoltaic module image, and a photovoltaic area image is obtained; and performing defect diagnosis on the photovoltaic region image by using a pre-trained second deep learning model to obtain a defect diagnosis result. According to the technical scheme, the limitation of an infrared inspection technology can be broken through, and the accuracy and reliability of photovoltaic defect diagnosis in photovoltaic power station inspection are ensured.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic inspection technology, and in particular to a photovoltaic module defect diagnosis method, device and equipment. Background Art

[0002] The long-term stable operation of large-scale photovoltaic power stations depends on the reliability and stability of photovoltaic power generation units. With the large-scale and rapid expansion of photovoltaic power station infrastructure, its subsequent operation and maintenance market is also expanding. There is an urgent need for an efficient and reliable operation and maintenance method to timely and accurately discover and diagnose defects in photovoltaic components to ensure the safe and reliable operation of large-scale photovoltaic power stations.

[0003] Using drones with cameras to inspect photovoltaic power stations can greatly improve the efficiency of photovoltaic power station operation and maintenance and reduce the manual labor required during the operation and maintenance process. However, the current drone automatic inspection technology is mainly based on infrared image detection technology. However, due to the low resolution of thermal imagers, infrared images only contain thermal radiation information, and the high flight speed of drones, this detection scheme can only detect hot spot defects with large-scale high temperature differences. It can be seen that the current technology of using drones to automatically inspect photovoltaic modules in photovoltaic power stations has certain limitations, and there is a problem of insufficient accuracy of detection results. Summary of the invention

[0004] The purpose of the present invention is to provide a photovoltaic module defect diagnosis method, device and equipment, which can break through the limitations of infrared inspection technology and ensure the accuracy and reliability of photovoltaic defect diagnosis in photovoltaic power station inspections.

[0005] In order to solve the above technical problems, the present invention provides a photovoltaic module defect diagnosis method, comprising:

[0006] Slice the aerial image to obtain a slice image; wherein the aerial image is an image acquired by a drone at an altitude of not less than 60 meters from the target photovoltaic power station;

[0007] Using a pre-trained first deep learning model, the photovoltaic components in the slice images are identified, and the photovoltaic components in each of the slice images are marked;

[0008] splicing the slice images to obtain a photovoltaic module image with photovoltaic markings;

[0009] Determine a photovoltaic imaging region in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging region in the photovoltaic component image to obtain a photovoltaic region image;

[0010] Using the pre-trained second deep learning model, defect diagnosis is performed on the photovoltaic area image to obtain a defect diagnosis result.

[0011] In an optional embodiment of the present application, slicing the aerial image to obtain a slice image includes:

[0012] The aerial image is divided into slice images with overlapping areas by using image slicing technology; wherein the overlapping width of two adjacent slice images is not less than 300 pixels;

[0013] Using a pre-trained first deep learning model, identifying photovoltaic components in the slice images, and marking the photovoltaic components in each of the slice images, including:

[0014] Using the first deep learning model to identify and output the photovoltaic components in each of the slice images, forming a prediction frame marking each photovoltaic imaging area on the slice image, and obtaining first coordinate values ​​of four corner points of each of the prediction frames and confidences corresponding to the first coordinate values ​​of the four corner points;

[0015] The slice images are stitched together to obtain a photovoltaic module image with a photovoltaic mark, including:

[0016] Converting the first coordinate values ​​of the four corner points corresponding to each of the prediction boxes in each of the slice images into second coordinate values ​​on the aerial image;

[0017] Determine, according to the second coordinate values ​​corresponding to the prediction boxes, an area intersection-and-union ratio between any two prediction boxes;

[0018] Among the two prediction frames whose area intersection-to-union ratio is greater than a set threshold, the prediction frame whose corresponding first coordinate value has the highest confidence is retained, and the slice images are spliced ​​to obtain the photovoltaic assembly image.

[0019] In an optional embodiment of the present application, after marking the photovoltaic components in each of the slice images, before stitching the slice images to obtain the photovoltaic component image with the photovoltaic mark, the method further includes:

[0020] According to the prediction frame in the slice image, obtaining the width and height of each photovoltaic imaging area in the slice image;

[0021] According to the width and height dimensions, filter and delete the slice images that do not satisfy the filtering formula The prediction frame is used to screen out photovoltaic components that are incorrectly detected in the slice image; wherein, , , , is the width, height, area, and aspect ratio of the i-th prediction box detected in the slice image, and are respectively the average area value of all the prediction boxes in the slice image and the standard aspect ratio of the photovoltaic module, and They represent the area error coefficient and the aspect ratio error coefficient respectively.

[0022] In an optional embodiment of the present application, determining a photovoltaic imaging area in the photovoltaic component image according to a photovoltaic mark in the photovoltaic component image includes:

[0023] Converting the photovoltaic module image from the RGB color space to the HSV color space;

[0024] Expanding each prediction frame in the photovoltaic component image outward by a preset ratio;

[0025] In each of the expanded prediction frames, the photovoltaic imaging area is determined according to a set color threshold; wherein the ranges corresponding to the H threshold, the S threshold, and the V threshold in the set color threshold are (100, 124), (43, 255), and (46, 255), respectively.

[0026] In an optional embodiment of the present application, in each expanded prediction frame, determining the photovoltaic imaging area according to a set color threshold includes:

[0027] Delimiting the area where each pixel point in the prediction frame whose HSV value is within the set color threshold range is located as the initial photovoltaic imaging area;

[0028] Converting the photovoltaic assembly image into a photovoltaic assembly mask image highlighting the initial photovoltaic imaging area;

[0029] Using the image erosion formula The photovoltaic assembly mask image is corroded to obtain a photovoltaic assembly mask image after corrosion processing, so that the initial photovoltaic imaging area is corrected to the photovoltaic imaging area, wherein: and are the pixels in the mask image of the photovoltaic module Pixel values ​​before and after erosion, is the kernel size;

[0030] The photovoltaic assembly mask image is converted into a photovoltaic assembly image that determines the photovoltaic imaging area.

[0031] In an optional embodiment of the present application, after obtaining the mask image of the photovoltaic component after the corrosion treatment, the method further includes:

[0032] Performing contour detection and quadrilateral fitting on the photovoltaic imaging area in the mask image of the photovoltaic assembly after corrosion, and obtaining the coordinates of four corner points of the quadrilateral photovoltaic imaging area;

[0033] According to the coordinates of the four corner points, the quadrilateral photovoltaic imaging area is perspectively transformed to obtain a photovoltaic imaging area with a rectangular shape, and then the photovoltaic component mask image is converted into a photovoltaic component image that determines the photovoltaic imaging area.

[0034] In an optional embodiment of the present application, the process of pre-training the second deep learning model includes:

[0035] Marking each pre-collected photovoltaic image sample with a defect label, wherein the defect label includes a cell missing defect label, a weed occlusion defect label, a wire occlusion defect label, a shadow occlusion defect label, and a bird droppings occlusion defect label;

[0036] A Yolov8 deep network model is constructed, and deep learning training is performed using the photovoltaic image samples marked with defect labels as a training set to obtain the second deep learning model.

[0037] In an optional embodiment of the present application, after marking defect labels on each pre-collected photovoltaic image sample, the method further includes:

[0038] Expanding the photovoltaic image sample using a diffusion model to obtain a first expanded sample;

[0039] The photovoltaic image sample is translated, scaled, HSV space enhanced, flipped upside down, flipped left to right, rotated 90 degrees, 180 degrees, 270 degrees, or the defect object is copied and pasted, or a combination of multiple processing is performed to obtain a second expanded sample;

[0040] Accordingly, deep learning training is performed using the photovoltaic image samples marked with defect labels as a training set to obtain the second deep learning model, including:

[0041] The first expanded sample, the second expanded sample, and the photovoltaic image sample are collectively used as a training set for learning and training to obtain the second deep learning model.

[0042] A photovoltaic module defect diagnosis device, comprising:

[0043] A slicing processing module is used to perform slicing processing on the aerial image to obtain a slicing image; wherein the aerial image is an image collected by a drone at an altitude of not less than 60 meters from the target photovoltaic power station;

[0044] An image recognition module, used to identify the photovoltaic components in the slice images by using a pre-trained first deep learning model, and mark the photovoltaic components in each of the slice images;

[0045] A slice splicing module, used for splicing the slice images to obtain a photovoltaic module image with a photovoltaic mark;

[0046] An image processing module, used to determine a photovoltaic imaging area in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging area in the photovoltaic component image to obtain a photovoltaic area image;

[0047] The defect diagnosis module is used to perform defect diagnosis on the photovoltaic area image by using a pre-trained second deep learning model to obtain a defect diagnosis result.

[0048] A photovoltaic module defect diagnosis device, comprising:

[0049] Memory for storing computer programs;

[0050] A processor is used to execute the computer program to implement the steps of the photovoltaic module defect diagnosis method as described in any one of the above items.

[0051] The photovoltaic module defect diagnosis method, device and equipment provided by the present invention include: slicing the aerial image to obtain a slice image; wherein the aerial image is an image of a target photovoltaic power station acquired by a drone at an altitude of not less than 60 meters; using a pre-trained first deep learning model to identify the photovoltaic modules in the slice image, and marking the photovoltaic modules in each slice image; splicing each slice image to obtain a photovoltaic module image with a photovoltaic mark; determining the photovoltaic imaging area in the photovoltaic module image according to the photovoltaic mark in the photovoltaic module image, and cutting out the photovoltaic imaging area in the photovoltaic module image to obtain a photovoltaic area image; using a pre-trained second deep learning model to perform defect diagnosis on the photovoltaic area image to obtain a defect diagnosis result.

[0052] The aerial images in the present application are collected by a drone at an altitude of not less than 60 meters, so that each frame of aerial images collected by the drone can capture a larger area within the photovoltaic power station, thereby avoiding the problem of insufficient battery life of the drone during the process of collecting aerial images due to the large area of ​​the photovoltaic power station; on this basis, in the process of diagnosing defects of photovoltaic components based on aerial images in the present application, the photovoltaic components in the aerial images are first identified. In this process, considering that the aerial images are taken at high altitudes, the imaging area of ​​the photovoltaic components accounts for a relatively small proportion of the aerial images, and therefore the aerial images are first sliced, and the sliced ​​images are used to identify the defects of the photovoltaic components. For example, identifying photovoltaic components can improve the accuracy of photovoltaic component identification in aerial images to a certain extent; after identifying the photovoltaic components in the aerial image, the photovoltaic imaging area is separately segmented to form a photovoltaic area image, and finally the photovoltaic area image can be targeted for defect diagnosis to eliminate interference from other backgrounds. Compared with the method of directly performing photovoltaic defect diagnosis on the entire aerial image, the defect diagnosis method in the present application can effectively ensure the accuracy of photovoltaic defect diagnosis; and, the defect diagnosis in the present application is based on visible light images, and when performing defect diagnosis, it is not limited to the diagnosis and detection of hot spot defects.

[0053] It can be seen that the method of detecting photovoltaic modules in this application can break through the limitations of infrared inspection technology and ensure the accuracy and reliability of photovoltaic defect diagnosis in photovoltaic power station inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] FIG1 is a schematic diagram of a process flow of a photovoltaic module defect diagnosis method provided in an embodiment of the present application;

[0056] FIG2 is a schematic diagram of an image of slicing an aerial image provided in an embodiment of the present application;

[0057] FIG3 is a schematic diagram of the relative position relationship between two overlapping prediction frames provided in an embodiment of the present application;

[0058] FIG4 is a schematic diagram of different defect images provided in an embodiment of the present application;

[0059] FIG5 is a structural block diagram of a photovoltaic module defect diagnosis device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The core of the present invention is to provide a photovoltaic module defect diagnosis method, device and equipment, which improves the accuracy and reliability of photovoltaic module defect diagnosis and realizes diagnostic detection that is not limited to hot spot defects.

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] As shown in FIG. 1 , FIG. 1 is a schematic flow chart of a photovoltaic module defect diagnosis method provided in an embodiment of the present application.

[0063] In a specific embodiment of the present application, the photovoltaic module defect diagnosis method may include:

[0064] S10: Slice the aerial image to obtain a slice image; wherein the aerial image is an image captured by a drone at an altitude of not less than 60 meters from the target photovoltaic power station.

[0065] In actual applications, when drones carrying high-definition cameras conduct high-altitude inspections and sampling of target photovoltaic power stations, the flight altitude is maintained at 70 meters to 90 meters. At the same time, the flight trajectory should be kept as level as possible with the photovoltaic panels to ensure that the tilt angle of the photovoltaic panels in the picture is less than 30 degrees.

[0066] On this basis, considering that the scale of the photovoltaic imaging area in the high-resolution aerial images obtained by high-altitude sampling of drones is relatively small, the accuracy of direct photovoltaic component identification and detection is low and there are many missed detections; therefore, in this embodiment, image slicing technology is used to divide the aerial images into slice images.

[0067] In an optional implementation of this embodiment, the process of slicing the aerial image may include:

[0068] The aerial image is segmented into slice images with overlapping areas using image slicing technology; wherein the overlapping width of two adjacent slice images is not less than 300 pixels.

[0069] 2 , each frame of the aerial image can be segmented into 6 slice images arranged in a 3×2 arrangement, and the overlap between two adjacent slice images is maintained at 300 pixels, and the slice images are obtained after the slicing operation.

[0070] It is understandable that compared with aerial images, the area of ​​a single photovoltaic imaging region in each slice image is obviously relatively larger. Therefore, the identification of photovoltaic components in each slice image is less difficult and more accurate than directly identifying photovoltaic components in aerial images, which is conducive to providing reliable data basis for subsequent steps.

[0071] S20: Using a pre-trained first deep learning model, the photovoltaic components in the slice image are identified, and the photovoltaic components in each slice image are marked.

[0072] The first deep learning model in this embodiment may be a photovoltaic component detection model obtained by learning and training based on the Yolov8 network model.

[0073] In this embodiment, the training steps of the first deep learning model may include:

[0074] 201) Use LableImg to annotate photovoltaic modules in slice image samples, and use the slice images as training datasets.

[0075] Among them, the size of the training data set is 55,212, and the number of photovoltaic module instances marked in each slice image sample is 2,041,886.

[0076] 202) Divide the training data set into training set, validation set, and test set in a ratio of 7:2:1.

[0077] 203) Build a network model based on Yolov8, use the training set for training, and verify and adjust the process parameters in the network model through the validation set. Save the weight file every specific number of iterations. Use the early stopping mechanism in the training process, and set the number of early stopping cycles to 50, that is, terminate the training if there is no significant improvement in the results within 50 consecutive cycles.

[0078] 204) After the training is completed, the best weight file in the training process is selected as the final weight file to obtain a pre-trained photovoltaic component detection model based on deep learning, that is, the first deep learning model.

[0079] It should be noted that, in the process of obtaining the first deep learning model through the above-mentioned training, the prediction box formed on each slice image to mark each photovoltaic imaging area, the coordinate values ​​of the four corner points of the prediction box, and the confidence of the coordinate values ​​of the four corner points of the prediction box can be used as the output data of the first deep learning model.

[0080] Therefore, the process of identifying the photovoltaic components in the slice image using the first deep learning model may include:

[0081] The first deep learning model is used to identify and output the photovoltaic components in each slice image, a prediction frame marking each photovoltaic imaging area is formed on the slice image, and the first coordinate values ​​of the four corner points of each prediction frame and the confidence corresponding to the first coordinate values ​​of the four corner points are obtained.

[0082] In this embodiment, a prediction box is used to mark each photovoltaic imaging area in the slice image; generally, the area enclosed by the prediction box should be a quadrilateral area, and the four corner points of the prediction box are also the four corner points of the quadrilateral area; on this basis, the first coordinate values ​​of the four corner points of the prediction box (that is, the coordinate values ​​in the coordinate system of the slice image) also mark the position of the photovoltaic imaging area in the slice image.

[0083] After the photovoltaic components in the slice image are identified by the first deep learning model, it cannot be guaranteed that the accuracy of the first deep learning model in identifying the photovoltaic components in the slice image is 100%; therefore, in an optional implementation of this embodiment, after the photovoltaic components in the slice image are identified by the first deep learning model, the following may be further included:

[0084] According to the prediction box in the slice image, the width and height of each photovoltaic imaging area in the slice image are obtained;

[0085] According to the width and height, filter and delete the sliced ​​images that do not meet the filtering formula The prediction box of is used to filter out the photovoltaic modules that are misdetected in the slice image; among them, , , , is the width, height, area, and aspect ratio of the i-th prediction box detected in the slice image, and are the average area of ​​all prediction boxes in the slice image and the standard aspect ratio of the photovoltaic module, and denote the area error coefficient and the aspect ratio error coefficient respectively, for example, and Both can be set to 0.2.

[0086] In this embodiment, the size specifications of photovoltaic modules in the slice image should be basically consistent, and the aspect ratio and area size of the photovoltaic imaging area in the slice image are used to filter out incomplete or incorrectly identified photovoltaic modules. It can be understood that the prediction frame is divided based on the photovoltaic imaging area, which is equivalent to marking the edge line of the photovoltaic module in the slice image. The width and height of the photovoltaic imaging area marked by it can be determined based on the prediction frame; thus, the imaging area size corresponding to each photovoltaic imaging area can be calculated based on the width and height size corresponding to each prediction frame, that is, the area size of the prediction frame; the area of ​​each prediction frame is averaged; obviously, under normal circumstances, the area size of each prediction frame should fluctuate around the average area value, and the difference from the average area value is not large; thus, in this embodiment, this can be used as a basis for judging whether the area enclosed by each prediction frame is a photovoltaic imaging area. Once the area of ​​the prediction frame is too large or too small relative to the average area value, it can be considered that the area determined by the prediction frame is not a photovoltaic imaging area or a complete photovoltaic imaging area, and the prediction frame can be deleted.

[0087] In addition, when determining the average area value corresponding to each prediction box, the average area value corresponding to all prediction boxes in all slice images can be obtained by calculating the average area value, that is, increasing the amount of data for calculating the average area value to ensure the reliability of the average area value.

[0088] Similarly, under normal circumstances, in the slice image, the aspect ratio in the photovoltaic imaging area is also fixed and does not change with the scaling of the photovoltaic module; therefore, this embodiment can directly determine the standard aspect ratio of the photovoltaic module based on the width and height dimensions of the current conventional photovoltaic module, and then the aspect ratio of the photovoltaic imaging area determined by each prediction frame should also fluctuate around the standard aspect ratio of the photovoltaic module. Once the aspect ratio of a certain prediction frame differs greatly from the standard aspect ratio of the photovoltaic module, it can be considered that the prediction frame does not determine the photovoltaic imaging area or is not a complete photovoltaic imaging area, and the prediction frame can be deleted.

[0089] S30: stitching the slice images to obtain a photovoltaic module image with a photovoltaic mark.

[0090] It can be understood that when stitching the slice images, it is to restore the slice images to aerial images, but the difference is that each photovoltaic imaging area is marked in the restored aerial image, that is, the photovoltaic component image with photovoltaic marking. When stitching and restoring the slice images, it is only necessary to stitch them again based on the slicing method of the above-mentioned aerial images. However, as mentioned above, when slicing the aerial images, the aerial images are divided and cut into slice images with overlapping areas. Therefore, when actually stitching the slice images, it is also necessary to stitch the overlapping areas of the two adjacent slice images, and the multiple prediction boxes in the overlapping areas that mark the same photovoltaic imaging area should also be stitched to avoid repeated marking of the same photovoltaic imaging area.

[0091] In an optional embodiment of this embodiment, the NMS algorithm may be used to achieve the splicing of overlapping areas between slice images, which may specifically include:

[0092] S31: converting the first coordinate values ​​of the four corner points corresponding to each prediction box in each slice image into second coordinate values ​​on the aerial image;

[0093] S32: determining the area intersection-and-union ratio between any two prediction boxes according to the second coordinate values ​​corresponding to the prediction boxes;

[0094] S33: retain the prediction frame with the highest confidence level of the first coordinate value among the two prediction frames whose area intersection-union ratio is greater than the set threshold, and splice the slice images to obtain the photovoltaic module image.

[0095] As described above, in this embodiment, the first coordinate values ​​of the four corner points of each prediction box on each slice image are the coordinate values ​​of the four corner points of the prediction box in the rectangular coordinate system of the slice image where it is located; and the second coordinate values ​​are the coordinate values ​​of the four corner points of the prediction box in the rectangular coordinate system of the aerial image when the slice image is restored and spliced ​​into the aerial image. Obviously, the correspondence between the rectangular coordinate system of the slice image and the rectangular coordinate system of the aerial image can be known based on the slicing process, and the conversion relationship between the first coordinate value and the second coordinate value of each prediction box can also be known, and this will not be repeated in this embodiment.

[0096] On this basis, the first coordinate values ​​of the four corner points of the prediction box on each slice image are converted into second coordinate values, which is equivalent to converting the corner point coordinates of the prediction box in each slice image used to splice into the same frame into the same coordinate system. In this way, it can be determined whether there are multiple prediction boxes that repeatedly mark the same photovoltaic imaging area between the prediction boxes.

[0097] As shown in FIG3 , in an optional embodiment, taking the same frame of aerial image being segmented into 6 slice images as an example, the process of merging the prediction frames repeatedly marking the same photovoltaic imaging area may include:

[0098] 301) Return the coordinates of the prediction boxes marking the imaging positions of the photovoltaic modules in the six slice images to the coordinate system of the initial aerial image, and obtain the real coordinates (i.e., the coordinate values ​​in the coordinate system of the aerial image, i.e., the second coordinate values) and the corresponding confidence values ​​(obtained based on the output of the first deep learning model) of the two diagonal corner points (such as the first corner point and the second corner point in FIG. 3 ) of all the prediction boxes;

[0099] 302) Arrange the confidences corresponding to each prediction box in descending order, and select the prediction box with the highest confidence as the current prediction box;

[0100] 303) The intersection-over-union (IOU) ratio between the current prediction box and the remaining prediction boxes is calculated using the intersection-over-union (IOU) ratio formula. The intersection-over-union (IOU) ratio formula is as follows:

[0101] ;

[0102] in, , , respectively represent the true coordinates of the first corner point and the second corner point on the prediction box A, , They represent the true coordinates of the first and second corner points on the predicted box B, respectively. and Represent the intersection area and union area of ​​prediction box A and prediction box B respectively, 0 means that the two prediction boxes do not intersect, and IOU represents the ratio of the intersection area to the union area of ​​the two prediction boxes, that is, the intersection-union ratio;

[0103] 304) Compare the size between the intersection over union (IOU) and the set threshold. If the IOU is greater than the set threshold, it is considered that the two prediction boxes repeatedly detect the same target (i.e., the same photovoltaic module). It is necessary to filter and delete the prediction box with low confidence according to the confidence comparison principle.

[0104] 305) reselect the prediction box with the highest confidence from the remaining prediction boxes as the current prediction box, and repeat the above 303) and 304) until all the remaining prediction boxes have been filtered;

[0105] 305) The prediction frames processed by the NMS algorithm are merged to obtain the final labeling results of the photovoltaic components in each slice image.

[0106] In the actual process of stitching the slice images, the prediction boxes that repeatedly mark the same photovoltaic module have been deleted, that is, the accuracy of the prediction boxes marking each photovoltaic imaging area in the final stitched and restored photovoltaic module image is guaranteed.

[0107] S40: Determine a photovoltaic imaging region in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging region in the photovoltaic component image to obtain a photovoltaic region image.

[0108] In this embodiment, on the basis of marking the photovoltaic imaging area in the photovoltaic component image, in order to further reduce the difficulty of subsequent diagnosis of whether the photovoltaic component marked by the prediction box has defects and ensure the diagnosis accuracy, the photovoltaic imaging area in the photovoltaic component image is further segmented to form a photovoltaic area image. The obtained photovoltaic area image is equivalent to removing the image information of all background imaging areas outside the photovoltaic imaging area, thereby avoiding the image information of the background imaging area interfering with the diagnosis process during the subsequent defect diagnosis, thereby reducing the difficulty of defect diagnosis and improving the defect diagnosis accuracy.

[0109] In an optional implementation of this embodiment, the process of obtaining a photovoltaic area image based on a photovoltaic component image may include:

[0110] S41: Convert photovoltaic module images from RGB color space to HSV color space;

[0111] S42: Expanding each prediction box in the photovoltaic component image outward by a preset ratio;

[0112] S43: In each expanded prediction frame, the photovoltaic imaging area is determined according to a set color threshold; wherein the ranges corresponding to the H threshold, the S threshold, and the V threshold in the set color threshold are (100, 124), (43, 255), and (46, 255), respectively.

[0113] In order to more accurately detect the photovoltaic imaging area in the photovoltaic module image, the prediction box is first appropriately expanded outward. For example, the outward expansion scale of the prediction box can be set to 40%, thereby ensuring that the prediction box completely encloses each photovoltaic imaging area; then the photovoltaic module image is converted from the RGB color space to the HSV color space; in the photovoltaic module image, the photovoltaic imaging area has the significant feature of being blue, so the pixel value of the photovoltaic imaging area should satisfy the color threshold range of the pixel value corresponding to the blue; for this reason, the color threshold can be set based on the pixel value corresponding to the blue, because the photovoltaic module is converted from the RGB color space to the HSV color space, and accordingly, the setting of the color threshold can include the H threshold, the S threshold and the V threshold, wherein the range of the H threshold is (100, 124), the range of the S threshold is (43, 255), and the range of the V threshold is (46, 255). In the process of segmenting the photovoltaic imaging area in the expanded prediction frame, it can be judged in turn whether the H value, S value and V value of each pixel point in the prediction frame are within the corresponding threshold range respectively. If so, it can be considered that the pixel point belongs to the photovoltaic imaging area, otherwise it does not belong to the photovoltaic imaging area, thereby accurately segmenting the photovoltaic imaging area.

[0114] Obviously, by segmenting the photovoltaic imaging area in the prediction frame according to the set color threshold, on the one hand, it can further eliminate the device structure whose outer contour is similar to the outer contour of the photovoltaic module and is mistakenly identified as a photovoltaic module. On the other hand, it can also segment out the border imaging area of ​​the photovoltaic module, and only highlight the area within the border of the photovoltaic module, further reducing the target area for subsequent defect diagnosis.

[0115] After the photovoltaic imaging area within the prediction frame on the photovoltaic module image is segmented, in order to further ensure the accuracy of subsequent defect diagnosis, in another optional embodiment of the present embodiment, the process of determining the photovoltaic imaging area according to the set color threshold may further include:

[0116] The area where each pixel point whose HSV value in the prediction box is within the set color threshold range is defined as the initial photovoltaic imaging area;

[0117] Converting the photovoltaic module image into a photovoltaic module mask image highlighting the initial photovoltaic imaging area;

[0118] Using the image erosion formula The photovoltaic component mask image is corroded to obtain a photovoltaic component mask image after the corrosion process, so that the initial photovoltaic imaging area is corrected to the photovoltaic imaging area; wherein, and They are the pixels in the mask image of the photovoltaic module. Pixel values ​​before and after erosion, is the kernel size; the kernel size can be set to 9×9;

[0119] The photovoltaic module mask image is converted into a photovoltaic module image that determines the photovoltaic imaging area.

[0120] It can be understood that the mask image is also a mask image, and the photovoltaic module mask image in this embodiment is an image that masks the area except the photovoltaic imaging area in the image, so that the photovoltaic imaging area in the photovoltaic module image is white, and all areas except the photovoltaic imaging area are converted to black. After obtaining the photovoltaic module mask image, in order to use the morphological image processing method to corrode the photovoltaic module mask image, the brighter noise points are eliminated, thereby improving the accuracy of segmenting the optical imaging area.

[0121] Further optionally, after the photovoltaic component mask image is corroded, the following steps may also be included:

[0122] Perform contour detection and quadrilateral fitting on the photovoltaic imaging area in the mask image of the photovoltaic module after corrosion, and obtain the coordinates of the four corner points of the quadrilateral photovoltaic imaging area;

[0123] The photovoltaic imaging area of ​​the quadrilateral is transformed in perspective according to the coordinates of the four corner points to obtain a photovoltaic imaging area with a rectangular shape.

[0124] It is understandable that when a drone is actually collecting and shooting aerial images of a target photovoltaic power station, it cannot guarantee that the optical axis of the camera carried by the drone is completely perpendicular to the plane where each photovoltaic component is located. Therefore, the aerial images collected and shot by the drone of each photovoltaic component may be tilted at a certain angle.

[0125] To this end, in this embodiment, a perspective transformation model for converting the oblique imaging of the photovoltaic component to a plane imaging can be determined based on the posture of the drone when collecting aerial images and the inclination angle of the photovoltaic component when it is actually installed; of course, the perspective transformation model can also be determined directly based on the coordinates of the four corner points of the quadrilateral photovoltaic imaging area; based on the perspective transformation model, the quadrilateral photovoltaic imaging area can be perspectively transformed, and finally a rectangular photovoltaic imaging area can be obtained. The rectangular photovoltaic imaging area after the perspective transformation can correct the distortion of the quadrilateral photovoltaic imaging area formed by the oblique imaging of the photovoltaic component to a certain extent, which is conducive to the subsequent accurate diagnosis of defects and faults.

[0126] S50: Perform defect diagnosis on the photovoltaic area image using the pre-trained second deep learning model to obtain a defect diagnosis result.

[0127] The photovoltaic area image obtained in the above steps is used as input and input into the second deep learning model. The second deep learning model can output the defect diagnosis result of the photovoltaic component in the photovoltaic area image.

[0128] The second deep learning model in this embodiment can be a photovoltaic component defect diagnosis model obtained by deep learning training based on the Yolov8 network model.

[0129] Optionally, the training process of the second deep learning model may include:

[0130] Defect labels are marked on each pre-collected photovoltaic image sample, and the defect labels include cell missing defect labels, weed occlusion defect labels, wire occlusion defect labels, shadow occlusion defect labels, and bird droppings occlusion defect labels;

[0131] A Yolov8 deep network model was constructed, and photovoltaic image samples marked with defect labels were used as training sets for deep learning training to obtain a second deep learning model.

[0132] As shown in Figure 4, in this embodiment, when training the second deep learning model, photovoltaic image samples containing a variety of different defects can be collected as comprehensively as possible; for example, the photovoltaic image samples can be image samples containing battery cell defects, weed occlusion defects, wire occlusion defects, shadow occlusion defects, bird droppings occlusion defects, etc.; thereby, it can be ensured that the second deep learning model obtained through training can comprehensively identify and diagnose the defects of photovoltaic modules.

[0133] In addition, as one of the most powerful tools for image processing tasks such as recognition and target detection, the detection performance and effect of deep learning models are completely dependent on the training of a sufficient number of image samples. In reality, there are relatively few samples of defective photovoltaic modules in photovoltaic power stations. On the one hand, when a photovoltaic module has a major fault, it will be replaced immediately and there is no time to record it. On the other hand, it is costly and difficult to collect defective photovoltaic module samples based on drone inspections.

[0134] To this end, in the process of learning and training the second deep learning model in this embodiment, after the photovoltaic image samples that have been collected and the defects on the photovoltaic image samples are labeled, the following steps may be further included:

[0135] Expanding the photovoltaic image sample using a diffusion model to obtain a first expanded sample;

[0136] The photovoltaic image sample is translated, scaled, HSV space enhanced, flipped up and down, flipped left and right, rotated 90 degrees, 180 degrees, 270 degrees, or copied and pasted on the defect object, or a combination of multiple processing is performed to obtain a second expanded sample;

[0137] Accordingly, the photovoltaic image samples marked with defect labels are used as training sets for deep learning training to obtain a second deep learning model, including:

[0138] The first expanded samples, the second expanded samples, and the photovoltaic image samples are collectively used as training sets for learning and training to obtain a second deep learning model.

[0139] In this embodiment, a diffusion model is used as an image generation model. In practical applications, the existing corresponding photovoltaic image samples can be input into a diffusion model. The diffusion model is trained through the forward process and the reverse process through the photovoltaic image samples to form an image generation model that can identify photovoltaic defect features in the image. The forward process gradually adds Gaussian noise to each input image to destroy the training data until it becomes pure Gaussian noise, and the reverse process reverses the noise process to learn to recover the data. The image generation model that has been trained can denoise any Gaussian noise image, and finally output an image with photovoltaic defect features, which can be used as the first expansion sample.

[0140] In the process of training the diffusion model, the corresponding diffusion model should be trained separately for each defect, so that each diffusion model can be used to output the first expanded samples for different defects respectively.

[0141] On this basis, in order to avoid the overfitting problem of the second deep learning model due to the high proportion of the first expanded samples expanded based on the diffusion model; the photovoltaic image samples that have been collected are further subjected to various expansion sample operations such as translation, scaling, HSV space enhancement, upside down flipping, left-right flipping, 90 degree rotation, 180 degree rotation, 270 degree rotation or copying and pasting of defective objects, so as to obtain the second expanded samples; finally, the first expanded samples, the second expanded samples and the originally collected photovoltaic image samples are used as training sets to realize the learning and training of the second deep learning model, thereby ensuring the accuracy and reliability of the second deep learning model.

[0142] In a specific implementation of this embodiment, the learning and training process of the second deep learning model may include:

[0143] 501) Use LableImg to annotate different types of defects in photovoltaic image samples to create a training data set. The photovoltaic image samples in the training data set include 500 samples with missing cell defects, 470 samples with weed occlusion defects, 452 samples with wire occlusion defects, 288 samples with shadow occlusion defects, and 457 samples with bird droppings occlusion defects.

[0144] 502) First, the diffusion model is used to perform image generation operations on the collected photovoltaic image samples containing different types of defects to obtain new photovoltaic image samples with similar defect features to expand the training data set; then the photovoltaic image samples in the training data set are translated, scaled, HSV space enhanced, flipped up and down, flipped left and right, rotated 90 degrees, 180 degrees, 270 degrees, or the defect objects are copied and pasted to expand the samples in the training data set again.

[0145] 503) Divide the prepared training data set into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0146] 504) Build a network model based on Yolov8, use the training set for training, adjust the process parameters on the validation set, save the weight file every specific number of iterations, and use the early stopping mechanism during training. The number of early stopping cycles is set to 50, that is, if there is no significant improvement in the results within 50 consecutive cycles, the training will be terminated.

[0147] 505) After the training is completed, the best weight file in the training process is selected as the final weight file to obtain a pre-trained photovoltaic module defect diagnosis model based on deep learning, that is, a second deep learning model.

[0148] In addition, as described above, the image input to the second deep learning model in this embodiment is a photovoltaic component mask image, and thus the photovoltaic image samples in this embodiment should also be mask image samples.

[0149] Based on the above discussion, in the process of diagnosing photovoltaic defects based on aerial images collected by drones, the photovoltaic component identification and photovoltaic component defect diagnosis in the image are divided into two stages using two deep learning models, thereby improving the accuracy of deep learning model identification and diagnosis in the two-stage process and reducing the difficulty of diagnosing photovoltaic component defects; and, after identifying the photovoltaic components in the aerial image, the photovoltaic imaging area in the photovoltaic component image is separately segmented to form a highlight of the photovoltaic area image without background interference, thereby further improving the accuracy of the second deep learning model in defect diagnosis.

[0150] In summary, the aerial images in the present application are collected by a drone at an altitude of not less than 60 meters, so that each frame of aerial images collected by the drone can capture a larger area within the photovoltaic power station, thereby avoiding the problem of insufficient battery life of the drone during the collection of aerial images due to the large area of ​​the photovoltaic power station; on this basis, in the process of defect diagnosis of photovoltaic components based on aerial images in the present application, the photovoltaic components in the aerial images are first identified. In this process, considering that the aerial images are taken at high altitudes, the imaging area of ​​the photovoltaic components accounts for a relatively small proportion of the aerial images, and therefore the aerial images are first sliced, and then the images are processed. Identifying photovoltaic components with sliced ​​images can improve the accuracy of identifying photovoltaic components in aerial images to a certain extent; after identifying the photovoltaic components in the aerial images, the photovoltaic imaging area is separately segmented to form a photovoltaic area image, and finally the photovoltaic area image can be targeted for defect diagnosis, eliminating interference from other backgrounds. Compared with the method of directly performing photovoltaic defect diagnosis on the entire aerial image, the defect diagnosis method in this application can effectively ensure the accuracy of photovoltaic defect diagnosis; and, the defect diagnosis in this application is based on visible light images, and when performing defect diagnosis, it is not limited to the diagnosis and detection of hot spot defects. It can be seen that the method of detecting photovoltaic components in this application can break through the limitations of infrared inspection technology and ensure the accuracy and reliability of photovoltaic defect diagnosis in photovoltaic power station inspections.

[0151] The photovoltaic module defect diagnosis device provided by an embodiment of the present invention is introduced below. The photovoltaic module defect diagnosis device described below and the photovoltaic module defect diagnosis method described above can be referred to each other.

[0152] FIG5 is a structural block diagram of a photovoltaic module defect diagnosis device provided by an embodiment of the present invention. Referring to FIG5 , the photovoltaic module defect diagnosis device may include:

[0153] The slicing processing module 100 is used to perform slicing processing on the aerial image to obtain a slicing image; wherein the aerial image is an image collected by a drone at an altitude of not less than 60 meters from the target photovoltaic power station;

[0154] An image recognition module 200 is used to recognize the photovoltaic components in the slice images by using a pre-trained first deep learning model, and mark the photovoltaic components in each slice image;

[0155] The slice stitching module 300 is used to stitch the slice images to obtain a photovoltaic module image with a photovoltaic mark;

[0156] The image processing module 400 is used to determine the photovoltaic imaging area in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging area in the photovoltaic component image to obtain a photovoltaic area image;

[0157] The defect diagnosis module 500 is used to perform defect diagnosis on the photovoltaic area image by using a pre-trained second deep learning model to obtain a defect diagnosis result.

[0158] In an optional embodiment of the present application, the slice processing module 100 is specifically used to segment the aerial image into the slice images with overlapping areas by using the image slicing technology; wherein the overlapping width of two adjacent slice images is not less than 300 pixels;

[0159] The image recognition module 200 is specifically used to use the first deep learning model to identify and output the photovoltaic components in each of the slice images, form a prediction frame marking each photovoltaic imaging area on the slice image, and obtain the first coordinate values ​​of the four corner points of each of the prediction frames and the confidence corresponding to the first coordinate values ​​of the four corner points;

[0160] The slice stitching module 300 is specifically used to convert the first coordinate values ​​of the four corner points corresponding to each prediction box in each slice image into second coordinate values ​​on the aerial image; determine the area intersection and union ratio between any two prediction boxes according to the second coordinate values ​​corresponding to each prediction box; retain the prediction box with the highest confidence level of the first coordinate value among the two prediction boxes whose area intersection and union ratio is greater than a set threshold, and stitch the slice images to obtain the photovoltaic module image.

[0161] In an optional embodiment of the present application, the photovoltaic module defect diagnosis device also includes a filtering and screening module, which is used to obtain the width and height dimensions of each photovoltaic imaging area in the slice image according to the prediction box in the slice image after marking the photovoltaic modules in each slice image, and before obtaining the photovoltaic module image with the photovoltaic mark; according to the width and height dimensions, filter and delete the slice image that does not satisfy the filtering formula The prediction frame is used to screen out photovoltaic components that are incorrectly detected in the slice image; wherein, , , , is the width, height, area, and aspect ratio of the i-th prediction box detected in the slice image, and are respectively the average area value of all the prediction boxes in the slice image and the standard aspect ratio of the photovoltaic module, and They represent the area error coefficient and the aspect ratio error coefficient respectively.

[0162] In an optional embodiment of the present application, the image processing module 400 is specifically used to convert the photovoltaic component image from the RGB color space to the HSV color space; expand each of the prediction boxes in the photovoltaic component image outward by a preset proportion; in each of the expanded prediction boxes, segment the photovoltaic imaging area according to a set color threshold, and determine the photovoltaic imaging area according to the set color threshold; wherein the ranges corresponding to the H threshold, S threshold, and V threshold in the set color threshold are (100, 124), (43, 255), and (46, 255), respectively.

[0163] In an optional embodiment of the present application, the image processing module 400 is specifically used to define the area where each pixel point in the prediction box with an HSV value within the set color threshold range is located as the initial photovoltaic imaging area; convert the photovoltaic component image into a photovoltaic component Mask image that highlights the initial photovoltaic imaging area; use the image corrosion formula The photovoltaic component mask image is corroded to obtain the photovoltaic component mask image after corrosion processing, so that the initial photovoltaic imaging area is corrected to the photovoltaic imaging area; wherein, and are the pixels in the mask image of the photovoltaic module Pixel values ​​before and after erosion, is the kernel size; converting the photovoltaic component mask image into a photovoltaic component image that determines the photovoltaic imaging area.

[0164] In an optional embodiment of the present application, the image processing module 400 is also used to perform contour detection and quadrilateral fitting on the photovoltaic imaging area in the photovoltaic component mask image after corrosion, and obtain the four corner point coordinates of the quadrilateral photovoltaic imaging area after corrosion; according to the four corner point coordinates, the quadrilateral photovoltaic imaging area is perspective transformed to obtain a photovoltaic imaging area with a rectangular shape, and then the step of converting the photovoltaic component mask image into a photovoltaic component image that determines the photovoltaic imaging area is executed.

[0165] In an optional embodiment of the present application, the photovoltaic module defect diagnosis device also includes a model training module, which is used to mark defect labels for each pre-collected photovoltaic image sample, wherein the defect labels include cell missing defect labels, weed occlusion defect labels, wire occlusion defect labels, shadow occlusion defect labels, and bird droppings occlusion defect labels; construct a Yolov8 deep network model, use the photovoltaic image samples marked with defect labels as a training set to perform deep learning training, and obtain the second deep learning model.

[0166] In an optional embodiment of the present application, the model training module is further used to, after marking defect labels on each pre-collected photovoltaic image sample, expand the photovoltaic image sample using a diffusion model to obtain a first expanded sample; translate, scale, HSV space enhancement, flip up and down, flip left and right, rotate 90 degrees, rotate 180 degrees, rotate 270 degrees, or copy and paste the defect object, or a combination of multiple processing, to obtain a second expanded sample;

[0167] Correspondingly, the model training module is specifically used to use the first expanded sample, the second expanded sample and the photovoltaic image sample as a training set for learning and training to obtain the second deep learning model.

[0168] The photovoltaic module defect diagnosis device of this embodiment is used to implement the aforementioned photovoltaic module defect diagnosis method. Therefore, the specific implementation method of the photovoltaic module defect diagnosis device can be seen in the embodiment part of the photovoltaic module defect diagnosis method in the previous text. Its specific implementation method can refer to the description of the corresponding each part embodiment, which will not be repeated here.

[0169] The present application also provides an embodiment of a photovoltaic component defect diagnosis device, the photovoltaic component defect diagnosis device comprising:

[0170] Memory for storing computer programs;

[0171] A processor is used to execute the computer program to implement the steps of the photovoltaic module defect diagnosis method as described in any one of the above items.

[0172] The memory may be a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0173] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "includes", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment comprising a series of elements are inherent to the elements. In the absence of more restrictions, the elements limited by the sentence "comprising one..." do not exclude the presence of other identical elements in the process, method, article or equipment comprising the elements. In addition, the above-mentioned technical solution provided in the embodiment of the present application is consistent with the corresponding technical solution in the prior art in principle, and the part is not described in detail, so as to avoid too much redundancy.

[0174] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A photovoltaic module defect diagnosis method, characterized in that: include: Slice the aerial image to obtain a slice image; wherein the aerial image is an image acquired by a drone at an altitude of not less than 60 meters from the target photovoltaic power station; Using a pre-trained first deep learning model, the photovoltaic components in the slice images are identified, and the photovoltaic components in each of the slice images are marked; splicing the slice images to obtain a photovoltaic module image with a photovoltaic mark; Determine a photovoltaic imaging region in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging region in the photovoltaic component image to obtain a photovoltaic region image; Using the pre-trained second deep learning model, defect diagnosis is performed on the photovoltaic area image to obtain a defect diagnosis result.

2. The photovoltaic module defect diagnosis method according to claim 1, characterized in that: Slice the aerial images to obtain slice images, including: The aerial image is divided into slice images with overlapping areas by using image slicing technology; wherein the overlapping width of two adjacent slice images is not less than 300 pixels; Using a pre-trained first deep learning model, identifying photovoltaic components in the slice images, and marking the photovoltaic components in each slice image, including: Using the first deep learning model to identify and output the photovoltaic components in each of the slice images, forming a prediction frame marking each photovoltaic imaging area on the slice image, and obtaining first coordinate values ​​of four corner points of each of the prediction frames and confidences corresponding to the first coordinate values ​​of the four corner points; The slice images are stitched together to obtain a photovoltaic module image with a photovoltaic mark, including: Converting the first coordinate values ​​of the four corner points corresponding to each of the prediction boxes in each of the slice images into second coordinate values ​​on the aerial image; Determining an area intersection-to-union ratio between any two of the prediction boxes according to the second coordinate values ​​corresponding to the prediction boxes; Among the two prediction frames whose area intersection-to-union ratio is greater than a set threshold, the prediction frame whose corresponding first coordinate value has the highest confidence is retained, and the slice images are spliced ​​to obtain the photovoltaic assembly image.

3. The photovoltaic module defect diagnosis method according to claim 2, characterized in that: After marking the photovoltaic components in each of the slice images, before stitching the slice images to obtain a photovoltaic component image with a photovoltaic mark, the method further includes: According to the prediction frame in the slice image, obtaining the width and height of each photovoltaic imaging area in the slice image; According to the width and height dimensions, filter and delete the slice images that do not satisfy the filtering formula The prediction frame is used to screen out photovoltaic components that are misdetected in the slice image; wherein, , , , is the width, height, area, and aspect ratio of the i-th prediction box detected in the slice image, and are respectively the average area value of all the prediction boxes in the slice image and the standard aspect ratio of the photovoltaic module, and They represent the area error coefficient and the aspect ratio error coefficient respectively.

4. The photovoltaic module defect diagnosis method according to claim 2, characterized in that: Determining a photovoltaic imaging area in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image includes: Converting the photovoltaic module image from the RGB color space to the HSV color space; Expanding each prediction frame in the photovoltaic component image outward by a preset ratio; In each of the expanded prediction frames, the photovoltaic imaging area is determined according to a set color threshold; wherein the ranges corresponding to the H threshold, the S threshold, and the V threshold in the set color threshold are (100, 124), (43, 255), and (46, 255), respectively.

5. The photovoltaic module defect diagnosis method according to claim 4, characterized in that: In each of the expanded prediction frames, the photovoltaic imaging area is determined according to a set color threshold, including: Delimiting the area where each pixel point in the prediction frame whose HSV value is within the set color threshold range is located as the initial photovoltaic imaging area; Converting the photovoltaic assembly image into a photovoltaic assembly Mask image highlighting the initial photovoltaic imaging area; Using the image erosion formula The photovoltaic assembly mask image is corroded to obtain a photovoltaic assembly mask image after corrosion processing, so that the initial photovoltaic imaging area is corrected to the photovoltaic imaging area, wherein: and are the pixels in the mask image of the photovoltaic module Pixel values ​​before and after erosion, is the kernel size; The photovoltaic assembly mask image is converted into a photovoltaic assembly image that determines the photovoltaic imaging area.

6. The photovoltaic module defect diagnosis method according to claim 5, characterized in that: After obtaining the mask image of the photovoltaic module after corrosion treatment, it also includes: The photovoltaic imaging area in the photovoltaic component mask image after corrosion is subjected to contour detection and quadrilateral fitting to obtain the coordinates of the four corner points of the quadrilateral photovoltaic imaging area; according to the four corner point coordinates, the quadrilateral photovoltaic imaging area is subjected to perspective transformation to obtain a photovoltaic imaging area having a rectangular shape, and then the photovoltaic component mask image is converted into a photovoltaic component image that determines the photovoltaic imaging area.

7. The photovoltaic module defect diagnosis method according to any one of claims 1 to 6, characterized in that: The process of pre-training the second deep learning model includes: Marking each pre-collected photovoltaic image sample with a defect label, wherein the defect label includes a cell missing defect label, a weed occlusion defect label, a wire occlusion defect label, a shadow occlusion defect label, and a bird droppings occlusion defect label; A Yolov8 deep network model is constructed, and deep learning training is performed using the photovoltaic image samples marked with defect labels as a training set to obtain the second deep learning model.

8. The photovoltaic module defect diagnosis method according to claim 7, characterized in that: After marking defect labels on each pre-collected photovoltaic image sample, it also includes: Expanding the photovoltaic image sample using a diffusion model to obtain a first expanded sample; The photovoltaic image sample is translated, scaled, HSV space enhanced, flipped upside down, flipped left to right, rotated 90 degrees, 180 degrees, 270 degrees, or the defect object is copied and pasted, or a combination of multiple processing is performed to obtain a second expanded sample; Accordingly, deep learning training is performed using the photovoltaic image samples marked with defect labels as a training set to obtain the second deep learning model, including: The first expanded sample, the second expanded sample, and the photovoltaic image sample are collectively used as a training set for learning and training to obtain the second deep learning model.

9. A photovoltaic module defect diagnosis device, characterized in that: include: A slicing processing module is used to perform slicing processing on the aerial image to obtain a slicing image; wherein the aerial image is an image collected by a drone at an altitude of not less than 60 meters from the target photovoltaic power station; An image recognition module, used to identify the photovoltaic components in the slice images by using a pre-trained first deep learning model, and mark the photovoltaic components in each of the slice images; A slice splicing module, used for splicing the slice images to obtain a photovoltaic module image with a photovoltaic mark; An image processing module, used to determine a photovoltaic imaging area in the photovoltaic component image according to the photovoltaic mark in the photovoltaic component image, and cut out the photovoltaic imaging area in the photovoltaic component image to obtain a photovoltaic area image; The defect diagnosis module is used to perform defect diagnosis on the photovoltaic area image by using a pre-trained second deep learning model to obtain a defect diagnosis result.

10. A photovoltaic module defect diagnosis device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program to implement the steps of the photovoltaic module defect diagnosis method according to any one of claims 1 to 8.

Images