Photovoltaic module infrared image segmentation method and device and storage medium

Abstract

The invention provides a photovoltaic module infrared image segmentation method and device and a storage medium, and the method comprises the steps: carrying out the detection of at least two edge straight lines of a photovoltaic module infrared image, and obtaining at least two edge straight line groups; based on a confidence coefficient judgment mechanism, performing fusion processing on the at least two edge straight line groups to obtain a fused edge straight line set; and performing image segmentation on the photovoltaic module infrared image by using the plurality of fused edge straight lines in the fused edge straight line set to obtain a plurality of segmented images. According to the method provided by the invention, multi-algorithm scoring can be carried out on the infrared image of the photovoltaic module in combination with multiple edge detection processing to obtain the edge detection result, the edge detection result is further fused, the flexibility and robustness are higher, and the image segmentation precision and effect are improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a method, apparatus and storage medium for infrared image segmentation of photovoltaic modules. Background Technology

[0002] As a crucial component of clean energy, photovoltaic (PV) power generation is prone to various problems as its core component, the photovoltaic panel, ages. These problems include cracks, component damage, dust accumulation, and shading, impacting power generation efficiency, durability, and cost-effectiveness. Therefore, it is essential to detect defects in PV modules promptly during operation and maintenance. Summary of the Invention

[0003] To address the problems existing in related technologies, this disclosure proposes a method, apparatus, and storage medium for infrared image segmentation of photovoltaic modules.

[0004] The first aspect of this disclosure provides a method for segmenting infrared images of photovoltaic modules, comprising: performing at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups; performing fusion processing on the at least two edge line groups based on a confidence judgment mechanism to obtain a fused edge line set; and using multiple fused edge lines from the fused edge line set to segment the infrared image of the photovoltaic module to obtain multiple segmented images.

[0005] In some embodiments of this disclosure, the method further includes: acquiring a group of infrared images of a photovoltaic panel; performing full-scene modeling and stitching processing on the group of infrared images of the photovoltaic panel to obtain a panoramic image; and preprocessing the panoramic image to obtain an infrared image of the photovoltaic module.

[0006] In some embodiments of this disclosure, preprocessing of the panoramic image to obtain an infrared image of the photovoltaic module includes: performing photovoltaic string detection processing on the panoramic image to determine the photovoltaic strings in the panoramic image; and performing background removal processing and image correction processing on the photovoltaic strings in the panoramic image to obtain an infrared image of the photovoltaic module.

[0007] In some embodiments of this disclosure, at least two edge line detection processes are performed on the infrared image of the photovoltaic module to obtain at least two edge line groups. This includes: performing edge detection on the infrared image of the photovoltaic module using at least two edge line detection processes from the edge line detection processing set to obtain at least two edge line groups. The edge line detection processing set includes one-way histogram detection processing, Hough transform detection processing, line segment detection processing (LSD), Freeman line detection processing, and inchworm crawling detection processing.

[0008] In some embodiments of this disclosure, a fusion process is performed on at least two edge line groups based on a confidence level determination mechanism to obtain a fused edge line set. This includes: sorting multiple first edge lines of a first edge line group in the at least two edge line groups based on peak gradients to obtain a first sorting value corresponding to the first edge lines; and sorting multiple second edge lines of a second edge line group in the at least two edge line groups based on a voting mechanism to obtain a second sorting value corresponding to the second edge lines; determining a first confidence level of the first edge lines based on the first sorting value; and determining a second confidence level of the second edge lines based on the second sorting value; and performing a fusion process on the edge lines in the at least two edge line groups based on the first confidence level and the second confidence level to obtain a fused edge line set.

[0009] In some embodiments of this disclosure, determining a first confidence level and a second confidence level of a first edge line based on a first ranking value and a second ranking value includes: determining the confidence level of the edge line of the first ranking value and the second ranking value in a first interval as a first value; determining the confidence level of the edge line of the first ranking value and the second ranking value in a second interval as a second value; and determining the confidence level of the edge line of the first ranking value and the second ranking value in a third interval as a third value, wherein the first value, the second value, and the third value decrease sequentially.

[0010] In some embodiments of this disclosure, edge lines in at least two edge line groups are fused according to a first confidence level and a second confidence level to obtain a fused edge line set, including: determining at least two third edge lines within a first range of the first and second confidence levels; and determining a target slope and a target intercept based on the slope, intercept, and confidence level of at least two edge lines that satisfy preset conditions among the at least two third edge lines to obtain a first fused edge line, wherein the first range is greater than the third value and less than the first value.

[0011] In some embodiments of this disclosure, the preset conditions are: at least two edge lines satisfy a parallel relationship; and the distance between at least two edge lines is less than a preset threshold.

[0012] In some embodiments of this disclosure, the method further includes: determining a fourth edge line with a first confidence level or a second confidence level of a first value as a second fusion edge line, wherein the set of fusion edge lines includes the first fusion edge line and the second fusion edge line.

[0013] In the above embodiments, the photovoltaic module infrared image segmentation method can combine multiple edge detection processes to perform edge detection on the preprocessed photovoltaic module infrared image, and further fuse the edge detection results, which is more flexible and robust, and improves the segmentation accuracy and effect.

[0014] A second aspect of this disclosure provides a photovoltaic module infrared image segmentation device, which includes: a detection module for performing at least two edge line detection processes on the photovoltaic module infrared image to obtain at least two edge line groups; a fusion module for performing fusion processing on the at least two edge line groups based on a confidence judgment mechanism to obtain a fused edge line set; and a segmentation module for performing image segmentation on the photovoltaic module infrared image using multiple fused edge lines from the fused edge line set to obtain multiple segmented images.

[0015] A third aspect of this disclosure provides an electronic device comprising: one or more processors; a storage device communicatively connected to the one or more processors and storing one or more programs thereon; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in the first aspect of this disclosure.

[0016] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in the first aspect of this disclosure.

[0017] A fifth aspect of this disclosure provides a computer program product including a computer program that, when executed, implements the methods described in the first aspect of this disclosure.

[0018] In summary, the photovoltaic module infrared image segmentation method, apparatus and storage medium proposed in this disclosure can combine multiple edge detection processes to perform edge detection on the preprocessed photovoltaic module infrared image, and further fuse the edge detection results, which is more flexible and robust, and improves the segmentation accuracy and effect.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0021] Figure 1 This is a flowchart illustrating the infrared image segmentation method for photovoltaic modules proposed in this embodiment.

[0022] Figure 2 This is a schematic diagram of the preprocessing process for panoramic images proposed in an embodiment of this disclosure;

[0023] Figure 3This is a schematic diagram of the edge line detection process proposed in the embodiments of this disclosure;

[0024] Figure 4 This is a schematic diagram of the fusion processing flow proposed in the embodiments of this disclosure;

[0025] Figure 5A A flowchart of a method for segmenting infrared images of photovoltaic modules;

[0026] Figure 5B Infrared images of photovoltaic panels captured and photovoltaic strings inspected;

[0027] Figure 5C This is an example image of a photovoltaic string after preprocessing.

[0028] Figure 5D A schematic diagram of the edge line scoring and fusion process detected by multiple algorithms;

[0029] Figure 5E This is a schematic diagram of the image segmentation results;

[0030] Figure 6 This is a schematic diagram of the structure of an infrared image segmentation device for a photovoltaic module provided in an embodiment of the present disclosure;

[0031] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments are described below with reference to the accompanying drawings.

[0033] Currently, defect detection of photovoltaic modules is achieved by using high-resolution cameras or other inspection equipment mounted on drones to take aerial photos of the photovoltaic panels. Algorithms are then used to segment the photovoltaic modules from the background to identify defects, damage, or potential problems in the photovoltaic system, ensuring the maintenance of the photovoltaic power generation system's operational efficiency and safety.

[0034] Currently, algorithms related to photovoltaic (PV) module segmentation include traditional detection algorithms and deep learning algorithms. Traditional detection algorithms, such as edge detection, extract each cell unit in the PV module. Although they perform well on PV modules with clear regularity and minimal appearance variations, they are prone to over-segmentation or under-segmentation due to the uncertainties in imaging quality and equipment stability in real-world industrial inspection scenarios. This can lead to compromised cell unit integrity and reduced reliability of the detection results. Deep learning algorithms, such as Mask-RCNN and U-Net, perform average segmentation on the infrared image of the PV module to obtain several smaller image units for subsequent defect detection. However, due to the randomness of defect location and size, the obtained image units may only contain a portion of the defect, resulting in incomplete feature information and increasing the difficulty of subsequent defect detection.

[0035] Therefore, this disclosure aims to provide a method for infrared image segmentation of photovoltaic modules, which solves the problems of limited positioning accuracy and low efficiency in the image segmentation process of photovoltaic modules.

[0036] The following section provides a detailed description of the photovoltaic module infrared image segmentation method and photovoltaic module infrared image segmentation device provided in this application, with reference to the accompanying drawings.

[0037] The method disclosed herein can segment infrared images of photovoltaic modules based on multi-algorithm scoring, thereby improving the accuracy and effectiveness of image segmentation.

[0038] Figure 1 This is a flowchart illustrating the infrared image segmentation method for photovoltaic modules proposed in this disclosure. Figure 1 As shown, the method includes the following steps:

[0039] Step 101: Perform at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups.

[0040] In some embodiments, the method further includes: acquiring a group of infrared images of a photovoltaic panel; performing full-scene modeling and stitching processing on the group of infrared images of the photovoltaic panel to obtain a panoramic image; and preprocessing the panoramic image to obtain an infrared image of the photovoltaic module.

[0041] In some embodiments, acquiring a group of infrared images of a photovoltaic panel can be achieved by using a camera to photograph a photovoltaic panel with overlapping areas at a fixed frequency, thereby obtaining multiple infrared images of the photovoltaic panel.

[0042] For example, a drone is used to fly along a planned route and capture infrared images of photovoltaic panels with overlapping areas at a fixed frequency. Figure 5B (a) shows an infrared image of a photovoltaic panel.

[0043] In some embodiments, the full-scene modeling and stitching of the photovoltaic panel infrared image group can be performed by using a 3D reconstruction algorithm to model the entire scene and then stitching it together to form a panoramic image of the shooting area.

[0044] In some embodiments, the 3D reconstruction algorithm can be any of a variety of 3D reconstruction algorithms such as OpenDroneMap, COLMAP, DeepSDF, and Occupancy Networks, and this disclosure does not limit it.

[0045] In some embodiments, the splicing process can be any of a variety of splicing methods such as OpenCV / MeshLab, Laplacian Blending, SIFT / SURF, etc., and this disclosure does not limit it.

[0046] For example, the entire scene is modeled using the OpenDroneMap 3D reconstruction algorithm, and finally stitched together to form a panoramic image of the captured area.

[0047] In some embodiments, preprocessing of the panoramic image may involve identifying the photovoltaic strings in the panoramic image and performing noise reduction processing to obtain an image of the photovoltaic strings.

[0048] In some embodiments, preprocessing of the panoramic image may involve identifying photovoltaic strings in the panoramic image and performing correction processing to obtain a corrected image of the photovoltaic strings.

[0049] In some embodiments, preprocessing of the panoramic image may involve identifying photovoltaic strings in the panoramic image and obtaining an image of the photovoltaic strings after removing interfering pixels and correcting them through noise reduction and correction processing.

[0050] In some embodiments, preprocessing the panoramic image can yield an infrared image of the photovoltaic module for image segmentation, making the infrared image of the photovoltaic module clearer for subsequent processing, improving the effect of image segmentation, and providing a reference basis for subsequent defect detection.

[0051] In some embodiments, at least two edge line detection processes are performed on the infrared image of the photovoltaic module to obtain at least two edge line groups. Multiple edge line detection processes can be used to perform edge line detection on the infrared image of the photovoltaic module. In each edge line detection process, an edge line group can be obtained. The edge line group includes multiple edge lines, including horizontal edge lines and vertical edge lines.

[0052] In some embodiments, for infrared images of photovoltaic modules, different edge line detection processes can produce the same or different edge lines. In other words, since different edge line detection processes are used in different ways, the final detection results are not the same. Therefore, by using multiple edge line detection processes, different detection results can be obtained, and multiple detection results can be used for subsequent fusion processing to enrich the reference data.

[0053] In some embodiments, the specific method for edge line detection processing is not limited in this disclosure, and may be at least two of a variety of edge line detection methods such as one-way histogram method, Hough transform, and line segment detection method.

[0054] In the above embodiments, the edge line detection processing can be a deep learning method to improve the efficiency and accuracy of detection, reduce manual intervention, and have stronger adaptability and robustness.

[0055] Step 102: Based on the confidence judgment mechanism, at least two edge line groups are fused to obtain a fused edge line set.

[0056] In some embodiments, the confidence judgment mechanism can be a pre-set confidence judgment mechanism, that is, for different edge lines, the confidence can be determined according to the confidence judgment mechanism. For example, the edge lines in the edge line group obtained by each edge line detection processing can be sorted according to their characteristics, and the confidence of each edge line can be determined according to the sorting value.

[0057] In some embodiments, based on a confidence level judgment mechanism, at least two edge line groups are fused. This can be done by determining the edge lines that can be used for fusion processing and those that cannot be used for fusion processing based on the confidence level of the edge lines in each edge line group. That is, edge lines with low or high confidence levels are discarded and not included in the fused edge line set. The edge lines used for fusion processing are then further fused to obtain a fused edge line set.

[0058] In some embodiments, based on a confidence level judgment mechanism, at least two edge line groups are fused. This can be done by fusing edge lines whose confidence levels meet a certain range to obtain fused edge lines.

[0059] In some embodiments, based on a confidence level determination mechanism, at least two edge line groups are fused. This can be done by fusion of edge lines with the same confidence level, that is, generating a new fused edge line based on the confidence level of multiple edge lines with the same confidence level in two edge line groups.

[0060] In some embodiments, based on a confidence level judgment mechanism, at least two sets of edge lines are fused to obtain multiple fused edge lines in the vertical direction and multiple fused edge lines in the horizontal direction. In particular, multiple fused edge lines in the vertical direction and one fused edge line in the horizontal direction can be obtained.

[0061] In the above embodiments, the results obtained from multiple edge line detection processes are fused to make the fused edge line more accurate and robust.

[0062] Step 103: Use multiple fusion edge lines from the fusion edge line set to perform image segmentation on the infrared image of the photovoltaic module to obtain multiple segmented images.

[0063] In some embodiments, image segmentation of the infrared image of a photovoltaic module using multiple fusion edge lines in the fusion edge line set can be performed by using fusion edge lines in the vertical direction and fusion edge lines in the horizontal direction to obtain multiple segmented images.

[0064] In the above embodiments, edge line detection is performed on the infrared image of the photovoltaic module by using multiple edge line detection processes. The detection results obtained from each processing method are fused to segment the infrared image of the photovoltaic module, making the image segmentation more accurate, improving the accuracy and effect of segmentation, increasing flexibility, reducing manual intervention, and enhancing adaptability and robustness.

[0065] Figure 2 This is a schematic diagram of the preprocessing flow for panoramic images proposed in this disclosure. Figure 2 based on Figure 1 The illustrated embodiment further explains step 101. For example... Figure 2 As shown, the method includes the following steps:

[0066] Step 201: Perform photovoltaic string detection processing on the panoramic image to determine the photovoltaic strings in the panoramic image.

[0067] In some embodiments, photovoltaic string detection processing of panoramic images can be performed by using YOLO-8 with a transfer learning strategy to automatically detect photovoltaic strings in order to obtain the pixel position of each photovoltaic string in the panoramic image.

[0068] In some embodiments, the YOLO-8 transfer learning strategy can involve dividing the stitched panoramic image into a training set, a validation set, and a test set according to a certain proportion. The model is trained using the training and validation sets, and tested using the test set. Using this model, the photovoltaic strings in the panoramic image can be obtained. For example... Figure 5B (b) shows the detected photovoltaic strings.

[0069] In some embodiments, photovoltaic string detection processing of panoramic images can be performed using edge detection + morphological processing to obtain photovoltaic strings in the panoramic image. The edge detection + morphological processing can be performed by using the Canny algorithm to extract edges and by using the Hough transform to detect straight lines.

[0070] In some embodiments, photovoltaic string detection processing of panoramic images can be performed using a threshold segmentation method, wherein the threshold segmentation method can be based on separating photovoltaic panels from the background using the HSV color space to obtain the photovoltaic strings in the panoramic image.

[0071] In some embodiments, photovoltaic string detection processing of panoramic images can be performed using a deep learning model, such as YOLO-5, UNet++, Mask R-CNN, etc.

[0072] In the above embodiments, the specific detection method for photovoltaic string detection processing of panoramic images is not limited in this disclosure, and any of the above methods can be used.

[0073] Step 202: Perform background removal and image correction processing on the photovoltaic strings in the panoramic image to obtain an infrared image of the photovoltaic module.

[0074] In some embodiments, background removal processing of photovoltaic strings in panoramic images can be based on the Segment Anything Model (SAM), using point prompts to automatically select foreground photovoltaic strings, thereby obtaining infrared images of photovoltaic modules after background removal.

[0075] Specifically, when automatically removing the background of photovoltaic strings based on SAM point hints, two points are automatically selected from the image, namely P1(x1,y1) and P2(x2,y2), and the calculation formula is as follows:

[0076] Where w and h are the width and height of the image, respectively.

[0077] In some embodiments, the image after background removal is further subjected to image correction processing, which may be a combination of automatic rotation correction and automatic perspective transformation to eliminate the influence of shooting angle and depth of field.

[0078] In some embodiments, automatic rotation correction is used to correct the horizontal line to be parallel to the x-axis. Specifically, the infrared image of the photovoltaic module is first converted into a grayscale image, straight lines are detected using Hough transform, and the angle of each straight line is calculated. If the angle is within a specified range, it is determined to be a horizontal line. The difference between the average angle of the horizontal line and the theoretical angle is the angle to be rotated.

[0079] In some embodiments, automatic perspective transformation is used to correct vertical lines to be parallel to the y-axis. Specifically, the infrared image of the photovoltaic module is first converted to a grayscale image. A Hough transform is used to detect straight lines, and the angle of each line is calculated. If the angle is within a specified range, it is determined to be a vertical line. A parallelogram is constructed with its four vertices as the original coordinate points: Q1(0,0), Q2(w-diff-1,0), Q3(w-1,h-1), and Q4(diff-1,h-1), where w and h are the width and height of the image, respectively. The calculation formula is as follows: diff = h(tanA); where A is the difference between the average angle of the vertical line and the theoretical angle. The parallelogram is then transformed into a rectangle with vertex coordinates M1(0,0), M2(w-diff-1,0), M3(w-diff-1,h-1), and M4(0,h-1), thus obtaining the image perspective transformation matrix [Q,M]. Based on this matrix, the perspective transformation of the photovoltaic module's infrared image is completed.

[0080] For example, such as Figure 5C The image shown is an example of a photovoltaic module infrared image after preprocessing. Figure (a) shows the result of removing the background based on SAM point cues, and Figure (b) shows the result after removing the background and performing rotation correction and perspective transformation.

[0081] In some embodiments, the edges of the photovoltaic module infrared image obtained through background removal and image correction processing are the edges of the photovoltaic string, and the photovoltaic module infrared image can be made rectangular, which can be further used for subsequent edge line detection process, thereby improving the efficiency and accuracy of edge detection, and thus improving the efficiency and accuracy of image segmentation.

[0082] In the above embodiments, by preprocessing the panoramic image, the influence of environmental factors, shooting interference and other factors on the obtained infrared image of the photovoltaic module can be reduced, thereby improving robustness.

[0083] Figure 3 This is a schematic diagram of the edge line detection process proposed in this disclosure. Figure 3 based on Figures 1-2 The illustrated embodiment is for Figure 1 Step 101 in the document will be explained further. For example... Figure 3 As shown, the method includes the following steps:

[0084] Step 301: Use at least two edge line detection processes from the edge line detection processing set to perform edge detection on the infrared image of the photovoltaic module to obtain at least two edge line groups.

[0085] In some embodiments, the edge line detection processing set includes one-way histogram detection processing, Hough transform detection processing, line segment detection processing (LSD), Freeman line detection processing, and inchworm crawling detection processing.

[0086] In some embodiments, the one-way histogram detection processing method is the same as the one-way histogram method used in related technologies. Specifically, it can be to calculate the average gray value of the image from the x-axis and y-axis directions respectively, and detect the vertical edge line of the photovoltaic module based on the x-axis direction result; and detect the horizontal edge line of the photovoltaic module based on the y-axis direction result. In detecting the vertical edge line, firstly, the number range of modules is determined according to the data characteristics (the aspect ratio of the string image), that is, the width value of the module. Then, the negative of the average gray value in the x-axis direction is converted into a set of signal values. Based on the width value (here also called peak width), the peak of the signal is found, and the index corresponding to the peak is the position of the vertical edge line. In detecting the horizontal edge line, since each photovoltaic string has one and only one horizontal edge line, and theoretically vertically centered in the image, and to avoid interference from the positioning points of the photovoltaic panel itself, the peak width is set to 2h / 3. Then, the negative of the average gray value in the y-axis direction is converted into a set of signal values. Based on the peak width, the peak of the signal is found, and the index corresponding to the peak is the position of the horizontal edge line.

[0087] In some embodiments, edge detection is performed on the infrared image of the photovoltaic module using one-way histogram detection processing, which yields a series of horizontal and vertical edge lines in the infrared image of the photovoltaic module. Each edge line has a corresponding peak.

[0088] In some embodiments, edge detection of the infrared image of the photovoltaic module is performed using at least two edge line detection processes from the above set. This can be achieved by using one-way histogram detection and Hough transform detection to perform edge detection on the infrared image of the photovoltaic module, resulting in two sets of edge detection lines, each set of edge detection lines including a horizontal edge line and a vertical edge line.

[0089] Furthermore, the Hough transform detection process can first perform edge detection, specifically including: using Gaussian filtering to filter noise in the infrared image of the photovoltaic module; calculating the magnitude and direction of the image gradient; using non-maximum suppression to process regions with high gradients; and using a double threshold algorithm to filter edges. Then, the Hough transform is used to filter edge lines from the detected edges, setting parameters such as the minimum length of the line and the maximum interval between two lines allowed to merge in the same direction, and merging two parallel lines that are close to each other.

[0090] In some embodiments, line segment detection (LSD) can be used to perform edge detection on the infrared image of the photovoltaic module. The specific processing flow of LSD is the same as that in related technologies, and will not be described again here.

[0091] In some embodiments, Freeman line detection can be used, and the specific processing flow is the same as that in related technologies, so it will not be described again here.

[0092] In some embodiments, inchworm crawling detection can be used, and the specific processing flow is the same as that in related technologies, so it will not be described again here.

[0093] In some embodiments, edge detection of the infrared image of the photovoltaic module can be performed using any two or more of the above detection processes. Each detection process can obtain a set of edge lines, including horizontal edge lines and vertical edge lines.

[0094] In the above embodiments, by employing multiple edge line detection processes to perform edge detection on the infrared image of the photovoltaic module, edge line groups corresponding to each detection process can be obtained for subsequent fusion processing. The edge lines in multiple edge line groups are then filtered and fused to improve the accuracy of the final edge lines.

[0095] Figure 4 This is a schematic diagram of the fusion processing proposed in this disclosure. Figure 4 based on Figures 1-3 The illustrated embodiment further explains step 102. For example... Figure 4 As shown, the method includes the following steps:

[0096] Step 401: Based on the peak gradient, sort the multiple first edge lines of the first edge line group in at least two edge line groups to obtain the first sorting value corresponding to the first edge line; and based on the voting mechanism, sort the multiple second edge lines of the second edge line group in at least two edge line groups to obtain the second sorting value corresponding to the second edge line.

[0097] In some embodiments, the first set of edge lines may be a set of edge lines obtained by edge detection of the infrared image of the photovoltaic module using one-way histogram detection processing, and the second set of edge lines may be a set of edge lines obtained by edge detection of the infrared image of the photovoltaic module using Hough transform detection processing, line segment detection processing (LSD), Freeman line detection processing, or inchworm crawling detection processing.

[0098] In some embodiments, the number of the first edge line group and the second edge line group may be one or more, which is not limited in this disclosure.

[0099] In some embodiments, the first set of edge lines can be the result of detecting the infrared image of the photovoltaic module by any of the detection processing methods in the above-mentioned edge line detection processing set.

[0100] In some embodiments, the second edge line group can be the result of detecting the infrared image of the photovoltaic module by any of the detection processing methods in the above edge line detection processing set.

[0101] In some embodiments, the sorting order can be a descending order from largest to smallest or an ascending order from smallest to largest. In different embodiments, different order rules can be set according to needs or scenarios.

[0102] In some embodiments, sorting multiple first edge lines in a first edge line group of at least two edge line groups based on peak gradient and sorting multiple second edge lines in a second edge line group of at least two edge line groups based on a voting mechanism can be done by sorting multiple first edge lines in a first edge line group based on peak gradient and sorting multiple second edge lines in a second edge line group based on a voting mechanism.

[0103] In some embodiments, sorting multiple first edge lines in a first edge line group of at least two edge line groups based on peak gradient, and sorting multiple second edge lines in a second edge line group of at least two edge line groups based on a voting mechanism, may be based on peak gradient to sort multiple first edge lines in the first edge line group and sort multiple second edge lines in the second edge line group.

[0104] In some embodiments, sorting multiple first edge lines in a first edge line group of at least two edge line groups based on peak gradient, and sorting multiple second edge lines in a second edge line group of at least two edge line groups based on a voting mechanism, may be sorting multiple first edge lines obtained by unidirectional histogram detection processing based on peak gradient, and sorting multiple second edge lines obtained by Hough transform detection processing based on a voting mechanism.

[0105] For example, the confidence of edge lines detected by the one-way histogram method is calculated by the gradient of the peak. For N lines detected from the same image, they are sorted according to their gradient.

[0106] For example, the confidence level of edge lines detected by the Hough transform is based on its voting mechanism. For each edge point in the image space, it is mapped to the parameter space, with the mapping position (θ, ρ) depending on the angle of the line and the distance ρ from the origin. Each mapping is equivalent to one vote. Iterating through all edge points, edge points mapped to the parameter space (θ, ρ) are considered collinear. Similarly, for N lines detected from the same image, they are sorted according to the number of votes.

[0107] In some embodiments, when using any one or more of the line segment detection processing (LSD), Freeman line detection processing, and inchworm crawling detection processing for detection, the edge lines can be sorted based on the peak gradient or a voting mechanism. The selection can be made according to specific needs, scenarios, and applicability, and this disclosure does not limit this.

[0108] Step 402: Determine the first confidence level of the first edge line based on the first ranking value, and determine the second confidence level of the second edge line based on the second ranking value.

[0109] In some embodiments, determining a first confidence level of a first edge line and a second confidence level of a second edge line based on a first ranking value and a second ranking value includes: determining the confidence level of the edge line of the first ranking value and the second ranking value in a first interval as a first value; determining the confidence level of the edge line of the first ranking value and the second ranking value in a second interval as a second value; and determining the confidence level of the edge line of the first ranking value and the second ranking value in a third interval as a third value, wherein the first value, the second value, and the third value decrease sequentially.

[0110] In some embodiments, the first interval, the second interval, and the third interval are pre-set sorting intervals. For example, the first interval is the first 20% of the sorting order, the second interval is 20% to 70% of the sorting order, and the third interval is the remaining 30% of the sorting order.

[0111] In some embodiments, the first value, the second value, and the third value can be preset confidence level values, with the first value > the second value > the third value. For example, the first value is c1; the second value is c1. The third value is c2, where N is the number of edge lines obtained by each detection process, and i is the sorting index.

[0112] For example, for the confidence level of edge lines detected by the one-way histogram method, after sorting by peak gradient, the confidence level of the top 20% of lines is defined as c1; the confidence level of lines in the top 20%-70% is c1. Where i is the index arranged in descending order of gradient; the confidence level of the remaining lines is set as c2.

[0113] For example, regarding the confidence level of edge lines detected by the Hough transform, after sorting according to a voting mechanism, the confidence level of the top 20% of lines is set as c1; the confidence level of lines in the top 20%-70% is... Where i is the index sorted in descending order of vote count; the confidence level of the remaining lines is set to c2.

[0114] Step 403: Based on the first confidence level and the second confidence level, merge the edge lines in at least two edge line groups to obtain a merged edge line set.

[0115] In some embodiments, fusing edge lines in at least two edge line groups according to a first confidence level and a second confidence level may involve: determining at least two third edge lines within a first range based on the first confidence level and the second confidence level; and determining a target slope and a target intercept based on the slope, intercept, and confidence level of at least two edge lines that satisfy preset conditions among the at least two third edge lines to obtain a first fused edge line.

[0116] In some embodiments, the first range is greater than the third value and less than the first value.

[0117] In some embodiments, the preset conditions are: at least two edge lines satisfy a parallel relationship; and the distance between at least two edge lines is less than a preset threshold.

[0118] In some embodiments, the preset threshold may be a pre-set distance value, the specific value of which may be customized according to the scenario or requirements, and this disclosure does not limit it.

[0119] In some embodiments, the distance between at least two edge lines being less than a preset threshold can be defined as the distance between any two edge lines being less than a preset threshold.

[0120] In some embodiments, at least two edge lines satisfying a parallel relationship can mean that at least two edge lines are approximately parallel, that is, the included angle between any two edge lines is less than a preset angle, and the preset angle can be a pre-set minimum angle value.

[0121] In some embodiments, firstly, a plurality of third edge lines with confidence levels within a first range are determined in the first edge line group and the second edge line group. Further, at least two edge lines that satisfy preset conditions are determined from the plurality of third edge lines, and the at least two edge lines are merged to obtain a first merged edge line.

[0122] In some embodiments, a target slope and a target intercept are determined based on the slope, intercept, and confidence level corresponding to at least two edge lines to obtain a first fused edge line. This can be achieved by determining the target slope using the following formula, based on the slope and confidence level corresponding to each edge line:

[0123]

[0124] Where n is the number of lines with at least two edge lines, a i c is the slope of the i-th edge line. i It is the cth i Confidence level of the edge line.

[0125] Based on the intercept and confidence level corresponding to each edge line, the target intercept is determined using the following formula:

[0126]

[0127] Where n is the number of edge lines, b i c is the intercept of the i-th edge line. i It is the cth i Confidence level of the edge line.

[0128] In the above embodiment, by obtaining the target slope and the target intercept, the first fusion edge line can be obtained: y = ax + b.

[0129] In some embodiments, the method may further include: determining a fourth edge line with a first confidence level or a second confidence level of a first value as a second fusion edge line, wherein the set of fusion edge lines includes the first fusion edge line and the second fusion edge line.

[0130] For example, edge detection results of photovoltaic modules are fused from multiple algorithms using a scoring strategy, such as... Figure 5D The diagram shows the scoring and fusion process of edge lines detected by multiple algorithms. Specifically: 1) When any algorithm detects an edge line with a confidence level higher than or equal to c1, that line is included in the final result set; 2) When at least two algorithms detect two or more edge lines with a confidence level within the interval (c2, c1), and the edge lines are approximately parallel with a minimum distance less than ε, a new line y = ax + b is generated based on the confidence level of each edge line and included in the result set. The calculation method is shown in formulas (6) and (7); 3) Other detection results are not included in the result set. Figure 5E The image shown is a schematic diagram of the image segmentation results. (a) is a visualization of the edge line results fused based on the scoring strategy, and (b) is a single component image segmented based on the edge lines.

[0131]

[0132] Where n is the number of parallel lines whose distance is less than ε, a i ,b i These are the slope and intercept of the i-th parallel line, respectively, and c i It is the cth i The confidence level of a pair of parallel lines.

[0133] In the above embodiments, by combining confidence levels, edge lines that meet the conditions among multiple edge detection processes are fused to obtain fused edge lines, which improves the accuracy and efficiency of edge lines. Furthermore, using fused edge lines for image segmentation improves the accuracy and effect of infrared image segmentation of photovoltaic modules, and enhances flexibility and robustness.

[0134] In summary, the infrared image segmentation method for photovoltaic modules proposed in this disclosure can combine edge detection processing with multi-algorithm scoring to perform edge detection on the preprocessed infrared image of the photovoltaic module, and further fuse the edge detection results, which is more flexible and robust, and improves the segmentation accuracy and effect.

[0135] The following provides a detailed description of the specific implementation of a photovoltaic module infrared image segmentation method provided in this disclosure. For example... Figure 5A The diagram shown is a flowchart of this solution.

[0136] Step 1: Acquire infrared images of the photovoltaic panel;

[0137] Step 2, inspect the photovoltaic strings;

[0138] Step 3: Preprocess the string image to obtain a background-removed and corrected photovoltaic string image;

[0139] Step 4: Use multiple algorithms to detect the straight lines at the edges of the photovoltaic modules;

[0140] Step 5: Calculate the confidence level of the detected edge lines and normalize the confidence level;

[0141] Step 6: Based on the normalized confidence level, the edge detection results of photovoltaic modules from multiple algorithms are fused through a scoring strategy to complete the segmentation of the photovoltaic modules;

[0142] Furthermore, in step 1, a drone is first used to fly along a planned route to capture infrared images of photovoltaic panels with overlapping areas at a fixed frequency. Then, the OpenDroneMap 3D reconstruction algorithm is used to model the entire scene, and finally, the images are stitched together to form a panoramic image of the captured area. Figure 5B (a) shows an infrared image of a photovoltaic panel.

[0143] Furthermore, the method for automatically detecting photovoltaic strings in step 2 employs YOLO-V8 with a transfer learning strategy, dividing the stitched panoramic images into training, validation, and test sets according to proportions. The training and validation sets are used to train the model, and the test set is used to test the model. Figure 5B (b) shows the detected photovoltaic strings.

[0144] Furthermore, the preprocessing of the photovoltaic string image in step 3 includes automatic background removal and automatic image correction. For example... Figure 5C The image shown is an example of a photovoltaic string image after preprocessing. Figure (a) shows the result of removing the background based on SAM point cues, and Figure (b) shows the result after removing the background and performing rotation correction and perspective transformation.

[0145] The method for automatic background removal is based on the Segment Anything Model (SAM) and uses point prompts to automatically select the foreground photovoltaic strings.

[0146] The automatic image correction includes automatic rotation correction and automatic perspective transformation to eliminate the effects of shooting angle and depth of field.

[0147] The automatic rotation correction primarily corrects the horizontal lines to be parallel to the x-axis. First, the photovoltaic string image is converted to grayscale. Then, a Hough transform is used to detect straight lines, and the angle of each line is calculated. If the angle is within a specified range, it is determined to be a horizontal line. The difference between the average angle of the horizontal line and the theoretical angle is the angle to be rotated.

[0148] The automatic perspective transformation primarily corrects vertical lines to be parallel to the y-axis. First, the photovoltaic string image is converted to grayscale. Then, a Hough transform is used to detect straight lines, and the angle of each line is calculated. If the angle is within a specified range, it is determined to be a vertical line. A parallelogram is constructed, with its four vertices as the original coordinate points: Q1(0,0), Q2(w-diff-1,0), Q3(w-1,h-1), and Q4(diff-1,h-1), where w and h are the width and height of the image, respectively. The calculation formula is as follows:

[0149] (1) diff = h(tanA);

[0150] Where A is the difference between the average angle and the theoretical angle of the vertical line. The parallelogram is transformed into a rectangle, with vertex coordinates M1(0,0), M2(w-diff-1,0), M3(w-diff-1,h-1), and M4(0,h-1), thus obtaining the image perspective transformation matrix [Q,M]. Based on this matrix, the perspective transformation of the photovoltaic string image is completed.

[0151] Furthermore, when automatically removing the background of the photovoltaic string based on SAM point hints, two points are automatically selected from the image, namely P1(x1,y1) and P2(x2,y2), and the calculation formula is as follows:

[0152]

[0153] Where w and h are the width and height of the image, respectively.

[0154] Moreover, the various photovoltaic module edge straight line detection algorithms described in step 4 are the one-way histogram method and Hough transform.

[0155] The one-way histogram method calculates the average grayscale value of the image along both the x-axis and y-axis. Based on the x-axis result, it detects the vertical edge lines of the photovoltaic modules; based on the y-axis result, it detects the horizontal edge lines of the photovoltaic modules. In detecting the vertical edge lines, the number range of modules is first determined based on data characteristics (the aspect ratio of the string image), i.e., the width value of the modules. Then, the negative of the average grayscale value along the x-axis is converted into a set of signal values. Based on the width value (also called peak width), the peak of the signal is found, and the index corresponding to the peak is the position of the vertical edge line. In detecting the horizontal edge lines, since each photovoltaic string has exactly one horizontal edge line, which is theoretically vertically centered in the image, and to avoid interference from the positioning points of the photovoltaic panels, the peak width is set to 2h / 3. Then, the negative of the average grayscale value along the y-axis is converted into a set of signal values. Based on the peak width, the peak of the signal is found, and the index corresponding to the peak is the position of the horizontal edge line.

[0156] In the process of detecting straight lines at the edges of photovoltaic modules using the Hough transform, edge detection is first performed, specifically including: using Gaussian filtering to filter noise in the photovoltaic string image; calculating the magnitude and direction of the image gradient; using non-maximum suppression to process regions with high gradients; and using a double threshold algorithm to filter edges. Then, the Hough transform is used to filter straight lines from the detected edges, setting parameters such as the minimum length of the line and the maximum interval between two lines allowed to merge in the same direction, and merging two parallel lines that are close to each other.

[0157] Furthermore, the multi-algorithm confidence calculation and normalization method described in step 5 is as follows:

[0158] The confidence score of the edge lines detected by the one-way histogram method is calculated based on the gradient of the peaks. For N lines detected from the same image, they are sorted according to their gradients: the confidence score of the top 20% of lines is set as c1; the confidence score of lines between 20% and 70% is... Where i is the index arranged in descending order of gradient; the confidence level of the remaining lines is set as c2.

[0159] The confidence level of the edge lines detected by the Hough transform is based on its voting mechanism. For each edge point in the image space, it is mapped to the parameter space, and the mapping position (θ, ρ) depends on the angle of the line and the distance ρ from the origin. One mapping is equivalent to one vote. Iterating through all edge points, edge points mapped to the parameter space (θ, ρ) are considered collinear. Similarly, for N lines detected from the same image, they are sorted according to the number of votes: the confidence level of the top 20% of lines is set to c1; the confidence level of lines in the top 20%-70% is c1. Where i is the index sorted in descending order of vote count; the confidence level of the remaining lines is set to c2.

[0160] Furthermore, in step 6, based on normalized confidence, the edge detection results of photovoltaic modules from multiple algorithms are fused using a scoring strategy, such as... Figure 5D The diagram shows the scoring and fusion process of edge lines detected by multiple algorithms. Specifically: 1) When any algorithm detects an edge line with a confidence level higher than or equal to c1, that line is included in the final result set; 2) When at least two algorithms detect two or more edge lines with a confidence level within the interval (c2, c1), and the edge lines are approximately parallel with a minimum distance less than ε, a new line y = ax + b is generated based on the confidence level of each edge line and included in the result set. The calculation method is shown in formulas (6) and (7); 3) Other detection results are not included in the result set. Figure 5E The image shown is a schematic diagram of the image segmentation results. (a) is a visualization of the edge line results fused based on the scoring strategy, and (b) is a single component image segmented based on the edge lines.

[0161]

[0162] Where n is the number of parallel lines whose distance is less than ε, a i ,b i These are the slope and intercept of the i-th parallel line, respectively, and c i It is the cth i The confidence level of a pair of parallel lines.

[0163] In summary, the above method has the following beneficial effects: it integrates the results of multiple algorithms by adopting a scoring strategy, draws on the advantages of ensemble learning, overcomes the influence of environmental factors and shooting interference, improves the robustness of the algorithm, and develops a lightweight segmentation algorithm based on the characteristics of photovoltaic panels for application scenarios that are deployed in the field, providing technical support for the subsequent automatic inspection of photovoltaic power plants.

[0164] like Figure 6 As shown, this disclosure provides a structural schematic diagram of a photovoltaic module infrared image segmentation device 600. It is used to achieve, for example... Figures 1-4 The photovoltaic module infrared image segmentation method shown includes an apparatus comprising:

[0165] The detection module 610 is used to perform at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups;

[0166] The fusion module 620 is used to perform fusion processing on at least two edge line groups based on a confidence judgment mechanism to obtain a fused edge line set.

[0167] The segmentation module 630 is used to segment the infrared image of the photovoltaic module using multiple fusion edge lines from the fusion edge line set, and obtain multiple segmented images.

[0168] In the embodiments of this disclosure, the detection module is further configured to acquire a group of infrared images of the photovoltaic panel; perform full-scene modeling and stitching processing on the group of infrared images of the photovoltaic panel to obtain a panoramic image; and preprocess the panoramic image to obtain an infrared image of the photovoltaic module.

[0169] In embodiments of this disclosure, the detection module is further configured to perform photovoltaic string detection processing on the panoramic image to determine the photovoltaic strings in the panoramic image; and to perform background removal processing and image correction processing on the photovoltaic strings in the panoramic image to obtain an infrared image of the photovoltaic module.

[0170] In embodiments of this disclosure, the detection module is further configured to perform edge detection on the infrared image of the photovoltaic module using at least two edge line detection processes from the edge line detection processing set, to obtain at least two edge line groups, wherein the edge line detection processing set includes one-way histogram detection processing, Hough transform detection processing, line segment detection processing (LSD), Freeman line detection processing, and inchworm crawling detection processing.

[0171] In some embodiments of this disclosure, the fusion module is further configured to sort multiple first edge lines of a first edge line group in at least two edge line groups based on peak gradient to obtain a first sorting value corresponding to the first edge line; and sort multiple second edge lines of a second edge line group in at least two edge line groups based on a voting mechanism to obtain a second sorting value corresponding to the second edge line; determine a first confidence level of the first edge line based on the first sorting value; and determine a second confidence level of the second edge line based on the second sorting value; and perform fusion processing on the edge lines in at least two edge line groups according to the first confidence level and the second confidence level to obtain a fused edge line set.

[0172] In embodiments of this disclosure, the fusion module is further configured to determine the confidence level of the first ranking value and the second ranking value on the edge line of the first interval as a first value; determine the confidence level of the first ranking value and the second ranking value on the edge line of the second interval as a second value; and determine the confidence level of the first ranking value and the second ranking value on the edge line of the third interval as a third value, wherein the first value, the second value, and the third value decrease sequentially.

[0173] In embodiments of this disclosure, the fusion module is further configured to determine at least two third edge lines within a first range, based on the slope, intercept, and confidence level of at least two edge lines that satisfy preset conditions among the at least two third edge lines, and to determine a target slope and a target intercept to obtain a first fused edge line, wherein the first range is greater than the third value and less than the first value.

[0174] In the embodiments of this disclosure, the preset conditions are: at least two edge lines satisfy a parallel relationship; and the distance between at least two edge lines is less than a preset threshold.

[0175] In embodiments of this disclosure, the fusion module is further configured to determine the fourth edge line with a first confidence level or a second confidence level of a first value as the second fusion edge line, and the fusion edge line set includes the first fusion edge line and the second fusion edge line.

[0176] In summary, the photovoltaic module infrared image segmentation device proposed in this embodiment can combine multiple edge detection processes to perform edge detection on the preprocessed photovoltaic module infrared image, and further fuse the edge detection results, which is more flexible and robust, and improves the segmentation accuracy and effect.

[0177] The methods and apparatus provided in the embodiments of this application have been described above. To implement the functions of the methods provided in the embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0178] Figure 7 This is a block diagram illustrating an electronic device 700 for implementing the above-described infrared image segmentation method for photovoltaic modules, according to an exemplary embodiment.

[0179] Reference Figure 7The electronic device 700 may include a communication interface 701, capable of interacting with other devices; a processor 702, connected to the communication interface 701 to enable interaction with other devices, used to execute the methods provided by one or more of the above-described technical solutions when running a computer program; and a memory 703, on which the computer program is stored. Specifically, the specific processing procedure of the processor 702 can refer to the image segmentation method described in the above embodiments of this disclosure.

[0180] Of course, in practical applications, the various components in electronic device 700 are coupled together through bus system 704. It can be understood that bus system 704 is used to realize the connection and communication between these components. In addition to a data bus, bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 7 The general designated all buses as Bus System 704.

[0181] The memory 703 in this embodiment is used to store various types of data to support the operation of the electronic device 700. Examples of such data include any computer program used to operate on the electronic device 700.

[0182] The methods disclosed in the embodiments of this application can be applied to processor 702, or implemented by processor 702. Processor 702 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 702 or by instructions in the form of software. The processor 702 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 702 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 703. Processor 702 reads the information in memory 703 and combines its hardware to complete the steps of the aforementioned method.

[0183] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0184] Embodiments of this disclosure also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the photovoltaic module infrared image segmentation method described in the above embodiments of this disclosure.

[0185] Embodiments of this disclosure also provide a computer program product, including a computer program that is executed by a processor using the photovoltaic module infrared image segmentation method described in the above embodiments of this disclosure.

[0186] Embodiments of this disclosure also propose a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, it causes the electronic device to perform the photovoltaic module infrared image segmentation method described in the above embodiments of this disclosure.

[0187] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0188] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0189] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0190] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0191] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0192] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0193] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0194] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for segmenting infrared images of photovoltaic modules, characterized in that, The method includes: Perform at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups; Based on the confidence judgment mechanism, the at least two edge line groups are fused to obtain a fused edge line set. The infrared image of the photovoltaic module is segmented using multiple fusion edge lines from the set of fusion edge lines to obtain multiple segmented images.

2. The method according to claim 1, characterized in that, The method further includes: Acquire infrared image sets of photovoltaic panels; The infrared image group of the photovoltaic panel is subjected to full-scene modeling and stitching processing to obtain a panoramic image; The panoramic image is preprocessed to obtain the infrared image of the photovoltaic module.

3. The method according to claim 2, characterized in that, The preprocessing of the panoramic image to obtain the infrared image of the photovoltaic module includes: The panoramic image is subjected to photovoltaic string detection processing to determine the photovoltaic strings in the panoramic image; Background removal and image correction are performed on the photovoltaic strings in the panoramic image to obtain the infrared image of the photovoltaic module.

4. The method according to claim 3, characterized in that, The process of performing at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups includes: Edge detection is performed on the infrared image of the photovoltaic module using at least two edge line detection processes from the edge line detection processing set to obtain the at least two edge line groups. The edge line detection processing set includes one-way histogram detection processing, Hough transform detection processing, line segment detection processing (LSD), Freeman line detection processing, and inchworm crawling detection processing.

5. The method according to claim 4, characterized in that, The confidence-based judgment mechanism performs a fusion process on the at least two edge line groups to obtain a fused edge line set, including: Based on the peak gradient, multiple first edge lines in the first edge line group of the at least two edge line groups are sorted to obtain a first sorting value corresponding to the first edge line; and based on the voting mechanism, multiple second edge lines in the second edge line group of the at least two edge line groups are sorted to obtain a second sorting value corresponding to the second edge line. A first confidence level for the first edge line is determined based on the first ranking value, and a second confidence level for the second edge line is determined based on the second ranking value; Based on the first confidence level and the second confidence level, the edge lines in the at least two edge line groups are subjected to the fusion process to obtain the fused edge line set.

6. The method according to claim 5, characterized in that, The step of determining the first confidence level of the first edge line and the second confidence level of the second edge line based on the first ranking value and the second ranking value, respectively, includes: The confidence levels of the first sorted value and the second sorted value on the edge line of the first interval are respectively determined as the first value; The confidence levels of the first sorted value and the second sorted value on the edge lines of the second interval are respectively determined as the second value; The confidence levels of the first sort value and the second sort value on the edge lines of the third interval are respectively determined as the third value, and the first value, the second value, and the third value decrease sequentially.

7. The method according to claim 6, characterized in that, The step of performing the fusion process on the edge lines in the at least two edge line groups based on the first confidence level and the second confidence level to obtain the fused edge line set includes: Determine at least two third edge lines within a first range for the first confidence level and the second confidence level; Based on the slope, intercept, and confidence level of at least two edge lines that meet the preset conditions among the at least two third edge lines, the target slope and target intercept are determined to obtain the first fused edge line, wherein the first range is greater than the third value and less than the first value.

8. The method according to claim 7, characterized in that, The preset conditions are: The at least two edge lines satisfy a parallel relationship; The distance between at least two edge lines is less than a preset threshold.

9. The method according to claim 8, characterized in that, The method further includes: The fourth edge line with the first confidence level or the second confidence level equal to the first value is determined as the second fusion edge line, and the fusion edge line set includes the first fusion edge line and the second fusion edge line.

10. A photovoltaic module infrared image segmentation device, characterized in that, include: The detection module is used to perform at least two edge line detection processes on the infrared image of the photovoltaic module to obtain at least two edge line groups; The fusion module is used to perform fusion processing on the at least two edge line groups based on a confidence judgment mechanism to obtain a fused edge line set; The segmentation module is used to segment the infrared image of the photovoltaic module using multiple fusion edge lines from the set of fusion edge lines, thereby obtaining multiple segmented images.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method as described in any one of claims 1-9.

12. A computer program product, characterized in that, Includes a computer program that, when executed, implements the method as described in any one of claims 1-9.

Images