Two-stage photovoltaic panel identification method and device based on deep learning

Abstract

The invention relates to the technical field of photovoltaic station operation and maintenance management, and discloses a two-stage photovoltaic panel identification method and device based on deep learning, and the method comprises the steps: obtaining a target photovoltaic string image, carrying out the detection of a photovoltaic string through a deep learning model based on the target photovoltaic string image, and generating an ROI image corresponding to the target photovoltaic string; performing edge detection on the ROI image to obtain an edge contour of the photovoltaic panel; and traversing the edge contours of the photovoltaic panels, and screening and identifying the edge contours of the photovoltaic panels to obtain a global photovoltaic panel corresponding to the target photovoltaic string. According to the invention, the statistical accuracy and real-time performance of the photovoltaic panel are remarkably improved, and core technical support is provided for construction management and intelligent operation and maintenance of a photovoltaic field station.

Description

A two-stage photovoltaic panel identification method and device based on deep learning Technical Field

[0001] This invention relates to the field of photovoltaic power plant operation and maintenance management technology, specifically to a two-stage photovoltaic panel identification method and device based on deep learning. Background Technology

[0002] In photovoltaic power plants, effective identification of photovoltaic panels enables more efficient maintenance and management of these panels, thereby improving the overall efficiency and reliability of photovoltaic power generation.

[0003] In the construction and operation and maintenance management of photovoltaic power plants, the number of photovoltaic panels is currently counted manually on-site. However, this manual counting method suffers from problems such as low efficiency and susceptibility to environmental influences, making it difficult to meet the needs of digital construction and operation and maintenance management for large-scale photovoltaic power plants. Summary of the Invention

[0004] This invention provides a two-stage photovoltaic panel identification method and device based on deep learning to solve the problems of low identification efficiency and accuracy caused by reliance on manual surveying for photovoltaic power plant construction and operation and maintenance management.

[0005] In a first aspect, the present invention provides a two-stage photovoltaic panel recognition method based on deep learning. The method includes: acquiring a target photovoltaic string image; performing photovoltaic string detection using a deep learning model based on the target photovoltaic string image to generate a ROI image corresponding to the target photovoltaic string; performing edge detection on the ROI image to obtain the edge contour of the photovoltaic panel; traversing the edge contour of the photovoltaic panel and filtering and recognizing the edge contour of the photovoltaic panel to obtain the global photovoltaic panel corresponding to the target photovoltaic string.

[0006] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. In the first stage, a deep learning model is used to detect photovoltaic strings based on the target photovoltaic string image, generating the ROI image corresponding to the target photovoltaic string. This achieves accurate and rapid detection of photovoltaic strings and extracts the bounding box ROI region. Then, in the second stage, edge detection is performed on the ROI image to obtain the edge contour of the photovoltaic panel. The edge contour of the photovoltaic panel is then filtered and identified to obtain the global photovoltaic panel corresponding to the target photovoltaic string. This significantly improves the statistical accuracy and real-time performance of photovoltaic panels, providing core technical support for the construction management and intelligent operation and maintenance of photovoltaic power plants.

[0007] In one optional implementation, the deep learning model adopts the YOLO model; based on the target photovoltaic string image, the deep learning model is used to detect the photovoltaic string and generate the ROI image corresponding to the target photovoltaic string, including: inputting the target photovoltaic string image into the YOLO model to obtain the bounding box coordinates of the target photovoltaic string; removing the overlapping bounding boxes in the bounding box coordinates of the target photovoltaic string to generate the ROI image corresponding to the target photovoltaic string.

[0008] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. It utilizes the YOLO model to generate the bounding box coordinates of target photovoltaic strings, achieving accurate and rapid detection of target photovoltaic string images. This avoids the subjectivity and omissions of manual selection. Furthermore, it removes overlapping bounding boxes from the bounding box coordinates of the target photovoltaic strings, ensuring that each photovoltaic string retains only one most accurate bounding box. This avoids repeated calculations for the same string in subsequent analysis, ensuring that each ROI corresponds to a complete photovoltaic string. Subsequent photovoltaic panel recognition only needs to be performed within each ROI, eliminating the need for global processing of the entire image, significantly reducing computational load. Moreover, each ROI region contains only photovoltaic strings, effectively isolating background interference and laying the foundation for accurate photovoltaic panel recognition.

[0009] In one optional implementation, edge detection is performed on the ROI image to obtain a closed contour, including: image preprocessing of the ROI image to obtain a preprocessed ROI image; and edge detection is performed on the preprocessed ROI image to extract the edge contour of the photovoltaic panel.

[0010] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. By preprocessing the ROI image, noise can be removed and contrast enhanced, providing a cleaner image with more prominent features for edge detection, thus improving the quality and stability of edge detection. Then, edge detection is performed on the preprocessed ROI image to extract the edge contour of the photovoltaic panel, laying the foundation for accurate recognition of the photovoltaic panel in the future.

[0011] In one optional implementation, image preprocessing is performed on the ROI image to obtain a preprocessed ROI image, including: normalizing the size of the ROI image to obtain a fixed-size ROI image; performing HSV threshold separation on the fixed-size ROI image to obtain a filtered image; performing image binarization on the filtered image to obtain a binarized image; and performing morphological operations on the binarized image to denoise, thereby obtaining the preprocessed ROI image.

[0012] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. By performing size normalization, HSV threshold separation, image binarization, and morphological denoising on the ROI image, the method enhances the edge contour of the photovoltaic panel in the ROI image and improves the recognition accuracy of the photovoltaic panel.

[0013] In one optional implementation, edge detection is performed on the preprocessed ROI image to extract the edge contour of the photovoltaic panel, including: applying Gaussian filtering to the preprocessed ROI image to obtain a Gaussian-filtered image; convolving the pixel values ​​corresponding to the Gaussian-filtered image with the horizontal gradient kernel and the vertical gradient kernel respectively to obtain the horizontal gradient and the vertical gradient; calculating the gradient magnitude and gradient direction based on the horizontal gradient and the vertical gradient respectively; finding local maxima pixels along the gradient direction, retaining the gradient magnitude corresponding to the local maxima pixels, and generating a non-maximum suppressed gradient magnitude; comparing the non-maximum suppressed gradient magnitude with a hysteresis threshold, and filtering out single-pixel-wide edges based on the comparison result to obtain the edge contour of the photovoltaic panel; wherein, the edge contour of the photovoltaic panel is a closed contour.

[0014] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. By performing Gaussian filtering, gradient calculation, non-maximum suppression, and hysteresis threshold determination on the preprocessed ROI image, it achieves accurate extraction of all closed contours in the ROI image, making the edge contours of the photovoltaic panel fit the rectangular boundary features of the photovoltaic panel, effectively suppressing noise and improving edge stability.

[0015] In one optional implementation, the photovoltaic panel edge contour is traversed, and the photovoltaic panel edge contour is filtered and identified to obtain the global photovoltaic panel corresponding to the target photovoltaic string. This includes: traversing the photovoltaic panel edge contour and calculating the size of the outer rectangle corresponding to the photovoltaic panel edge contour; comparing the outer rectangle size with a preset threshold range; if the outer rectangle size meets the preset threshold range, the photovoltaic panel edge contour corresponding to the outer rectangle size is determined as a valid photovoltaic panel; obtaining the valid photovoltaic panel corresponding to the target photovoltaic string; and determining the global photovoltaic panel based on the valid photovoltaic panel corresponding to the target photovoltaic string.

[0016] This invention provides a two-stage photovoltaic panel identification method based on deep learning. By using the dual judgment criteria of the circumscribed rectangle size and the preset threshold range, it can effectively filter out interference contours such as noise, shadows, and stains, avoid the subjectivity of manual judgment, reduce the false detection rate, and then determine the global photovoltaic panel based on the effective photovoltaic panels corresponding to the target photovoltaic string, thus achieving accurate identification of photovoltaic panels.

[0017] Secondly, the present invention provides a two-stage photovoltaic panel recognition device based on deep learning. The device includes: a photovoltaic string detection module, used to acquire a target photovoltaic string image, and based on the target photovoltaic string image, to perform photovoltaic string detection using a deep learning model to generate a ROI image corresponding to the target photovoltaic string; an edge detection module, used to perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel; and a recognition module, used to traverse the edge contour of the photovoltaic panel, filter and recognize the edge contour of the photovoltaic panel, and obtain the global photovoltaic panel corresponding to the target photovoltaic string.

[0018] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the two-stage photovoltaic panel identification method based on deep learning described in the first aspect or any corresponding embodiment.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the deep learning-based two-stage photovoltaic panel identification method of the first aspect or any corresponding embodiment described above.

[0020] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the deep learning-based two-stage photovoltaic panel identification method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 is a schematic diagram of a first process of a two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention; Figure 3 is a schematic diagram of a two-stage photovoltaic panel recognition statistical method according to an embodiment of the present invention; Figure 4 is a schematic diagram of a second process of a two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention; Figure 5 is a schematic diagram of a third process of a two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention; Figure 6 is a schematic diagram of a fourth process of a two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention; Figure 7 is a structural block diagram of a two-stage photovoltaic panel recognition device based on deep learning according to an embodiment of the present invention; Figure 8 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0025] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0026] As an optional application scenario of this invention, as shown in FIG1, the two-stage photovoltaic panel identification device based on deep learning may include at least one terminal device and at least one server. FIG1 exemplarily shows that the system includes a computer 101, a mobile terminal 102 and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through the network 110.

[0027] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0028] While detection methods based on single-stage deep learning can quickly locate photovoltaic modules, they have two major technical drawbacks: First, they cannot effectively distinguish the hierarchical relationship between "photovoltaic strings" and "photovoltaic panels," which will lead to confusion of detection targets; second, they are not capable of handling shadow areas under complex lighting conditions, and the outline of photovoltaic panels is easily interfered with by noise, resulting in a high statistical error rate.

[0029] Image processing methods (such as threshold segmentation + contour matching) rely heavily on manual feature design, resulting in poor generalization ability and slow recognition speed.

[0030] Therefore, there is an urgent need for a two-stage approach that combines the strong feature extraction capabilities of deep learning with the fine recognition processing of visual algorithms to address the pain point that photovoltaic power plant construction and operation and maintenance management rely on manual surveys and lack mature, high-precision, and fast detection solutions.

[0031] This invention provides a two-stage photovoltaic panel recognition method based on deep learning. The first stage utilizes an improved YOLO (You Only Look Once, a real-time object detection model) deep neural network model to accurately and quickly detect photovoltaic strings and extract bounding boxes (ROIs). The second stage, within each ROI, sequentially performs image preprocessing, morphological operations, edge detection, and contour feature filtering to locate and count the number of photovoltaic panels. This method overcomes the detection bottleneck in complex scenes. The first stage quickly locates photovoltaic strings in the image, while the second stage identifies photovoltaic panels in the ROI image based on morphological and edge detection, significantly improving the accuracy and real-time performance of photovoltaic panel statistical recognition. This provides core technical support for photovoltaic power plant construction management and intelligent operation and maintenance.

[0032] According to an embodiment of the present invention, a two-stage photovoltaic panel identification method based on deep learning is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning, which can be used in the aforementioned terminal device. Figure 2 is a flowchart of the two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention. As shown in Figure 2, the process includes the following steps: Step S201, obtaining a target photovoltaic string image, and using a deep learning model to detect photovoltaic strings based on the target photovoltaic string image to generate an ROI image corresponding to the target photovoltaic string.

[0034] Specifically, as shown in Figure 3, the first stage uses an improved YOLO deep neural network model to accurately and quickly detect and extract the bounding box ROI region of the photovoltaic string.

[0035] Furthermore, before using a deep learning model for photovoltaic string detection, the deep learning model needs to be trained. The specific steps are as follows: collect a photovoltaic string dataset (i.e., a set of images with photovoltaic strings taken by drone inspection) covering different seasons, time periods (differences in shadow length between morning and evening), and weather conditions (cloudy / light rain / dust coverage). Label the photovoltaic strings in each image in the photovoltaic string dataset to form an image set and a label set. Then, divide the image set and label set simultaneously according to the ratio of "training set: validation set: test set = 8:1:1".

[0036] Furthermore, assuming the deep learning model adopts the YOLO (You Only Look Once, a real-time object detection algorithm) model, the backbone network of the YOLO model consists of a series of convolutional layers, batch normalization layers, activation functions, and downsampling layers, which are used to extract multi-scale features from the input image.

[0037] Furthermore, a CBAM (Convolutional Block Attention Module) is inserted into the intermediate feature layer of the backbone network of the YOLO model, forming a new process of "convolution → batch normalization → activation → attention filtering → downsampling" to focus on the contour features of the photovoltaic string and suppress interference from invalid information such as shadows and background (features such as desert / grassland).

[0038] Furthermore, after the YOLO model is improved by embedding an attention mechanism, it is set to input the training set for 300 rounds, and the learning rate is dynamically adjusted by adopting a cosine annealing strategy to strengthen effective features (such as the edges of photovoltaic panels) and suppress noise (such as shadows, deserts, etc.).

[0039] Furthermore, the trained YOLO model is used to detect photovoltaic strings and generate ROI images corresponding to the target photovoltaic strings.

[0040] Step S202: Perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel.

[0041] Specifically, as shown in Figure 3, the second stage involves identifying the edge contours of the photovoltaic panel in the ROI image based on morphology and edge detection, and outputting all closed contours in the ROI region, with each closed contour corresponding to a potential photovoltaic panel boundary.

[0042] Step S203: Traverse the edge contours of the photovoltaic panel, filter and identify the edge contours of the photovoltaic panel, and obtain the global photovoltaic panel corresponding to the target photovoltaic string.

[0043] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning. In the first stage, a deep learning model is used to detect photovoltaic strings based on the target photovoltaic string image, generating the ROI image corresponding to the target photovoltaic string. This achieves accurate and rapid detection of photovoltaic strings and extracts the bounding box ROI region. Then, in the second stage, edge detection is performed on the ROI image to obtain the edge contour of the photovoltaic panel. The edge contour of the photovoltaic panel is then filtered and identified to obtain the global photovoltaic panel corresponding to the target photovoltaic string. Through the deep integration of retrained YOLO model and image processing technology, the detection accuracy and processing efficiency are comprehensively improved. This method can be adapted to the construction and operation and maintenance of large-scale photovoltaic power plants in complex scenarios such as the Northwest Desert, providing high-precision, high-efficiency, and high-reliability technical support for the intelligent construction and operation and maintenance of photovoltaic power plants.

[0044] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning, which can be used in the aforementioned terminal device. Figure 4 is a flowchart of the two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention. As shown in Figure 4, the process includes the following steps: Step S401, acquiring a target photovoltaic string image, and based on the target photovoltaic string image, using a deep learning model to detect the photovoltaic string and generate an ROI image corresponding to the target photovoltaic string.

[0045] Specifically, step S401 includes: step S4011, inputting the target photovoltaic string image into the YOLO model to obtain the bounding box coordinates of the target photovoltaic string.

[0046] Specifically, the image to be processed is input into the trained YOLO model, which outputs the bounding box coordinates (x1, y1, x2, y2) of all photovoltaic strings.

[0047] Step S4012: Remove the overlapping bounding boxes in the bounding box coordinates of the target photovoltaic string to generate the ROI image corresponding to the target photovoltaic string.

[0048] Specifically, the NMS (Non-Maximum Suppression) algorithm is used to remove overlapping bounding boxes and generate a set of ROI regions. Each ROI in this set corresponds to a photovoltaic string in the image.

[0049] Furthermore, the specific steps for removing overlapping bounding boxes using the NMS algorithm include: obtaining the bounding box coordinates and confidence scores (i.e., the predicted probability that the bounding box is a photovoltaic string) of all photovoltaic strings; sorting the confidence scores from high to low to obtain an ordered list; selecting the bounding box with the highest confidence score in the ordered list and adding it to the final output box set; calculating the intersection-union ratio (IUR) between the bounding box with the highest confidence score and other bounding boxes in the ordered list; comparing the IUR with a preset screening threshold and removing all bounding boxes with IUR greater than the preset screening threshold from the ordered list; processing all bounding boxes in the ordered list according to the above steps to obtain the retained box set, i.e., the deduplicated bounding boxes, each deduplicated bounding box corresponding to an independent photovoltaic string; and cropping the corresponding ROI region from the target photovoltaic string image according to the deduplicated bounding boxes to form the ROI image.

[0050] Step S402: Perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel. For details, please refer to step S202 of the embodiment shown in Figure 2, which will not be repeated here.

[0051] Step S403: Traverse the edge contours of the photovoltaic panel, filter and identify the edge contours of the photovoltaic panel to obtain the global photovoltaic panel corresponding to the target photovoltaic string. For details, please refer to step S203 of the embodiment shown in Figure 2, which will not be repeated here.

[0052] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning. It utilizes the YOLO model to generate the bounding box coordinates of the target photovoltaic string, achieving accurate and rapid detection of the target photovoltaic string image. This avoids the subjectivity and omissions of manual selection. Furthermore, overlapping bounding boxes in the bounding box coordinates of the target photovoltaic string are removed, ensuring that only one most accurate bounding box is retained for each photovoltaic string. This avoids repeated calculations for the same string in subsequent analysis, ensuring that each ROI corresponds to a complete photovoltaic string. Subsequent photovoltaic panel recognition only needs to be performed within each ROI region, eliminating the need for global processing of the entire image, significantly reducing computational load. Moreover, each ROI region contains only the photovoltaic string, effectively isolating background interference and laying the foundation for accurate photovoltaic panel recognition.

[0053] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning, which can be used in the aforementioned terminal device. Figure 5 is a flowchart of the two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention. As shown in Figure 5, the process includes the following steps: Step S501, acquiring a target photovoltaic string image; based on the target photovoltaic string image, using a deep learning model to detect the photovoltaic string and generate an ROI image corresponding to the target photovoltaic string. For details, please refer to step S401 of the embodiment shown in Figure 4, which will not be repeated here.

[0054] Step S502: Perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel.

[0055] Specifically, step S502 includes: step S5021, performing image preprocessing on the ROI image to obtain a preprocessed ROI image.

[0056] Specifically, image preprocessing includes operations such as image binarization, size normalization, and HSV (Hue, Saturation, Value) thresholding to enhance features and improve accuracy.

[0057] In some optional implementations, step S5021 above includes: step a1, normalizing the size of the ROI image to obtain a fixed-size ROI image.

[0058] Specifically, the ROI image is scaled to a fixed size, maintaining a fixed aspect ratio and a uniform input scale. The fixed-size ROI image is an RGB (red, green, blue) image.

[0059] Step a2: Perform HSV thresholding on the fixed-size ROI image to obtain the filtered image.

[0060] Specifically, the normalized RGB image is converted to the HSV color space. Based on the color characteristics of the photovoltaic string, a threshold range is set to filter out shadow areas and retain the high-brightness non-shadow parts, which can be represented as: (1) In the above formula, For pixels, These are the filtered pixel values, where 255 represents the retained candidate area for the support structure, and 0 represents the background or shadow area. For pixels hue value, , These are the upper and lower limits of the hue threshold, respectively. For pixels The saturation value, , These are the upper and lower limits of the saturation value, respectively. For pixels The brightness value, , These are the upper and lower limits of the brightness value, respectively.

[0061] Step a3: Perform image binarization on the filtered image to obtain a binarized image.

[0062] Specifically, by setting an appropriate threshold, the pixels in the filtered image are divided into foreground and background. The foreground consists of the photovoltaic strings and the edge of the panel, while the background is the part of the ROI region excluding the photovoltaic strings, which can be represented as: (2) In the above formula, These are the pixel values ​​after image binarization. The grayscale threshold is set to 255. If the grayscale value of a pixel in the filtered image is greater than or equal to that of the filtered image, the pixel value is set to 255. In this case, the pixel in the image is classified as the foreground and the edge of the photovoltaic string. Otherwise, the pixel value is set to 0.

[0063] Step a4: Perform morphological operations on the binarized image to denoise it, and obtain the preprocessed ROI image.

[0064] Specifically, morphological denoising involves performing erosion followed by dilation on a binarized image to remove small noise points and enhance edges. The specific steps include: Erosion: A 5×5 rectangular structuring element is used to traverse the binarized image. Only the center pixel is retained if the structuring element completely covers the bright area of ​​the photovoltaic string; otherwise, it is set as background. Small noise points are removed using the 5×5 rectangular structuring element, while simultaneously compressing the edges of the photovoltaic panel to prepare for subsequent dilation. The formula is as follows: (3) Expansion: A 3×3 cross-shaped structural element is used to traverse the image. As long as the structural element overlaps with the bright area of ​​the photovoltaic string, the center pixel is retained. The 3×3 cross-shaped structural element is used to strengthen the edge of the photovoltaic panel, compensate for the contour shrinkage caused by corrosion, and make the broken edges more complete. The formula is: (4) In the above formula, The object of processing in the binarization of images. It is a rectangular structural element of the operation rules. The coordinates of a single pixel in a binarized image. structural element according to The set after translation transformation.

[0065] Step S5022: Perform edge detection on the preprocessed ROI image to extract the edge contour of the photovoltaic panel.

[0066] Specifically, the Canny operator-based edge detection (a multi-level edge detection algorithm) extracts all closed contours in the ROI image through Gaussian filtering, gradient calculation, non-maximum suppression, and hysteresis threshold determination.

[0067] In some optional implementations, step S5022 above includes: step b1, performing Gaussian filtering on the preprocessed ROI image to obtain a Gaussian-filtered image.

[0068] Specifically, high-frequency noise is removed by convolving the image with a two-dimensional Gaussian kernel. for: (5) In the above formula, and These are the pixel coordinates in the preprocessed ROI image. Let be the standard deviation of the Gaussian distribution, and be the smoothing procedure for controlling the Gaussian filter; typically, is taken as . .

[0069] Furthermore, the expression for applying Gaussian filtering to the preprocessed ROI image is: (6) In the above formula, Indicates the coordinates of the image after Gaussian filtering. Pixel value at that location, For a two-dimensional Gaussian kernel in The weight value at the location, This represents the pixel value at the corresponding offset position in the preprocessed ROI image. This represents the range of the convolution operation. The relative offset coordinates of the Gaussian kernel in the horizontal direction represent from to Perform a traversal. The relative offset coordinates in the vertical direction of the Gaussian kernel represent from to Perform a traversal.

[0070] Step b2: Convolve the pixel values ​​corresponding to the Gaussian filtered image with the horizontal gradient kernel and the vertical gradient kernel respectively to obtain the horizontal gradient and the vertical gradient.

[0071] Specifically, by calculating the horizontal position of the Gaussian filtered image... and vertical The gradient of the direction gives the intensity (magnitude) and direction (angle) of the edge.

[0072] Furthermore, the Gaussian-filtered image is compared with the horizontal gradient kernel. and vertical gradient kernel Convolution yields the horizontal gradient. and vertical gradient : (7) (8) Step b3: Calculate the gradient magnitude and gradient direction based on the horizontal and vertical gradients respectively.

[0073] Specifically, gradient magnitude and gradient direction The calculation formula is as follows: (9) (10) Step b4: Find local maximum pixels along the gradient direction, retain the gradient magnitude corresponding to the local maximum pixels, and generate the gradient magnitude after non-maximum suppression.

[0074] Specifically, non-maximum suppression is used to retain only the local gradient maximum point, eliminating the "wide edge" of non-edge pixels and forming a single-pixel wide edge; non-maximum suppression is to find local maximum pixels along the gradient direction, retain the gradient magnitude of the pixel, and suppress non-maximum pixels.

[0075] Step b5: Compare the gradient magnitude after non-maximum suppression with the hysteresis threshold, and filter out single-pixel wide edges based on the comparison results to obtain the edge contour of the photovoltaic panel; wherein, the edge contour of the photovoltaic panel is a closed contour.

[0076] Specifically, the hysteresis threshold is divided into a high threshold and a low threshold. The gradient magnitude after non-maximum suppression is compared with the high and low thresholds. The judgment criteria are: "strong edge": the gradient magnitude exceeds the high threshold; "weak edge": the gradient magnitude is between the low threshold and the high threshold. If it is lower than the low threshold, it is considered noise.

[0077] Furthermore, high and low thresholds are set to distinguish between "strong edges" and "weak edges," and weak edges are connected to strong edges to form complete edges. That is, "strong edges" are screened by using a hysteresis threshold, and "weak edges" are retained only if they meet the additional condition of being connected to strong edges, thus forming complete photovoltaic panel edges.

[0078] Step S503: Traverse the edge contours of the photovoltaic panel, filter and identify the edge contours of the photovoltaic panel to obtain the global photovoltaic panel corresponding to the target photovoltaic string. For details, please refer to step S403 of the embodiment shown in Figure 4, which will not be repeated here.

[0079] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning. By preprocessing the ROI image, noise can be removed and contrast enhanced, providing a cleaner image with more prominent features for edge detection, thus improving the quality and stability of edge detection. Then, edge detection is performed on the preprocessed ROI image to extract the edge contour of the photovoltaic panel, laying the foundation for accurate recognition of the photovoltaic panel in the future.

[0080] This embodiment provides a two-stage photovoltaic panel recognition method based on deep learning, which can be used in the aforementioned terminal device. Figure 6 is a flowchart of the two-stage photovoltaic panel recognition method based on deep learning according to an embodiment of the present invention. As shown in Figure 6, the process includes the following steps: Step S601, acquiring a target photovoltaic string image; based on the target photovoltaic string image, using a deep learning model to detect the photovoltaic string and generating an ROI image corresponding to the target photovoltaic string. For details, please refer to step S501 of the embodiment shown in Figure 5, which will not be repeated here.

[0081] Step S602: Perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel. For details, please refer to step S502 of the embodiment shown in Figure 5, which will not be repeated here.

[0082] Step S603: Traverse the edge contours of the photovoltaic panel, filter and identify the edge contours of the photovoltaic panel, and obtain the global photovoltaic panel corresponding to the target photovoltaic string.

[0083] Specifically, step S603 includes: step S6031, traversing the edge contour of the photovoltaic panel and calculating the size of the outer rectangle corresponding to the edge contour of the photovoltaic panel.

[0084] Specifically, iterate through all the edge contours of the photovoltaic panels and calculate the length of the bounding rectangle corresponding to the edge contour of each photovoltaic panel. ,Width ,area and aspect ratio .

[0085] Step S6032: Compare the size of the outer rectangle with a preset threshold range. If the size of the outer rectangle meets the preset threshold range, then the edge contour of the photovoltaic panel corresponding to the size of the outer rectangle is determined as a valid photovoltaic panel.

[0086] Specifically, if the area and aspect ratio of the outer radius corresponding to the edge contour of the photovoltaic panel are within a preset threshold range, then the edge contour of the photovoltaic panel is determined to be a valid photovoltaic panel, and its number is counted. .

[0087] Step S6033: Obtain the valid photovoltaic panels corresponding to the target photovoltaic string, and determine the global photovoltaic panels based on the valid photovoltaic panels corresponding to the target photovoltaic string.

[0088] Specifically, all valid photovoltaic panels are counted to form the global photovoltaic panel, and the number of global photovoltaic panels is... The sum of the number of effective photovoltaic panels across all ROI regions can be expressed as: (11) Among them, This indicates the number of ROI regions.

[0089] This embodiment provides a two-stage photovoltaic panel identification method based on deep learning. By using the dual judgment criteria of the circumscribed rectangle size and the preset threshold range, it can effectively filter out interference contours such as noise, shadows, and stains, avoid the subjectivity of manual judgment, reduce the false detection rate, and then determine the global photovoltaic panel based on the valid photovoltaic panels corresponding to the target photovoltaic string, thus achieving accurate identification of photovoltaic panels.

[0090] This embodiment also provides a two-stage photovoltaic panel identification device based on deep learning, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0091] This embodiment provides a two-stage photovoltaic panel recognition device based on deep learning, as shown in Figure 7. It includes: a photovoltaic string detection module 701, used to acquire a target photovoltaic string image, and based on the target photovoltaic string image, use a deep learning model to detect the photovoltaic string and generate a ROI image corresponding to the target photovoltaic string; an edge detection module 702, used to perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel; and a recognition module 703, used to traverse the edge contour of the photovoltaic panel, filter and recognize the edge contour of the photovoltaic panel, and obtain the global photovoltaic panel corresponding to the target photovoltaic string.

[0092] In some optional implementations, the photovoltaic string detection module 701 includes: an identification unit for inputting the target photovoltaic string image into the YOLO model to obtain the bounding box coordinates of the target photovoltaic string; and a removal unit for removing overlapping bounding boxes from the bounding box coordinates of the target photovoltaic string to generate an ROI image corresponding to the target photovoltaic string.

[0093] In some optional implementations, the edge detection module 702 includes: an image preprocessing unit for preprocessing the ROI image to obtain a preprocessed ROI image; and an edge detection unit for performing edge detection on the preprocessed ROI image to extract the edge contour of the photovoltaic panel.

[0094] In some optional implementations, the image preprocessing unit includes: a normalization subunit for normalizing the size of the ROI image to obtain a fixed-size ROI image; a filtering subunit for performing HSV threshold separation on the fixed-size ROI image to obtain a filtered image; an image binarization subunit for performing image binarization processing on the filtered image to obtain a binary image; and a denoising subunit for performing morphological operations on the binarized image to obtain a preprocessed ROI image.

[0095] In some optional implementations, the edge detection unit includes: a Gaussian filtering subunit for performing Gaussian filtering on the preprocessed ROI image to obtain a Gaussian-filtered image; a convolution subunit for convolving the pixel values ​​corresponding to the Gaussian-filtered image with a horizontal gradient kernel and a vertical gradient kernel respectively to obtain a horizontal gradient and a vertical gradient; a calculation subunit for calculating the gradient magnitude and gradient direction based on the horizontal and vertical gradients respectively; a generation subunit for finding local maxima pixels along the gradient direction, retaining the gradient magnitude corresponding to the local maxima pixels, and generating a non-maximum suppressed gradient magnitude; and a comparison subunit for comparing the non-maximum suppressed gradient magnitude with a hysteresis threshold, filtering edges with a single pixel width based on the comparison result, and obtaining the photovoltaic panel edge contour; wherein the photovoltaic panel edge contour is a closed contour.

[0096] In some optional implementations, the identification module 703 includes: a traversal unit for traversing the edge contour of the photovoltaic panel and calculating the size of the outer rectangle corresponding to the edge contour of the photovoltaic panel; a comparison unit for comparing the size of the outer rectangle with a preset threshold range, and if the size of the outer rectangle meets the preset threshold range, then the edge contour of the photovoltaic panel corresponding to the size of the outer rectangle is determined as a valid photovoltaic panel; and a determination unit for obtaining the valid photovoltaic panels corresponding to the target photovoltaic string and determining the global photovoltaic panels based on the valid photovoltaic panels corresponding to the target photovoltaic string.

[0097] The two-stage photovoltaic panel identification device based on deep learning provided in this embodiment of the invention can execute the two-stage photovoltaic panel identification method based on deep learning provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments, and will not be repeated here.

[0098] Figure 8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0099] Referring specifically to Figure 8, a schematic diagram of a suitable electronic device for implementing embodiments of the present invention is shown below. The electronic device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from memory 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0100] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although FIG8 shows an electronic device with various devices, it should be understood that it is not required to implement or have all the devices shown, and more or fewer devices may be implemented or have alternatively.

[0101] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by the processor 801, it performs the functions defined in the deep learning-based two-stage photovoltaic panel identification method of the present invention.

[0102] The electronic device shown in Figure 8 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0103] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, it implements the two-stage photovoltaic panel identification method based on deep learning shown in the above embodiments.

[0104] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0105] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A two-stage photovoltaic panel identification method based on deep learning, characterized in that, The method includes: acquiring a target photovoltaic string image; based on the target photovoltaic string image, performing photovoltaic string detection using a deep learning model to generate a ROI image corresponding to the target photovoltaic string; performing edge detection on the ROI image to obtain the edge contour of the photovoltaic panel; traversing the edge contour of the photovoltaic panel, filtering and recognizing the edge contour of the photovoltaic panel to obtain the global photovoltaic panel corresponding to the target photovoltaic string.

2. The method according to claim 1, characterized in that, The deep learning model adopts the YOLO model; the step of using the deep learning model to detect photovoltaic strings based on the target photovoltaic string image and generate the ROI image corresponding to the target photovoltaic string includes: inputting the target photovoltaic string image into the YOLO model to obtain the bounding box coordinates of the target photovoltaic string; removing overlapping bounding boxes from the bounding box coordinates of the target photovoltaic string to generate the ROI image corresponding to the target photovoltaic string.

3. The method according to claim 1, characterized in that, The step of performing edge detection on the ROI image to obtain a closed contour includes: performing image preprocessing on the ROI image to obtain a preprocessed ROI image; and performing edge detection on the preprocessed ROI image to extract the edge contour of the photovoltaic panel.

4. The method according to claim 3, characterized in that, The step of preprocessing the ROI image to obtain a preprocessed ROI image includes: normalizing the size of the ROI image to obtain a fixed-size ROI image; performing HSV thresholding on the fixed-size ROI image to obtain a filtered image; performing image binarization on the filtered image to obtain a binarized image; and performing morphological operations on the binarized image to denoise, thereby obtaining the preprocessed ROI image.

5. The method according to claim 3, characterized in that, The step of edge detection in the preprocessed ROI image to extract the edge contour of the photovoltaic panel includes: performing Gaussian filtering on the preprocessed ROI image to obtain a Gaussian-filtered image; convolving the pixel values ​​corresponding to the Gaussian-filtered image with a horizontal gradient kernel and a vertical gradient kernel respectively to obtain a horizontal gradient and a vertical gradient; calculating the gradient magnitude and gradient direction based on the horizontal gradient and the vertical gradient respectively; finding local maxima pixels along the gradient direction, retaining the gradient magnitude corresponding to the local maxima pixels, and generating a non-maximum suppressed gradient magnitude; comparing the non-maximum suppressed gradient magnitude with a hysteresis threshold, and filtering out single-pixel-wide edges based on the comparison result to obtain the edge contour of the photovoltaic panel; wherein, the edge contour of the photovoltaic panel is a closed contour.

6. The method according to claim 1, characterized in that, The step of traversing the edge contours of the photovoltaic panel and filtering and identifying the edge contours of the photovoltaic panel to obtain the global photovoltaic panel corresponding to the target photovoltaic string includes: traversing the edge contours of the photovoltaic panel and calculating the size of the outer rectangle corresponding to the edge contour of the photovoltaic panel; comparing the size of the outer rectangle with a preset threshold range; if the size of the outer rectangle meets the preset threshold range, then determining the edge contour of the photovoltaic panel corresponding to the size of the outer rectangle as a valid photovoltaic panel; obtaining the valid photovoltaic panel corresponding to the target photovoltaic string, and determining the global photovoltaic panel based on the valid photovoltaic panel corresponding to the target photovoltaic string.

7. A two-stage photovoltaic panel identification device based on deep learning, characterized in that, The device includes: a photovoltaic string detection module, used to acquire a target photovoltaic string image, and based on the target photovoltaic string image, to perform photovoltaic string detection using a deep learning model to generate a ROI image corresponding to the target photovoltaic string; an edge detection module, used to perform edge detection on the ROI image to obtain the edge contour of the photovoltaic panel; and a recognition module, used to traverse the edge contour of the photovoltaic panel, filter and recognize the edge contour of the photovoltaic panel, and obtain the global photovoltaic panel corresponding to the target photovoltaic string.

8. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the two-stage photovoltaic panel identification method based on deep learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the two-stage photovoltaic panel identification method based on deep learning as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The method includes computer instructions for causing a computer to execute the two-stage photovoltaic panel identification method based on deep learning as described in any one of claims 1 to 6.

Images