Centralized photovoltaic panel identification method and device, electronic equipment and storage medium

Abstract

The invention provides a centralized photovoltaic panel identification method and device, electronic equipment and a storage medium, and belongs to the field of image processing. The method comprises the following steps: obtaining a detection frame image and an inclination angle of each photovoltaic panel according to a remote sensing image of the photovoltaic panel; according to the detection frame, obtaining a binary image corresponding to the detection frame image, and segmenting the binary image into a plurality of binary sub-images, each binary sub-image including an edge contour of a complete photovoltaic panel; and based on the inclination angle of the complete photovoltaic panel in the binary sub-image, carrying out edge fitting on the binary sub-image to obtain a bounding box of the complete photovoltaic panel. Thus, on the basis that the detection frame is obtained through detection, the image is further binarized and segmented, and then edge fitting is performed on the binary sub-image containing a complete photovoltaic panel according to the inclination angle of the photovoltaic panel, so that the interference of edge blur processing and photovoltaic panel angle inclination is reduced, the obtained photovoltaic panel bounding box is more accurate, and the accuracy of the photovoltaic panel bounding box is improved. And the precision and the accuracy of photovoltaic panel detection and identification are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to a method, device, electronic device and storage medium for identifying centralized photovoltaic panels. Background Art

[0002] With the continuous progress of photovoltaic power generation technology, the installed capacity of solar power generation has increased significantly, which further optimizes the energy structure and accelerates the pace of green and low-carbon transformation. The inspection of photovoltaic panels is a key link to ensure the normal operation of photovoltaic power stations and improve power generation efficiency. Given the rapid growth of the installed capacity of solar power generation, intelligent inspection by unmanned aerial vehicles (UAVs) has become an important means for the inspection of photovoltaic power stations.

[0003] Intelligent inspection by UAVs collects image or video information of a photovoltaic power station through UAVs equipped with lenses of different bands, and then detects and identifies this image or video information through manual or intelligent methods to understand the status of each photovoltaic panel in the photovoltaic power station. Networks such as YOLOv4 and YOLOv5, as well as semantic segmentation methods such as U-Net and DeepLabV3, have been widely used in the detection and identification of photovoltaic panel images or videos. However, these methods have poor recognition effects when facing images containing multiple photovoltaic panels. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for identifying centralized photovoltaic panels, which improves the problem of poor recognition effect of photovoltaic panel images and greatly improves the detection accuracy and precision.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for identifying centralized photovoltaic panels, the method comprising:

[0007] Obtaining a detection frame image and angle information according to a remote sensing image of a photovoltaic panel; wherein, the detection frame image includes detection frames of each photovoltaic panel, and the angle information includes the tilt angle of each photovoltaic panel;

[0008] Obtaining a binary image corresponding to the detection frame image based on the detection frame;

[0009] Segmenting the binary image to obtain multiple binary sub-images; wherein, each binary sub-image contains the edge contour of a complete photovoltaic panel;

[0010] For each binary sub-image, based on the tilt angle of the complete photovoltaic panel in the binary sub-image, performing edge fitting on the binary sub-image to obtain a bounding box of the complete photovoltaic panel.

[0011] Optionally, the step of performing edge fitting on the binary sub - graph based on the tilt angle of the complete photovoltaic panel in the binary sub - graph to obtain the bounding box of the complete photovoltaic panel includes:

[0012] Perform smoothing correction on the binary sub - graph;

[0013] According to the tilt angle of the complete photovoltaic panel in the binary sub - graph, obtain a plurality of random rectangles and the overlap values of the random rectangles; wherein, the random rectangles have the same center point and tilt angle as the complete photovoltaic panel, and the overlap value represents the pixel overlap degree between the random rectangle and the edge contour in the binary sub - graph;

[0014] According to the overlap value, obtain the bounding box of the complete photovoltaic panel from the plurality of random rectangles.

[0015] Optionally, the step of obtaining a plurality of random rectangles and the overlap values of the random rectangles according to the tilt angle of the complete photovoltaic panel in the binary sub - graph includes:

[0016] According to the tilt angle of the complete photovoltaic panel in the binary sub - graph, obtain a random rectangle;

[0017] According to the overlapping pixels between the random rectangle and the edge contour in the binary sub - graph, obtain the overlap value of the random rectangle;

[0018] When the overlap value of the random rectangle is less than the coverage threshold and the existing random rectangles do not meet the rectangle upper limit, adjust the size of the random rectangle to obtain the next random rectangle, and return to execute the step of obtaining the overlap value of the random rectangle according to the overlapping pixels between the random rectangle and the edge contour in the binary sub - graph;

[0019] When the overlap value of the random rectangle is not less than the coverage threshold, or the existing random rectangles meet the rectangle upper limit, end the edge fitting.

[0020] Optionally, the edge contour in the binary sub - graph includes a plurality of edge pixels, and the random rectangle includes a plurality of boundary pixels;

[0021] The step of obtaining the overlap value of the random rectangle according to the overlapping pixels between the random rectangle and the edge contour in the binary sub - graph includes:

[0022] According to the coordinates of the edge pixels and the coordinates of the boundary pixels, obtain the overlapping pixels;

[0023] According to the total number of the overlapping pixels and the total number of the edge pixels, obtain the overlap value between the random rectangle and the edge contour in the binary sub - graph.

[0024] Optionally, the step of obtaining a binary image corresponding to the detected frame image according to the detected frame includes:

[0025] For each detected frame in the detected frame image, expand the detected frame to obtain a photovoltaic region containing a complete photovoltaic panel;

[0026] According to the photovoltaic region and the edge features of the photovoltaic panel, obtain an edge image containing the edge contours of each photovoltaic panel;

[0027] According to each of the edge contours, convert the edge image into a binary image.

[0028] Optionally, the step of obtaining an edge image containing the edge contours of each photovoltaic panel according to the photovoltaic region and the edge features of the photovoltaic panel includes:

[0029] Input the image of the photovoltaic region into an edge detection network to obtain an intermediate image containing the initial edges of each photovoltaic region; wherein, the edge detection network is a model obtained by learning and training the edge features of the photovoltaic panel;

[0030] Erode and dilate each initial edge in the intermediate image to obtain an edge image containing the edge contours of each photovoltaic panel.

[0031] Optionally, the step of obtaining a detected frame image and tilt angle information according to the remote sensing image of the photovoltaic panel includes:

[0032] Input the remote sensing image of the photovoltaic panel into a target detection model to obtain a detected frame image and angle information; wherein, the target detection model is obtained by training an improved YOLO network, and the improved YOLO network is obtained by adding a spatial depth convolution module to the backbone network of the YOLOv11 network, and the spatial depth convolution module is used to convert the spatial dimension of the feature map into the depth dimension.

[0033] In a second aspect, the present invention provides a centralized photovoltaic panel recognition device, including a preliminary detection module, a binarization module, a cropping module, and an edge fitting module;

[0034] The preliminary detection module is used to obtain a detected frame image and angle information according to the remote sensing image of the photovoltaic panel; wherein, the detected frame image includes the detected frames of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel;

[0035] The binarization module is used to obtain a binary image corresponding to the detected frame image according to the detected frame;

[0036] The cropping module is used to segment the binary image to obtain multiple binary sub-images; wherein, each binary sub-image contains the edge contour of a complete photovoltaic panel;

[0037] The edge fitting module is configured to perform edge fitting on each of the binary sub - graphs based on the inclination angle of the complete photovoltaic panel in the binary sub - graph, so as to obtain the bounding box of the complete photovoltaic panel.

[0038] In a third aspect, the present invention provides an electronic device, including a processor and a memory. The memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the centralized photovoltaic panel recognition method as described in the first aspect above.

[0039] In a fourth aspect, the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the centralized photovoltaic panel recognition method as described in the first aspect above.

[0040] The centralized photovoltaic panel recognition method, device, electronic device and storage medium provided by the embodiments of the present invention. The method includes: obtaining a detection box image and angle information according to a remote sensing image of a photovoltaic panel. The detection box image includes detection boxes of each photovoltaic panel, and the angle information includes the inclination angles of each photovoltaic panel; obtaining a binary image corresponding to the detection box image based on the detection box, and performing segmentation to obtain multiple binary sub - graphs, each binary sub - graph containing the edge contour of a complete photovoltaic panel; for each binary sub - graph, performing edge fitting on the binary sub - graph based on the inclination angle of the complete photovoltaic panel in the binary sub - graph to obtain the bounding box of the complete photovoltaic panel. In this way, on the basis of detecting the detection boxes of each photovoltaic panel in the image, the image is further binarized and segmented, and then, based on the inclination angle of the photovoltaic panel, edge fitting is performed on the binary sub - graph containing a complete photovoltaic panel, thereby reducing the interference of blurred edges and inclined photovoltaic panel angles, making the obtained bounding box of the photovoltaic panel more accurate, and improving the accuracy and precision of photovoltaic panel detection and recognition.

[0041] To make the above - mentioned objects, features and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 Shows a schematic diagram of the system architecture of the centralized photovoltaic panel recognition system provided by the embodiments of the present invention.

[0044] Figure 2 Shows a schematic diagram of the module architecture of the electronic device provided by an embodiment of the present invention.

[0045] Figure 3 Shows a schematic flow diagram of the centralized photovoltaic panel recognition method provided by an embodiment of the present invention.

[0046] Figure 4 Shows a structural diagram of the backbone network of the improved YOLO network.

[0047] Figure 5 Shows Figure 3 A schematic flow diagram of partial sub-steps of step 13 in

[0048] Figure 6 Shows Figure 5 A schematic flow diagram of partial sub-steps of step 133 in

[0049] Figure 7 Shows a comparison diagram of a remote sensing image and a binary image.

[0050] Figure 8 Shows Figure 3 A schematic flow diagram of partial sub-steps of step 17 in

[0051] Figure 9 Shows Figure 8 A schematic flow diagram of partial sub-steps of step 173 in

[0052] Figure 10 Shows Figure 9 A schematic flow diagram of partial sub-steps of step 1733 in

[0053] Figure 11 Shows a schematic diagram of the remote sensing image and its detail image in the experiment.

[0054] Figure 12 Shows the detection result diagram of the common YOLO method.

[0055] Figure 13 Shows the detection result diagram of the method provided by this embodiment.

[0056] Figure 14 Shows a schematic diagram of the module architecture of the centralized photovoltaic panel recognition device provided by an embodiment of the present invention.

[0057] Icons: 10 - Centralized photovoltaic panel recognition system; 110 - Recognition device; 120 - Inspection device; 20 - Electronic device; 210 - Memory; 220 - Processor; 230 - Communication module; 30 - Centralized photovoltaic panel recognition device; 310 - Preliminary detection module; 320 - Binarization module; 330 - Cropping module; 340 - Edge fitting module. Detailed implementation manners

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0059] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0060] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0061] The centralized photovoltaic panel identification method provided by the embodiments of the present invention can be applied to Figure 1 the centralized photovoltaic panel identification system 10 as shown in the figure. The centralized photovoltaic panel identification system 10 includes an identification device 110 and an inspection device 120. The identification device 110 can communicate with the inspection device 120 through wired or wireless connection methods such as Ethernet and WiFi.

[0062] Among them, the inspection device 120 can be: a drone, an inspection robot, a camera installed in a photovoltaic power station, or any device that can automatically perform inspections to obtain images and videos. The identification device 110 can be an independent server, a server cluster, a personal computer, a laptop computer, or any computer device.

[0063] The inspection device 120 is used to perform inspections and shootings on the photovoltaic power station to obtain multiple remote sensing images containing photovoltaic panels, and transmit the remote sensing images to the identification device 110.

[0064] The recognition device 110 is used to implement the centralized photovoltaic panel recognition method provided by the embodiments of the present invention, including: obtaining a detection frame image and angle information according to the remote sensing image of the photovoltaic panel, where the detection frame image includes detection frames of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel; obtaining a binary image corresponding to the detection frame image based on the detection frame, and performing segmentation to obtain multiple binary sub-images, each binary sub-image containing the edge contour of a complete photovoltaic panel; for each binary sub-image, based on the tilt angle of the complete photovoltaic panel in the binary sub-image, performing edge fitting on the binary sub-image to obtain the bounding box of the complete photovoltaic panel.

[0065] Please refer to Figure 2 , which is a block diagram of the electronic device 20. The electronic device 20 may be Figure 1 the recognition device 110 in the centralized photovoltaic panel recognition system 10 shown. The electronic device 20 includes a memory 210, a processor 220, and a communication module 230. The elements of the memory 210, the processor 220, and the communication module 230 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these elements may be electrically connected to each other through one or more communication buses or signal lines.

[0066] Among them, the memory 210 is used to store programs or data. The memory 210 may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable read-only memory, an electrically erasable read-only memory, etc.

[0067] The processor 220 is used to read / write the data or programs stored in the memory 210 and execute corresponding functions. For example, Figure 1 in the centralized photovoltaic panel recognition system 10 shown, the processor 220 of the recognition device 110 executes the computer program stored in the memory 210 to implement the centralized photovoltaic panel recognition method provided by the embodiments of the present invention.

[0068] The communication module 230 is used to establish a communication connection between the server and other communication terminals (such as the inspection device 120) through the network, and is used to send and receive data through the network. For example, Figure 1 in the centralized photovoltaic panel recognition system 10 shown, the communication module 230 of the recognition device 110 is used to receive remote sensing images through the network and send data.

[0069] It should be understood that Figure 2 the structure shown is only a schematic diagram of the electronic device 20, and the electronic device 20 may also include more or fewer components than Figure 2 shown, or have a different configuration from Figure 2 shown. Figure 2 The components shown in

[0070] To address the problem of poor recognition performance of common photovoltaic panel recognition methods when dealing with images containing multiple photovoltaic panels, an embodiment of the present invention provides a centralized photovoltaic panel recognition method. Referring to Figure 3 , it includes steps 11 to 17. And, Figure 1 In the centralized photovoltaic panel recognition system 10 shown in Figure 2 , when the recognition device 110 has the structure shown in

[0071] Step 11, based on the remote sensing image of the photovoltaic panel, obtain the detection box image and the angle information.

[0072] Among them, the detection box image includes the detection boxes of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel.

[0073] Step 13, based on the detection boxes, obtain the binary image corresponding to the detection box image.

[0074] Step 15, segment the binary image to obtain multiple binary sub-images.

[0075] Among them, each binary sub-image contains the edge contour of a complete photovoltaic panel.

[0076] Step 17, for each binary sub-image, based on the tilt angle of the complete photovoltaic panel in the binary sub-image, perform edge fitting on the binary sub-image to obtain the bounding box of the complete photovoltaic panel.

[0077] Exemplarily, in combination with Figure 1 the centralized photovoltaic panel recognition system 10 shown in

[0078] one or more inspection devices 120 regularly inspect the photovoltaic power station and take pictures, obtaining multiple remote sensing images whose total shooting range covers the entire photovoltaic power station. Each remote sensing image is an image containing photovoltaic panels, and any two remote sensing images are different. After each inspection is completed or during the inspection process, the inspection device 120 transmits the remote sensing images obtained from the inspection shooting to the recognition device 110.

[0079] Finally, the boundary boxes obtained from the binary sub - images are synthesized to obtain the recognition result of the photovoltaic panel boundary of the photovoltaic power station.

[0080] In steps 11 to 17 of the above - mentioned centralized photovoltaic panel recognition method, based on the detection boxes of each photovoltaic panel in the detected image, the image is further binarized and segmented, and then according to the inclination angle of the photovoltaic panel, edge fitting is performed on the binary sub - image containing a complete photovoltaic panel, thereby reducing the interference of blurred edges and inclined photovoltaic panel angles, making the obtained photovoltaic panel boundary boxes more accurate, and improving the accuracy and precision of photovoltaic panel detection and recognition.

[0081] Among them, in step 11, the method of obtaining the detection box image and angle information can be flexibly selected. For example, the remote - sensing image can be input into the detection model trained by the YOLOv5 network to obtain the detection box image and angle information output by the detection model. It can also be that the remote - sensing image is input into the detection model trained by Faster R - CNN, and the detection box image and angle information output by this detection model. And the above methods are only examples, and their implementation methods are not limited.

[0082] YOLOv4 network, YOLOv5 network, and semantic segmentation networks such as U - Net and DeepLabV3 are common detection and recognition methods for photovoltaic panel images or videos. These networks are all trained by a large number of labeled photovoltaic panel data to learn the features of photovoltaic panels from images. Among them, the YOLO series uses multi - scale detection layers to improve the detection ability for objects of different scales. U - Net extracts and restores image features at different levels through an encoder - decoder structure. DeepLabV3 + uses dilated convolution and multi - scale feature extraction to enhance the network's sensitivity to objects of different scales.

[0083] Although the YOLO series of networks perform excellently in terms of real - time performance and speed, they are only applicable to simple scenarios with less background interference. In complex backgrounds, especially when there are various interfering objects or cluttered backgrounds around the photovoltaic panels, their ability to extract object details is weak, and it is difficult to process the detail parts of photovoltaic panels, such as edges and texture information, and it is very easy to produce false detections or missed detections, and both the detection accuracy and segmentation accuracy are relatively low.

[0084] Under different inspection environments, the sizes and shapes of photovoltaic panels vary greatly in remote - sensing images. The fixed structures of DeepLabV3 + and U - Net make their adaptability to different - scale targets poor. Especially when the size of the photovoltaic panel is small or blocked, the segmentation effect is poor.

[0085] In order to still obtain high-precision and high-accuracy detection box images and angle information under different target sizes, lighting conditions, background complexities, etc., in step 11, the idea of introducing a target detection model trained by the YOLOv11 network with a spatial depth convolution module added to detect remote sensing images is introduced.

[0086] Exemplarily, in step 11, the remote sensing image of the photovoltaic panel is input into the target detection model to obtain the detection box image and the tilt angle information. The target detection model is trained by an improved YOLO network, and the improved YOLO network is obtained by adding a spatial depth convolution module to the backbone network of the YOLOv11 network. The spatial depth convolution module is used to convert the spatial dimension of the feature map into the depth dimension.

[0087] Thus, the target detection model converts the spatial dimension of the feature map into the depth dimension through the spatial depth convolution module (which can also be called SPD convolution) to downsample the feature map without losing information and improve the ability to extract object detail information. Thus, the finally output detection box is more accurate.

[0088] The insertion position of the spatial depth convolution module on the backbone network of the YOLOv11 network can be flexibly selected, and its insertion position is not restricted.

[0089] In order to enhance the image processing quality of the network while ensuring the processing speed, referring to Figure 4 , the backbone network of the improved YOLO network can include a first convolution module Conv1, a second convolution module Conv2, a first C3k2 module C3k2_1, a third convolution module Conv3, a second C3k2 module C3k2_2, a first SPD convolution, a fourth convolution module Conv4, a third C3k2 module C3k2_3, a second SPD convolution, a fifth convolution module Conv5, a fourth C3k2 module C3k2_4, an SPPF layer, and a C3PSA layer connected in sequence.

[0090] Among them, Conv performs a convolution operation on the input data and the convolution kernel and outputs a new feature map. The C3k2 module extracts the features of the input data on different channels through parallel convolution and fuses the extracted features to obtain a new feature map. The SPD convolution converts the spatial dimension of the feature map into the depth dimension. The SPPF layer pools the features of different regions of the input data (feature map) by combining pooling operations of different scales to obtain a new feature map. The C3PSA layer enhances the importance of specific regions in the feature map by introducing a spatial attention mechanism, extracts the features of the input map with channels of different weights, and fuses the extracted features to obtain a new feature map, enabling the model to more effectively focus on the key regions in the image.

[0091] With the above structure, the SPD convolution is set after the second C3k2 module C3k2_2, so that the feature map input to the SPD convolution is the map after multiple downsamplings, that is, a map with a small amount of data (few pixel numbers), thus greatly reducing the computational amount of the SPD convolution, and then being able to improve the image processing speed and processing efficiency. In this way, both the image processing quality of the network can be enhanced and the processing efficiency can be ensured.

[0092] Furthermore, by training the improved YOLO network mentioned above, a target detection model can be obtained. Among them, the process of model training can include: preprocessing the photovoltaic panel dataset, including labeling with labelme software and data augmentation, etc., to obtain a dataset of labeled samples; using the processed dataset to iteratively train the improved YOLO network to obtain a mature target detection model. The target detection model can infer the position of the detection frame, the tilt angle and its confidence value of the photovoltaic panel in the remote sensing image.

[0093] After the detection frame image and angle information are inferred by using the above target detection model in step 11, in step 13, the pixels covered by the edge contour can be set to white and the remaining pixels can be set to black with the detection frame as the edge contour to obtain a binary image. It can also be converted into a binary image after being optimized first according to a preset rule, and its implementation method is not limited.

[0094] To make the edge contours of each photovoltaic panel more complete in the binary sub-image and avoid the incomplete contours of some photovoltaic panels, the idea of first expanding the detection frame, then performing edge detection, and finally converting it into a binary image is introduced in step 13. Refer to Figure 5 , the process of obtaining the binary image in step 13 includes steps 131 to 135.

[0095] Step 131, for each detection frame in the detection frame image, expand the detection frame to obtain a photovoltaic region containing a complete photovoltaic panel.

[0096] Step 133, according to the photovoltaic region and the edge features of the photovoltaic panel, obtain an edge image containing the edge contours of each photovoltaic panel.

[0097] Step 135, based on each edge contour, convert the edge image into a binary image.

[0098] In step 131, the detection frame can be expanded by any value or any ratio. It can also be expanded in a preset direction at a set ratio. And the above methods are all examples, and its implementation method is not limited.

[0099] To better capture the edges of the photovoltaic panels and prevent the loss of some edge information, ensuring a more comprehensive coverage of the photovoltaic area of the photovoltaic panels while simplifying the complexity of external expansion, in step 131, pixels can be expanded along the four directions of up, down, left, and right of the detection frame. For example, for each detection frame in the detection frame image, according to a set external expansion ratio (which can be 5%, 10%, 1%, 8%, etc.), the width and height of the detection frame are respectively expanded to obtain a photovoltaic area containing a complete photovoltaic panel.

[0100] For example, if the detection frame is a 10*10 rectangle, after expanding by 10%, a 11*11 photovoltaic area is obtained, that is, the four vertex coordinates of the detection frame are each increased by 1 pixel.

[0101] After expanding the detection frame in the above manner, an image marked with each photovoltaic area is obtained, where the marking can be directly outlined with a rectangular frame or marked with coordinates. Compared with the original detection frame, the photovoltaic area obtained by external expansion includes more background information, ensuring that the photovoltaic panel in the photovoltaic area is a complete photovoltaic panel and avoiding missed detection.

[0102] In step 133, Canny edge detection, Sobel operator or any edge detection technology can be used to detect the photovoltaic area to obtain the edge contours of each photovoltaic panel, and its implementation method is not limited.

[0103] To quickly perform fine edge extraction on the expanded photovoltaic area, the idea of using RCF edge detection to perform edge extraction on the photovoltaic area is introduced in step 133. For example, referring to Figure 6 , the process of obtaining the edge image in step 133 includes steps 1331 to 1333.

[0104] In step 1331, the image of the photovoltaic area is input into the edge detection network to obtain an intermediate image containing the initial edges of each photovoltaic area.

[0105] In step 1333, the initial edges in the intermediate image are eroded and dilated to obtain an edge image containing the edge contours of each photovoltaic panel.

[0106] Here, the image of the photovoltaic area refers to the image marked with each photovoltaic area. The edge detection network (i.e., the RCF network) is a model obtained by learning and training the edge features of the photovoltaic panel. The training method can be any iterative training method and will not be elaborated here. The RCF network optimizes the extraction of edge features in the image and can accurately capture edge information in complex backgrounds. Therefore, RCF is used for edge extraction to perform fine edge extraction on the expanded photovoltaic area and obtain finer initial edges.

[0107] Erosion and dilation are morphological operations. Erosion and dilation are used to further clean up the edge area, making the edge contour in the edge image more refined and accurate. After the intermediate image is eroded and dilated, an edge image containing multiple edge contours is obtained, that is, the edge image at this time is a grayscale image.

[0108] In the edge image, the closer to the edge contour, the higher the grayscale intensity value of the pixel, and the farther away from the edge contour, the lower the grayscale intensity value of the pixel. This also means that the higher the grayscale intensity value, the more likely it is the edge of the photovoltaic panel.

[0109] Therefore, in step 135, for the edge image, pixels whose grayscale intensity values are greater than or equal to the grayscale threshold are set to white, and pixels whose grayscale intensity values are lower than the grayscale threshold are set to black, so that a binary image can be obtained. At this time, the binary image includes multiple white edge contours (frames).

[0110] Through the above steps 131 to 135, the detection result of the target detection model (i.e., the detection frame image) is extended, edge extraction is performed using RCF edge detection, and binarization is performed, which further improves the detection accuracy and makes the edge contour in the binary image more refined and complete.

[0111] After obtaining the binary image, in order to reduce or eliminate the mutual interference of the edge contours of each photovoltaic panel and make the bounding box obtained after edge fitting in step 15 more accurate, in step 15, for the edge contour of each photovoltaic panel in each binary image, a rectangular frame containing only all the edges of the photovoltaic panel is determined, and the rectangular frame is used for cropping to obtain a binary sub-image.

[0112] Reference Figure 7 , Figure 7 (a) is a part of a complete photovoltaic panel in the remote sensing image. The binary sub-image containing the complete photovoltaic panel can be as follows Figure 7 As shown in (b), the image includes all the edges of the complete photovoltaic panel and the incomplete edges of some photovoltaic panels.

[0113] After obtaining the binary sub-image, in step 17, the method of performing edge fitting to obtain the bounding box can be flexibly set. For example, the binary sub-image and the tilt angle of the complete photovoltaic panel can be input into a pre-trained edge fitting model, and the edge fitting model can be used to infer the bounding box of the complete photovoltaic panel based on the edge fitting rules learned in the training. It is also possible to fit the binary sub-image according to preset rules to obtain the bounding box of the complete photovoltaic panel. The above two methods are examples, and their implementation methods are not limited.

[0114] In order to better handle the interference of edge blur and photovoltaic panel tilt angle, and obtain a more accurate bounding box, the edge fitting in step 17 introduces the concept of performing rectangle fitting according to the tilt angle of the complete photovoltaic panel, and obtaining the bounding box from multiple random rectangles fitted with overlapping values. Figure 8 The process of obtaining the bounding box in step 17 includes steps 171 to 175.

[0115] Step 171, performing smoothing correction on the binary sub-image.

[0116] Step 173 , obtaining a plurality of random rectangles and overlapping values of the random rectangles according to the tilt angle of the complete photovoltaic panel in the binary sub-image.

[0117] Among them, the random rectangle has the same center point and tilt angle as the complete photovoltaic panel, and the overlap value represents the pixel overlap between the random rectangle and the edge contour in the binary sub-image. That is, according to the center point and tilt angle of the complete photovoltaic panel, multiple random rectangles of different sizes are generated, and the overlap value of each random matrix is obtained.

[0118] Step 175 , obtaining a bounding box of the complete photovoltaic panel from a plurality of random rectangles according to the overlap value.

[0119] In the above step 171, the binary image is smoothed and corrected to remove the noise of the edge contour in the binary image and improve the continuity of the edge contour, so as to further improve the fitting accuracy.

[0120] The above steps 171 to 175 use rectangle fitting to better handle edge blur, photovoltaic panel tilt angle and other situations, and ensure the accuracy of the segmented boundary box.

[0121] Among them, in step 173, the maximum width and maximum length of the random rectangle can be obtained by using trigonometric functions based on the tilt angle and the length and width of the binary sub-image, and then a preset number (such as 50, 100, or any other value) of different random rectangles are generated with the maximum width and minimum length as restriction conditions, and the overlap value of each random rectangle is obtained. Alternatively, multiple random rectangles can be generated according to preset rules. The method of obtaining multiple random rectangles and their overlap values is not limited.

[0122] In order to make the obtained random rectangle contain the optimal rectangle as much as possible to ensure the accuracy of the bounding box, a dynamic adjustment concept of iteratively generating multiple random rectangles based on the overlap value is introduced in step 173. Figure 9 The process of obtaining multiple random rectangles and their overlapping values in step 173 includes steps 1731 to 1737.

[0123] Step 1731: Obtain a random rectangle based on the tilt angle of the complete photovoltaic panel in the binary sub - graph.

[0124] Step 1733: Obtain the overlap value of the random rectangle based on the overlapping pixels between the random rectangle and the edge contour in the binary sub - graph.

[0125] Step 1735: When the overlap value of the random rectangle is less than the coverage threshold and the existing random rectangles do not meet the rectangle upper limit, adjust the size of the random rectangle to obtain the next random rectangle, and return to execute Step 1733.

[0126] Step 1737: When the overlap value of the random rectangle is not less than the coverage threshold, or the existing random rectangles meet the rectangle upper limit, end the edge fitting.

[0127] Step 1731 can also be understood as initializing a random rectangle with a center point and a tilt angle consistent with the complete photovoltaic panel, and the width of the random rectangle does not exceed the maximum width of the random rectangle, and the length does not exceed the maximum length.

[0128] Furthermore, in Step 1733, any feasible implementation method can be used to calculate the overlap value of the random rectangle. For example, it can be to count the overlapping pixels between the pixels covered by the border of the random rectangle and the white pixels in the binary sub - graph to obtain the overlap value, or it can be to input the binary sub - graph marked with the random rectangle into a pre - trained overlap fitting model, and the model infers the overlap value. Its implementation method is not restricted.

[0129] Both the edge contour and the random rectangle will have pixels covered by the border. That is, the edge contour in the binary sub - graph includes multiple edge pixels, and the random rectangle includes multiple boundary pixels. The boundary pixels refer to the pixels covered by the border of the random rectangle, and the edge pixels refer to the pixels set to white in the binary sub - graph. To ensure the accuracy of the overlap value, the idea of obtaining overlapping pixels by coordinates is introduced in Step 1733. For example, referring to Figure 10 , the process of obtaining the overlap value in Step 1733 includes Step 33 - 1 to Step 33 - 3.

[0130] Step 33 - 1: Obtain the overlapping pixels according to the coordinates of the edge pixels and the coordinates of the boundary pixels.

[0131] Step 33 - 3: Obtain the overlap value between the random rectangle and the edge contour in the binary sub - graph according to the total number of overlapping pixels and the total number of edge pixels.

[0132] The edge pixels and boundary pixels with the same coordinates are the overlapping pixels, and the ratio of the total number of overlapping pixels to the total number of edge pixels is the overlap value between the border of the random rectangle and the edge contour in the binary sub - graph.

[0133] Through the above steps 33-1 to 33-3, an accurate overlapping value is obtained.

[0134] Furthermore, the overlapping value of the random rectangle can be compared with the coverage threshold. The coverage threshold can be a preset value. For example, it can be 95% or 97%, and its value is not limited. At the same time, it is determined whether the existing random rectangles meet the rectangle upper limit. The rectangle upper limit can be, and here the rectangle upper limit can be a set quantity threshold. At this time, if the overlapping value is less than the coverage threshold and none of the existing random rectangles is equal to the quantity threshold, the size of the random rectangle is adjusted.

[0135] The method of adjusting the size of the random rectangle can be flexibly set. For example, it can be randomly adjusted within the maximum length and maximum width of the random rectangle to generate the next random rectangle. It can also be pre-set with a length adjustment unit and a width adjustment unit. When adjusting the size of the random rectangle, a length adjustment unit is added to or subtracted from the length of the current random rectangle, and / or a width adjustment unit is added to or subtracted from the width of the random rectangle to obtain the next random rectangle.

[0136] When adjusting the size of the random rectangle according to the length adjustment unit and the width adjustment unit, the rectangle upper limit can also be the upper limit of the number corresponding to the maximum width and the upper limit of the number corresponding to the maximum length. When the number of random rectangles with different lengths is equal to the upper limit of the number corresponding to the maximum length and the number of random rectangles with different widths is equal to the upper limit of the number corresponding to the maximum width, the rectangle upper limit is reached. And the ratio of the maximum width to the width adjustment unit is used as the upper limit of the number corresponding to the maximum width, and the ratio of the maximum length to the length adjustment unit is used as the upper limit of the number corresponding to the maximum length.

[0137] Through the above steps 1731 to 1737, all possible optimal random rectangles are quickly searched, and interference such as edge blur and inclination of the photovoltaic panel angle can be better handled, further improving the accuracy of the bounding box.

[0138] After obtaining multiple random rectangles and their overlapping values in the above manner, in step 175, the random rectangle with the largest overlapping value is used as the bounding box of the complete photovoltaic panel.

[0139] Through the method of rectangle fitting, the edge contour of the photovoltaic panel is more accurately segmented, improving the segmentation accuracy, thus greatly improving the accuracy of the bounding box.

[0140] Through experiments, the above centralized photovoltaic panel recognition method is compared with common detection methods.

[0141] The remote sensing image is as Figure 11 (a) shown, and a partial detail view of the remote sensing image is as Figure 11(as shown in (b)). The common YOLO method is used to detect targets in the remote sensing image, and the results are as shown in Figure 12 . It can be seen that most of the detection boxes do not fit the original string boxes and there are omissions.

[0142] The centralized photovoltaic panel recognition method provided by the embodiment of the present invention is used to detect and recognize the photovoltaic panels, and the results are as shown in Figure 13 . It can be seen from Figure 13 that compared with the YOLO method for the scene, the centralized photovoltaic panel recognition method provided by the embodiment of the present invention can better handle situations such as blurred edges and complex backgrounds, and the accuracy of the segmentation area is greatly improved.

[0143] And through the comparison of the experimental statistical data, it is obtained that the centralized photovoltaic panel recognition method provided by the embodiment of the present invention has an average detection accuracy improvement of 12% in the detection of photovoltaic panels with complex backgrounds and small objects compared with common methods such as YOLOv4, YOLOv5, U-Net, and DeepLabV3+. Especially in an environment with uneven illumination, occlusion, and a lot of noise, the performance improvement is particularly significant, the segmentation result is more accurate, and the occurrence of false detection and missed detection can be effectively reduced.

[0144] Based on the same concept of the above-mentioned centralized photovoltaic panel recognition method, referring to Figure 14 , the embodiment of the present invention also provides a centralized photovoltaic panel recognition device 30, which includes a preliminary detection module 310, a binarization module 320, a cropping module 330, and an edge fitting module 340. The centralized photovoltaic panel recognition device 30 can be applied to the recognition device 110 in the centralized photovoltaic panel recognition system 10 as shown in Figure 1 .

[0145] The preliminary detection module 310 is used to obtain a detection box image and angle information according to the remote sensing image of the photovoltaic panel. Among them, the detection box image includes the detection boxes of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel.

[0146] The binarization module 320 is used to obtain a binary image corresponding to the detection box image based on the detection box.

[0147] The cropping module 330 is used to segment the binary image to obtain multiple binary sub-images. Among them, each binary sub-image contains the edge contour of a complete photovoltaic panel.

[0148] The edge fitting module 340 is used to perform edge fitting on each binary sub-image based on the tilt angle of the complete photovoltaic panel in the binary sub-image to obtain the bounding box of the complete photovoltaic panel.

[0149] Through the collaborative action of the preliminary detection module 310, the binarization module 320, the cropping module 330, and the edge fitting module 340, the above-mentioned centralized photovoltaic panel recognition device 30 further binarizes and segments the image on the basis of detecting the detection frames of each photovoltaic panel in the image, and then performs edge fitting on the binary sub-image containing a complete photovoltaic panel according to the inclination angle of the photovoltaic panel, thereby reducing the interference of blurred edges and inclined photovoltaic panel angles, making the obtained photovoltaic panel bounding box more accurate, and improving the accuracy and precision of photovoltaic panel detection and recognition.

[0150] For the specific implementation and effects of the centralized photovoltaic panel recognition device 30, reference can be made to the description of the implementation of the centralized photovoltaic panel recognition method in the above text. For example, for the specific implementation and effects of the preliminary detection module 310, reference can be made to the description of the relevant content in step 11 above; for the specific implementation and effects of the binarization module 320, reference can be made to the description of the relevant content in step 13 above; for the specific implementation and effects of the cropping module 330, reference can be made to the description of the relevant content in step 15 above; for the specific implementation and effects of the edge fitting module 340, reference can be made to the description of the relevant content in step 17 above, which will not be elaborated here.

[0151] In addition, each module of the above-mentioned centralized photovoltaic panel recognition device 30 can be implemented in whole or in part by software, hardware, and their combinations. The above-mentioned modules can be embedded in the processor 220 in the electronic device 20 in hardware form or independent of it, or stored in the memory 210 of the electronic device 20 in software form, so that the processor 220 can call and execute the operations corresponding to each of the above modules to implement the centralized photovoltaic panel recognition method provided above.

[0152] An embodiment of the present invention also provides an electronic device 20, including a processor 220 and a memory 210. The memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the centralized photovoltaic panel recognition method provided above.

[0153] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor 220, it implements the centralized photovoltaic panel recognition method provided above.

[0154] In summary, the centralized photovoltaic panel recognition method, device, electronic device, and storage medium provided by the embodiments of the present invention have at least the following beneficial effects:

[0155] (1) The present invention combines YOLOv11 with the SPD module. Through spatial depth convolution, the SPD module can effectively expand the receptive field, extract features at multiple scales, improve the ability to extract details and detect small objects, thus making the detection box and tilt angle more accurate. At the same time, the robustness and adaptability of the model to complex backgrounds and photovoltaic panels of different scales are enhanced;

[0156] (2) Based on the detection box output by the YOLOv11 model, combined with the RCF edge detection technology, the edge image is binarized, and then the photovoltaic panel is segmented with a more refined bounding box through rectangle fitting, fully considering factors such as different scales, complex backgrounds, and illumination changes of the photovoltaic panel, greatly improving the detection accuracy, segmentation accuracy, and stronger robustness.

[0157] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0158] In addition, the functional modules in each embodiment of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0159] When the above-described functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs.

[0160] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A centralized photovoltaic panel identification method, characterized in that, The method includes: Based on the remote sensing image of the photovoltaic panel, obtaining a detection box image and angle information; wherein, the detection box image includes detection boxes of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel; According to the detection boxes, obtaining a binary image corresponding to the detection box image; Segmenting the binary image to obtain multiple binary sub-images; wherein, each binary sub-image contains the edge contour of a complete photovoltaic panel; For each binary sub-image, based on the tilt angle of the complete photovoltaic panel in the binary sub-image, performing edge fitting on the binary sub-image to obtain the bounding box of the complete photovoltaic panel.

2. The centralized photovoltaic panel identification method according to claim 1, wherein The step of performing edge fitting on the binary sub-image based on the tilt angle of the complete photovoltaic panel in the binary sub-image to obtain the bounding box of the complete photovoltaic panel includes: Performing smoothing correction on the binary sub-image; According to the tilt angle of the complete photovoltaic panel in the binary sub-image, obtaining multiple random rectangles and the overlap values of the random rectangles; wherein, the random rectangles have the same center point and tilt angle as the complete photovoltaic panel, and the overlap value represents the pixel overlap degree between the random rectangle and the edge contour in the binary sub-image; According to the overlap value, obtaining the bounding box of the complete photovoltaic panel from multiple random rectangles.

3. The centralized photovoltaic panel identification method according to claim 2, wherein The step of obtaining multiple random rectangles and the overlap values of the random rectangles according to the tilt angle of the complete photovoltaic panel in the binary sub-image includes: According to the tilt angle of the complete photovoltaic panel in the binary sub-image, obtaining a random rectangle; According to the overlapping pixels between the random rectangle and the edge contour in the binary sub-image, obtaining the overlap value of the random rectangle; When the overlap value of the random rectangle is less than the coverage threshold and the existing random rectangles do not meet the rectangle upper limit, adjusting the size of the random rectangle to obtain the next random rectangle, and returning to execute the step of obtaining the overlap value of the random rectangle according to the overlapping pixels between the random rectangle and the edge contour in the binary sub-image; When the overlap value of the random rectangle is not less than the coverage threshold, or the existing random rectangles meet the rectangle upper limit, ending the edge fitting.

4. The centralized photovoltaic panel identification method according to claim 3, wherein, The edge contour in the binary sub-image includes multiple edge pixels, and the random rectangle includes multiple boundary pixels; The step of obtaining the overlap value of the random rectangle according to the overlapping pixels between the random rectangle and the edge contour in the binary sub-image includes: According to the coordinates of the edge pixels and the coordinates of the boundary pixels, obtaining the overlapping pixels; According to the total number of the overlapping pixels and the total number of the edge pixels, obtaining the overlap value between the random rectangle and the edge contour in the binary sub-image.

5. The centralized photovoltaic panel identification method according to any one of claims 1 to 4, characterized in that The step of obtaining a binary image corresponding to the detection box image according to the detection boxes includes: For each detection box in the detection box image, expanding the detection box to obtain a photovoltaic region containing a complete photovoltaic panel; According to the photovoltaic region and the edge features of the photovoltaic panel, obtaining an edge image containing the edge contours of each photovoltaic panel; According to each edge contour, converting the edge image into a binary image.

6. The centralized photovoltaic panel identification method according to claim 5, characterized in that, The step of obtaining an edge image including the edge contours of each photovoltaic panel according to the photovoltaic region and the edge features of the photovoltaic panel includes: Inputting the image of the photovoltaic region into an edge detection network to obtain an intermediate image including the initial edges of each photovoltaic region; wherein, the edge detection network is a model obtained by learning and training the edge features of the photovoltaic panel; Eroding and dilating each initial edge in the intermediate image to obtain an edge image including the edge contours of each photovoltaic panel.

7. The centralized photovoltaic panel identification method according to any one of claims 1 to 4, characterized in that The step of obtaining a detection box image and tilt angle information according to the remote sensing image of the photovoltaic panel includes: Inputting the remote sensing image of the photovoltaic panel into a target detection model to obtain a detection box image and angle information; wherein, the target detection model is obtained by training an improved YOLO network, and the improved YOLO network is obtained by adding a spatial depth convolution module to the backbone network of the YOLOv11 network, and the spatial depth convolution module is used to convert the spatial dimension of the feature map into the depth dimension.

8. A centralized photovoltaic panel identification device, characterized in that, Including a preliminary detection module, a binarization module, a cropping module, and an edge fitting module; The preliminary detection module is used to obtain a detection box image and angle information according to the remote sensing image of the photovoltaic panel; wherein, the detection box image includes the detection boxes of each photovoltaic panel, and the angle information includes the tilt angles of each photovoltaic panel; The binarization module is used to obtain a binary image corresponding to the detection box image based on the detection box; The cropping module is used to segment the binary image to obtain multiple binary sub-images; wherein, each binary sub-image includes the edge contour of a complete photovoltaic panel; The edge fitting module is used to perform edge fitting on each binary sub-image based on the tilt angle of the complete photovoltaic panel in the binary sub-image to obtain the bounding box of the complete photovoltaic panel.

9. An electronic device, characterized in that, Including a processor and a memory, the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the centralized photovoltaic panel recognition method according to any one of claims 1 to 7.

10. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the centralized photovoltaic panel recognition method according to any one of claims 1 to 7.

Images