Photovoltaic defect identification method and system based on unmanned aerial vehicle, and electronic equipment

Abstract

The invention provides a photovoltaic defect identification method and system based on an unmanned aerial vehicle and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: collecting an initial image of a photovoltaic array through the unmanned aerial vehicle; preprocessing the initial image to obtain a processed image; performing target detection on the processed image by using a yov11 model to obtain a first detection result, wherein the first detection result comprises first position information of the target photovoltaic string; cutting the processed image according to the first position information to obtain a target image; and detecting the target image by using a yov11-obb model to obtain a second detection result, and outputting the first position information, the defect type information of the target photovoltaic panel and the target row and column information when determining that a target defect exists in the target photovoltaic string. By implementing the technical scheme provided by the invention, the effect of improving the accuracy and efficiency of photovoltaic defect identification is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically to a photovoltaic defect recognition method, system and electronic equipment based on drones. Background Art

[0002] With the rapid development of the photovoltaic industry, the scale of photovoltaic power stations continues to expand, and the demand for inspection and maintenance of photovoltaic panels is increasing. In related technologies, photovoltaic defect detection mainly relies on manual inspections or fixed camera monitoring. Manual inspections are inefficient, labor-intensive, and prone to missed or false detections due to human factors. Fixed camera monitoring is limited by installation position and angle, making it difficult to fully cover the photovoltaic array, and its ability to identify defects in complex environments is limited. In addition, although related technologies include methods for using drones for photovoltaic inspections, the accuracy and efficiency of defect identification still need to be improved. For example, existing drone detection methods often use simple image acquisition followed by direct defect identification without effective image preprocessing and targeted target detection. This results in low defect identification accuracy in complex backgrounds and an inability to accurately output the specific location and type of defects. Summary of the Invention

[0003] In order to solve the above technical problems, the present application provides a photovoltaic defect identification method, system and electronic equipment based on drones.

[0004] In the first aspect, the present application provides a photovoltaic defect identification method based on a drone, comprising: using a drone to collect an initial image of a photovoltaic array; preprocessing the initial image to obtain a processed image; using the yolov11 model to perform target detection on the processed image to obtain a first detection result, the first detection result including the first position information of the target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; cropping the processed image according to the first position information to obtain a target image, wherein the target image includes the target photovoltaic string; using the yolov11-obb model to detect the target image to obtain a second detection result, and when the second detection result indicates that there is a target defect in the target photovoltaic string, outputting the first position information, the defect type information of the target photovoltaic panel and the target row and column information, wherein the target photovoltaic panel represents the photovoltaic panel with the target defect in the target photovoltaic string.

[0005] By adopting the above technical solution, using drones to collect images can avoid the problems of low efficiency of manual inspections, easy missed detections and false detections, and incomplete coverage of fixed camera monitoring; preprocessing of the initial image can optimize the image quality; using the yolov11 model for target detection can obtain the first position information of the target photovoltaic string, so that the target image can be obtained by subsequent image cropping, and the target image can focus on the target photovoltaic string; using the yolov11-obb model to detect the target image, and when the target defect is detected, the first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel, and the target row and column information of the target photovoltaic panel are output, thereby realizing the accurate identification and positioning of photovoltaic defects, improving the accuracy and efficiency of photovoltaic defect identification, and accurately outputting the specific location and type information of the defect.

[0006] Optionally, the yolov11-obb model is used to detect the target image to obtain a second detection result. When the second detection result indicates that there is a target defect in the target photovoltaic string, the first position information, the defect type information of the target photovoltaic panel and the target row and column information are output, including: the yolov11-obb model uses a rotating frame to detect a group of photovoltaic panels in the target image to obtain a set of position information, and identifies the local image corresponding to each rotating frame to obtain a set of recognition results, wherein the target photovoltaic string includes a group of photovoltaic panels, and the second detection result includes a set of position information and a set of recognition results; when a set of recognition results indicates that there is a target defect in the target photovoltaic panel, the target row and column information of the target photovoltaic panel is determined according to a set of position information, and the first position information, defect type information and target row and column information are output, wherein a set of recognition results includes defect type information.

[0007] By adopting the above technical solution, the yolov11-obb model is used to detect a group of photovoltaic panels in the target image through a rotating frame to obtain a group of position information, and the local image corresponding to the rotating frame is identified to obtain a group of recognition results. This can more accurately locate the position of the photovoltaic panel and identify defects. When it is determined that the target photovoltaic panel has a target defect, the target row and column information of the target photovoltaic panel is determined according to the set of position information. The first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel and the target row and column information of the target photovoltaic panel can be output, which effectively improves the accuracy of photovoltaic defect recognition and the integrity of the output information, and solves the problems of low defect recognition accuracy and inability to accurately output the specific location and type information of the defect in related technologies.

[0008] Optionally, target row and column information of a target photovoltaic panel is determined based on a set of position information, including: determining the center coordinates of each rotation frame based on a set of position information, wherein each photovoltaic panel in a group of photovoltaic panels corresponds to a rotation frame; determining the arrangement direction of each photovoltaic panel in a group of photovoltaic panels based on the long side direction of each rotation frame; sorting according to the x or y value of the center coordinate of each rotation frame to obtain a row and column distribution result of a group of photovoltaic panels, wherein the row and column distribution result includes the target row and column information.

[0009] By adopting the above technical solution, the center coordinates of each rotating frame are determined according to a set of position information, which can provide basic data for subsequent calculations; the arrangement direction of the photovoltaic panels is determined according to the long side direction of the rotating frame, which can clearly know the arrangement of the photovoltaic panels; the row and column distribution results are obtained by sorting the x or y values ​​of the center coordinates of the rotating frame to determine the target row and column information, which helps to accurately determine the position of the target photovoltaic panel in the photovoltaic string, thereby improving the accuracy and efficiency of photovoltaic defect identification and solving the problem that it is difficult to accurately output the specific location information of the defects in existing detection methods.

[0010] Optionally, the first position information includes first center coordinate information, first length information and first width information; a set of position information includes multiple second position information, each photovoltaic panel in a set of photovoltaic panels corresponds to a second position information, and the second position information includes second center coordinate information, second length information, second width information and rotation angle information.

[0011] By adopting the above technical solution, it is clear that the first position information includes the first center coordinate information, the first length information and the first width information. A set of position information includes multiple second position information. Each second position information includes the second center coordinate information, the second length information, the second width information and the rotation angle information. Each photovoltaic panel corresponds to one second position information, which can accurately characterize the position and size and other characteristics of the target photovoltaic string and photovoltaic panel, thereby more accurately locating and identifying the target photovoltaic panel and its defects, and improving the accuracy and precision of photovoltaic defect identification.

[0012] Optionally, the yolov11-obb model uses IncepitonNeXt as the backbone structure, and the yolov11-obb model uses a channel-enhanced feature pyramid network. The channel-enhanced feature pyramid network uses an adaptive channel weighting mechanism to dynamically adjust the weights of different channels during feature fusion to enhance defect-related features.

[0013] By adopting the above technical solution, the yolov11-obb model uses IncepitonNeXt as the backbone structure, and adopts a channel-enhanced feature pyramid network that dynamically adjusts the weights of different channels during feature fusion through an adaptive channel weighting mechanism, which can enhance defect-related features and thus improve the accuracy and precision of defect recognition.

[0014] Optionally, the yolov11-obb model introduces a coordinate attention mechanism in the detection head so that image features correspond one-to-one with position information, and the yolov11-obb model uses the KFIoU loss function instead of RotatedBboxLoss. The KFIoU loss function includes scale-insensitive center point loss and distance-independent terms.

[0015] By adopting the above technical solutions, the introduction of a coordinate attention mechanism in the detection head can make image features correspond to position information one-to-one, thereby improving the accuracy of target position judgment during detection; using the KFIoU loss function instead of RotatedBboxLoss, based on Gaussian modeling and Gaussian product approximation of SkewIoU, and including scale-insensitive center point loss and distance-independent terms, can more accurately reflect the degree of overlap between target boxes and improve the accuracy of the model in target detection and positioning.

[0016] Optionally, the yolov11-obb model is trained using defect sample data, wherein the defect sample data is obtained by one of the following methods: using a generative adversarial network (GAN) to generate defect sample data; simulating wind-blown sand, snow, and crack defects through image synthesis to obtain defect sample data; using transfer learning to perform data expansion on a small number of sample defects to obtain defect sample data.

[0017] By adopting the above technical solution, defect sample data is generated by using the generative adversarial network GAN, defect sample data is obtained by simulating wind-blown sand, snow and crack defects through image synthesis, and defect sample data is obtained by using transfer learning to expand the data of few sample defects to train the yolov11-obb model. This can increase the amount of defect sample data used for training, improve the situation of few sample data, and enable the model to better learn various photovoltaic defect characteristics, thereby improving the accuracy and generalization ability of the yolov11-obb model in photovoltaic defect recognition method based on drones.

[0018] Optionally, preprocessing the initial image to obtain a processed image includes: performing brightness adjustment and contrast enhancement processing on the initial image to obtain a processed image.

[0019] By adopting the above technical solution, the brightness of the collected initial image of the photovoltaic array is adjusted and the contrast is enhanced, which can improve the image quality and provide clearer and more accurate images for subsequent target detection using the YOLOv11 model and target image detection using the YOLOv11-OBB model, thereby improving the accuracy and efficiency of photovoltaic defect recognition and reducing misjudgments and missed judgments caused by poor image quality.

[0020] Optionally, the second detection result also includes area information of the target defect, and the above method further includes: determining a target level of the target defect based on the defect type information and the area information; and recommending a corresponding target maintenance strategy based on the target level.

[0021] By adopting the above technical solution, when using drones for photovoltaic defect identification, the area information of the target defect can be obtained, the target level of the target defect can be determined based on the defect type information and area information, and the corresponding maintenance strategy can be recommended based on the target level. This helps to more accurately assess the defect status, take appropriate maintenance measures, and improve the efficiency and effectiveness of photovoltaic panel maintenance.

[0022] In the second aspect of the present application, a photovoltaic defect identification system based on a drone is also provided, including: an acquisition module for using a drone to acquire an initial image of a photovoltaic array; a processing module for preprocessing the initial image to obtain a processed image; a first detection module for using a yolov11 model to perform target detection on the processed image to obtain a first detection result, the first detection result including the first position information of the target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; an acquisition module for cropping the processed image according to the first position information to obtain a target image, wherein the target image includes the target photovoltaic string; a second detection module for using a yolov11-obb model to detect the target image to obtain a second detection result, and when the second detection result indicates that there is a target defect in the target photovoltaic string, outputs the first position information, the defect type information of the target photovoltaic panel and the target row and column information, wherein the target photovoltaic panel represents the photovoltaic panel with the target defect in the target photovoltaic string.

[0023] In a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements any one of the above method steps when executing the program.

[0024] In a fourth aspect of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores instructions. When the instructions are executed, any one of the above method steps is performed.

[0025] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. Using drones to collect images can avoid the problems of low efficiency, easy missed detection and false detection of manual inspections, and incomplete coverage of fixed camera monitoring. Using the yolov11 model for target detection can obtain the first position information of the target photovoltaic string. Using the yolov11-obb model to detect the target image, and output the first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel, and the target row and column information of the target photovoltaic panel when the target defect is detected, it realizes the accurate identification and positioning of photovoltaic defects, improves the accuracy and efficiency of photovoltaic defect identification, and can accurately output the specific location and type information of the defect; 2. When it is determined that the target photovoltaic panel has a target defect, the target row and column information of the target photovoltaic panel is determined based on the set of position information, and the first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel, and the target row and column information of the target photovoltaic panel can be output. This effectively improves the accuracy of photovoltaic defect identification and the integrity of the output information, and solves the problem of low defect identification accuracy and inability to accurately output the specific location and type information of the defect in related technologies; 3. The yolov11-obb model uses IncepitonNeXt as its backbone structure and a channel-enhanced feature pyramid network that dynamically adjusts the weights of different channels during feature fusion through an adaptive channel weighting mechanism. This can enhance defect-related features and thus improve the accuracy and precision of defect recognition. 4. Generate defect sample data using generative adversarial network (GAN), obtain defect sample data by simulating wind-blown sand, snow and crack defects through image synthesis, and use transfer learning to expand the data of few-sample defects to obtain defect sample data for training the yolov11-obb model. This can increase the amount of defect sample data used for training, improve the situation of few-sample data, and enable the model to better learn various photovoltaic defect characteristics, thereby improving the accuracy and generalization ability of the yolov11-obb model in photovoltaic defect recognition method based on drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a photovoltaic defect identification method based on a drone provided in an embodiment of the present application; Figure 2 This is an example diagram of photovoltaic defect detection provided by an embodiment of the present application; Figure 3 This is an example of a photovoltaic defect recognition result provided by the embodiment of the present application. Figure 1 ; Figure 4 This is an example of a photovoltaic defect recognition result provided by the embodiment of the present application. Figure 2 ; Figure 5 This is a structural block diagram of a photovoltaic defect identification system based on a drone provided in an embodiment of the present application; Figure 6 This is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Description of reference numerals: 600 - electronic device; 601 - processor; 602 - communication bus; 603 - user interface; 604 - network interface; 605 - memory. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0030] In the description of the embodiments of the present application, the term "plurality" means two or more. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprise," "have" and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0031] This application provides a photovoltaic defect recognition method based on drones, referring to Figure 1 , Figure 1 : is a flow chart of a photovoltaic defect identification method based on a drone provided in an embodiment of the present application, the method comprising: Step S101, using a drone to collect an initial image of the photovoltaic array; Step S102, pre-processing the initial image to obtain a processed image; Step S103: Use the yolov11 model to perform target detection on the processed image to obtain a first detection result, where the first detection result includes first position information of a target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; Step S104, cropping the processed image according to the first position information to obtain a target image, wherein the target image includes the target photovoltaic string; Step S105: Use the yolov11-obb model to detect the target image and obtain a second detection result. When the second detection result indicates that there is a target defect in the target photovoltaic string, the first position information, the defect type information of the target photovoltaic panel, and the target row and column information are output, wherein the target photovoltaic panel indicates the photovoltaic panel with the target defect in the target photovoltaic string.

[0032] Through the above steps, using drones to collect images can avoid the problems of low efficiency of manual inspections, easy missed detections and false detections, and incomplete coverage of fixed camera monitoring; preprocessing of the initial image can optimize image quality; using the yolov11 model for target detection can obtain the first position information of the target photovoltaic string, so that the target image can be obtained by subsequent image cropping, and the target image can focus on the target photovoltaic string; using the yolov11-obb model to detect the target image, and when the target defect is detected, the first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel, and the target row and column information of the target photovoltaic panel are output, realizing the accurate identification and positioning of photovoltaic defects, improving the accuracy and efficiency of photovoltaic defect identification, and accurately outputting the specific location and type information of the defect.

[0033] This embodiment uses a drone to collect images of photovoltaic arrays. For example, the image acquisition device carried by the drone is used to collect the initial image of the photovoltaic array in the photovoltaic power station. The initial image is then processed and detected in stages. First, the collected initial image is preprocessed to improve image quality, reduce interference factors such as noise, and provide a better image basis for subsequent target detection. The processed image is then subjected to target detection using the yolov11 model to determine the first position information of the target photovoltaic string. That is, the panoramic image of the photovoltaic array collected by the drone is subjected to preliminary target detection using the yolov11 model to locate the boundary position of each photovoltaic string, including the first position information of the target photovoltaic string; and then the processed image is subjected to the first position information. Cropping is performed to obtain a target image containing only the target PV string. This allows focusing on the region of interest, specifically the target PV string, which is any PV string in a PV array. The YOLOv11-OBB model is then used to perform more detailed inspection of the cropped target image to determine whether defects exist. This involves employing an improved YOLOv11-OBB model (supporting rotated bounding box detection) to identify defects in the cropped image and output detailed information about the defect, including the initial location of the target PV string, the defect type of the target PV panel, and the target row and column of the target PV panel within the target PV string. The YOLOv11-OBB model is more adaptable to tilted or rotated PV panels. The initial image can include visible light images and infrared thermal images. In the related art, manual inspection of photovoltaic panels is time-consuming and labor-intensive, and is prone to missed detection or false detection due to factors such as fatigue and negligence. Fixed cameras are difficult to fully cover the photovoltaic array due to limitations in installation position and angle, and have limited defect recognition capabilities in complex environments. Related art also uses a single-model end-to-end detection method, but is prone to missed detection. This embodiment decomposes photovoltaic defect detection into three organically combined steps: "target positioning → regional focusing → defect recognition", and especially introduces an improved OBB (oriented bounding box) model to solve the problem of inaccurate detection frame matching caused by the regular tilted arrangement of photovoltaic panels. This embodiment uses drones to automatically collect images and perform inspections, greatly improving inspection efficiency. The flexibility of drones enables them to collect images from multiple angles, overcoming the limitations of fixed cameras. Through image preprocessing and staged detection models, especially the combination of YOLOv11 and YOLOv11-OBB models, defects on photovoltaic panels can be identified more accurately, false detections and missed detections can be reduced, and the first position information corresponding to the defective target photovoltaic component, the defect type information of the target photovoltaic panel, and the row and column information of the target photovoltaic panel can be output, which facilitates maintenance personnel to quickly locate and handle defects, thereby improving the maintenance efficiency and reliability of photovoltaic power stations.

[0034] In an optional embodiment, the yolov11-obb model is used to detect the target image to obtain a second detection result. When the second detection result indicates that there is a target defect in the target photovoltaic string, the first position information, the defect type information of the target photovoltaic panel and the target row and column information are output, including: the yolov11-obb model uses a rotating frame to detect a group of photovoltaic panels in the target image to obtain a group of position information, and identifies the local image corresponding to each rotating frame to obtain a group of recognition results, wherein the target photovoltaic string includes a group of photovoltaic panels, and the second detection result includes a group of position information and a group of recognition results; when a group of recognition results indicates that there is a target defect in the target photovoltaic panel, the target row and column information of the target photovoltaic panel is determined according to the group of position information, and the first position information, defect type information and target row and column information are output, wherein a group of recognition results includes defect type information.

[0035] In the above embodiment, the yolov11-obb model is used to obtain a set of position information by detecting a group of photovoltaic panels in the target image through a rotating frame, and a set of recognition results are obtained by identifying the local image corresponding to the rotating frame. This can more accurately locate the position of the photovoltaic panel and identify defects. When it is determined that the target photovoltaic panel has a target defect, the target row and column information of the target photovoltaic panel is determined based on the set of position information. The first position information of the target photovoltaic string, the defect type information of the target photovoltaic panel and the target row and column information of the target photovoltaic panel can be output, which effectively improves the accuracy of photovoltaic defect recognition and the integrity of the output information, and solves the problems of low defect recognition accuracy and inability to accurately output the specific location and type information of the defect in related technologies.

[0036] The yolov11-obb model uses a rotating frame to inspect a set of photovoltaic panels in a target image, obtaining a set of positional information for each panel. This set of positional information includes the positional information corresponding to each panel in the set. For example, the positional information for each panel may include the rotating frame's center coordinates, length, width, and rotation angle. Compared to conventional rectangular frames, rotating frames can better fit tilted or irregularly placed photovoltaic panels, allowing for more accurate positioning of the panels in the image. Each partial image corresponding to the rotating frame is recognized, resulting in a set of recognition results containing information such as the presence of a defect in the panel and the defect type. If a recognition result indicates a target defect in the target panel, the target row and column information of the target panel within the target photovoltaic string is determined based on the set of positional information. Specifically, the target panel's row and column number within the target photovoltaic string is determined, such as row 2, column 3. Combined with the first positional information of the target photovoltaic string from the first detection result, the target photovoltaic string's first positional information, the target panel's defect type, and the target row and column information are output, providing detailed and accurate data for subsequent maintenance work. In actual photovoltaic power stations, photovoltaic panels may be placed irregularly due to factors such as terrain and installation methods. Ordinary rectangular frame detection in related technologies is difficult to accurately locate these irregular photovoltaic panels. This embodiment adopts rotating frame detection. By identifying the local image corresponding to the rotating frame, it can accurately determine whether there are defects in the photovoltaic panel and the type of defects. For example, the defect type can be a hot spot, a crack, dirt (such as wind and sand or snow, etc.); at the same time, after the defect is detected, the target row and column information of the photovoltaic panel where the defect is located can be further determined, solving the problem of inaccurate positioning of irregular photovoltaic panels; in addition, the related technology may only be able to detect the approximate existence of the defect, but cannot accurately give the specific row and column information of the photovoltaic panel where the defect is located. This embodiment can not only identify the defect type, but also determine the target row and column information of the target photovoltaic panel based on a set of position information, and output the relevant information in full, solving the problem of accurate defect positioning and detailed information output.

[0037] In an optional embodiment, target row and column information of a target photovoltaic panel is determined based on a set of position information, including: determining the center coordinates of each rotation box based on a set of position information, wherein each photovoltaic panel in a group of photovoltaic panels corresponds to a rotation box; determining the arrangement direction of each photovoltaic panel in a group of photovoltaic panels based on the long side direction of each rotation box; sorting according to the x or y value of the center coordinate of each rotation box to obtain a row and column distribution result of a group of photovoltaic panels, wherein the row and column distribution result includes the target row and column information.

[0038] In the above embodiment, the center coordinates of each rotating frame are determined according to a set of position information, which can provide basic data for subsequent calculations; the arrangement direction of the photovoltaic panels is determined according to the long side direction of the rotating frame, so that the arrangement of the photovoltaic panels can be clearly known; the row and column distribution results are obtained by sorting the x or y values ​​of the center coordinates of the rotating frame to determine the target row and column information, which helps to accurately determine the position of the target photovoltaic panel in the photovoltaic string, thereby improving the accuracy and efficiency of photovoltaic defect identification and solving the problem that it is difficult to accurately output the specific location information of the defect in the existing detection method.

[0039] This embodiment analyzes the center coordinates and long-side directions of each rotating frame (corresponding to each photovoltaic panel) to infer the arrangement orientation of the photovoltaic panels. Based on this information, the panels are sorted to determine their row and column distribution within the string. Specifically, the center coordinates of each rotating frame are first determined based on a set of position information (the position information of each rotating frame). Each photovoltaic panel corresponds to a rotating frame, so the center coordinates of each rotating frame represent the center position of the corresponding photovoltaic panel. The arrangement orientation of each photovoltaic panel within the group is determined based on the long-side direction of each rotating frame. The long-side direction of the rotating frame reflects the primary extension direction of the photovoltaic panel in the image, thereby inferring the arrangement orientation of the photovoltaic panels (for example, horizontal or vertical arrangement). The rotating frames are then sorted according to the x or y value of the center coordinate to obtain a row and column distribution result for the group of photovoltaic panels. For example, if the photovoltaic panels are arranged horizontally, the rows can be determined by sorting according to the y value of the center coordinate; if they are arranged vertically, the columns can be determined by sorting according to the x value of the center coordinate. The row and column distribution result includes the target row and column information, namely the specific row and column position of the target photovoltaic panel within the photovoltaic string. In different installation environments, photovoltaic panels may be arranged at different angles or directions, which poses a challenge to defect detection. The method of this embodiment can adapt to various arrangement situations by rotating the direction of the frame, ensuring that the position of the photovoltaic panels can be accurately identified even in complex layouts.

[0040] For example, assuming the target photovoltaic string consists of 6 photovoltaic panels, the yolov11-obb model is used to detect these 6 photovoltaic panels and provide the following information: Plate 1: (100, 200), 10°; Plate 2: (200, 200), 10°; Plate 3: (300, 200), 10°; Plate 4: (100, 300), 10°; Plate 5: (200, 300), 10°; Plate 6: (300, 300), 10°; To group and sort photovoltaic panels, the system first clusters the center points based on the Y coordinate and identifies two rows: the first row (Y=200): panel 1, panel 2, panel 3; the second row (Y=300): panel 4, panel 5, panel 6; then, within each row, the system sorts the panels in ascending order of X coordinate: the first row: panel 1 → panel 2 → panel 3 → corresponding columns are 1, 2, and 3; the second row: panel 4 → panel 5 → panel 6 → corresponding columns are 1, 2, and 3; finally, the system labels each photovoltaic panel with its specific row and column information in the target photovoltaic string, including the row and column information of the target photovoltaic panel.

[0041] In an optional embodiment, the first position information includes first center coordinate information, first length information and first width information; a set of position information includes multiple second position information, each photovoltaic panel in a set of photovoltaic panels corresponds to a second position information, and the second position information includes second center coordinate information, second length information, second width information and rotation angle information.

[0042] In the above embodiment, it is clear that the first position information includes first center coordinate information, first length information and first width information. A set of position information includes multiple second position information. Each second position information includes second center coordinate information, second length information, second width information and rotation angle information. Each photovoltaic panel corresponds to one second position information, which can accurately characterize the position and size and other features of the target photovoltaic string and photovoltaic panel, thereby more accurately locating and identifying the target photovoltaic panel and its defects, and improving the accuracy and precision of photovoltaic defect identification.

[0043] The first position information describes the location of the target PV string and includes first center coordinates, first length, and first width. The first center coordinates determine the center position of the target PV string in the image, while the first length and first width define its size within the image. These three parameters fully describe the location and size of the target PV string in the initial image. A set of position information describes the position of each PV panel within the target PV string. Each PV panel corresponds to a second position information. In addition to the center coordinates, length, and width (i.e., the second center coordinates, second length, and second width) in the first position information, the second position information also includes rotation angle information. This rotation angle information accounts for the tilt or rotation of PV panels in real-world scenarios, providing a more accurate description of the panel's position in the image. This detailed position information enables the system to more accurately locate the target PV panel, especially in complex environments (such as tilted or irregularly arranged panels). The rotation angle information helps more accurately determine the panel's orientation, thereby improving defect location accuracy. For example, the first position information includes the first center coordinates (250, 250), the first length (300), and the first width (150); for example, the second position information corresponding to the first photovoltaic panel includes the second center coordinates (100, 200), the second length (150), the second width (50), and the rotation angle 10°. This is just an example.

[0044] In an optional embodiment, the yolov11-obb model adopts IncepitonNeXt as the backbone structure, and the yolov11-obb model adopts a channel-enhanced feature pyramid network. The channel-enhanced feature pyramid network uses an adaptive channel weighting mechanism to dynamically adjust the weights of different channels during feature fusion to enhance defect-related features.

[0045] In the above embodiment, the yolov11-obb model adopts IncepitonNeXt as the backbone structure, and adopts a channel-enhanced feature pyramid network that dynamically adjusts the weights of different channels during feature fusion through an adaptive channel weighting mechanism, which can enhance defect-related features and thereby improve the accuracy and precision of defect recognition.

[0046] InceptionNeXt is a novel convolutional neural network architecture that combines the design principles of the Inception module with the advantages of modern Transformer-style networks (such as ConvNeXt). As the backbone of the Yolov11-OBB model, InceptionNeXt efficiently extracts multi-scale features from images, providing rich feature representations for subsequent object detection. While maintaining high efficiency, it also boasts enhanced feature extraction capabilities, making it particularly suitable for object detection tasks in complex scenarios. The Channel Enhanced Feature Pyramid Network (FPN) introduces an adaptive channel weighting mechanism to dynamically adjust the importance of different channels during multi-scale feature fusion. During feature fusion, features from different channels may have different importance for defect detection. This adaptive channel weighting mechanism allows the model to dynamically adjust the weights of different channels based on the needs of the task, enhancing feature channels relevant to defect detection and suppressing irrelevant ones. This mechanism emphasizes relevant feature channels and suppresses irrelevant or interfering information, thereby improving the recognition of small objects and low-contrast defects. This dynamic adjustment mechanism enables the model to more effectively extract and utilize relevant features, enhancing the accuracy and robustness of defect detection.

[0047] In an optional embodiment, the yolov11-obb model introduces a coordinate attention mechanism in the detection head so that image features correspond one-to-one with position information, and the yolov11-obb model uses the KFIoU loss function instead of RotatedBboxLoss, and the KFIoU loss function includes a scale-insensitive center point loss and a distance-independent term.

[0048] In the above embodiment, the introduction of the coordinate attention mechanism in the detection head can make the image features correspond to the position information one by one, thereby improving the accuracy of the target position judgment during detection; the KFIoU loss function is used instead of the RotatedBboxLoss, based on Gaussian modeling and Gaussian product approximation of SkewIoU, and includes scale-insensitive center point loss and distance-independent terms, which can more accurately reflect the degree of overlap between target boxes and improve the accuracy of the model in target detection and positioning.

[0049] A position-aware feature enhancement module is introduced into the detection head to dynamically associate image features with spatial coordinates (x, y), allowing the model to pay more attention to the precise boundary position of the photovoltaic panel and reduce the positioning deviation of the rotating frame. The yolov11-obb model introduces a coordinate attention mechanism into the detection head. This mechanism can establish a one-to-one correspondence between image features and position information. Specifically, it processes the feature map and learns the connection between the features at each position and the actual target position by paying attention to the feature responses at different positions in the feature map. In this way, the model can more accurately locate the target position based on the features when performing target detection, especially for rotated frame target detection (obb), which can better adapt to different angles and directions of the target. The traditional rotated frame loss RotatedBboxLoss has certain limitations when processing rotated frame target detection. The KFIoU loss function is designed based on Gaussian modeling and the Gaussian product approximation SkewIoU (SkewIoU). It includes a scale-insensitive center point loss and a distance-independent term. The scale-insensitive center point loss means that the loss function is more stable in center point localization when dealing with objects of varying scales, and does not incur large errors due to changes in object scale. The distance-independent term reduces unnecessary interference from distance factors during the loss function calculation, allowing it to focus more on the actual overlap of objects. The use of the KFIoU loss function enables more effective parameter optimization during model training. The scale-insensitive center point loss and distance-independent term reduce unnecessary interference, allowing the model to focus more on the actual overlap of objects, improving model convergence speed and stability. As a result, the model's performance in the rotated box object detection task is significantly improved, enabling more accurate detection of photovoltaic panel defects and reducing false positives and missed detections. In this way, the KFIoU loss function can more accurately measure the difference between the predicted and true boxes, guiding the model to learn more accurate detection results.

[0050] In an optional embodiment, the yolov11-obb model is trained using defect sample data, wherein the defect sample data is obtained by one of the following methods: using a generative adversarial network (GAN) to generate defect sample data; simulating wind and sand, snow, and crack defects through image synthesis to obtain defect sample data; using transfer learning to expand data on a small number of sample defects to obtain defect sample data.

[0051] In the above embodiment, defect sample data is generated by using a generative adversarial network (GAN), defect sample data is obtained by simulating wind, sand, snow and crack defects through image synthesis, and defect sample data is obtained by using transfer learning to expand the data of a few sample defects to train the yolov11-obb model. This can increase the amount of defect sample data used for training, improve the situation of few sample data, and enable the model to better learn various photovoltaic defect characteristics, thereby improving the accuracy and generalization ability of the yolov11-obb model in photovoltaic defect recognition method based on drones.

[0052] Defect sample data is generated using a generative adversarial network (GAN). The GAN consists of a generator and a discriminator. The generator attempts to generate data similar to real defect samples, while the discriminator distinguishes between the generated data and real defect samples. Through continuous adversarial training, the generator gradually learns to generate increasingly realistic defect sample data. This generated data can be used to expand the training set, exposing the model to more defect samples of different types and characteristics, thereby improving its generalization ability. Defect sample data is generated by simulating windblown sand, snow, and cracks through image synthesis. This method uses image processing techniques to synthesize defect effects such as windblown sand, snow, and cracks into existing photovoltaic panel images. For example, operations such as image filtering and texture addition can be used to simulate windblown sand, pixel modification can be used to simulate snow cover, and line drawing can be used to simulate crack defects. This method can quickly generate a large amount of sample data with simulated defects, providing rich material for model training. Transfer learning is used to expand data for few-shot defects: Transfer learning leverages models pre-trained on other related datasets to transfer the features and knowledge learned from them to the current defect detection task. For defect types with limited sample sizes, features can be extracted using a pre-trained model and fine-tuned on these small samples. This approach can unlock the potential information within the limited sample defect data and expand it, allowing the model to learn more features related to these defects during training, thereby enhancing its ability to identify these defects. In real-world photovoltaic defect detection scenarios, collecting a large amount of sample data encompassing a wide range of defect types is challenging, and some defect types may have very few samples. This can result in the model failing to fully learn the characteristics of various defects during training, leading to overfitting and impacting its generalization and detection accuracy. The three methods described above for obtaining defect sample data can effectively expand the training set and address the issue of insufficient sample data. Real-world photovoltaic power plant environments are complex and highly variable, with diverse environmental disturbances such as wind, sand, and snow, as well as various types of crack defects. Collecting sample data covering all conditions in real-world environments is costly and time-consuming. Image synthesis methods can simulate these complex real-world scenarios, allowing the model to be trained with sample data that is more realistic, thereby improving its detection performance in real-world scenarios.

[0053] In an optional embodiment, preprocessing the initial image to obtain a processed image includes: performing brightness adjustment and contrast enhancement processing on the initial image to obtain a processed image.

[0054] In the above embodiment, the brightness adjustment and contrast enhancement processing of the collected initial image of the photovoltaic array can improve the image quality, provide a clearer and more accurate image for the subsequent target detection using the yolov11 model and the target image detection using the yolov11-obb model, thereby improving the accuracy and efficiency of photovoltaic defect recognition and reducing misjudgments and missed judgments caused by poor image quality.

[0055] Image brightness and contrast directly impact the legibility of information within the image. In photovoltaic defect detection scenarios, the initial images captured by drones may be overly bright, dark, or lack contrast due to varying lighting conditions (such as strong light, weak light, or shadows). This can hinder the clear visualization of features and potential defects in the photovoltaic panels. By adjusting the image brightness, pixel values ​​are made more uniform, preventing overly bright or dark areas from interfering with subsequent detection. Contrast enhancement can enhance the brightness differences between pixels within an image, making it easier to distinguish between objects and background. This is crucial for subsequent object detection and defect identification, as clearer images improve model recognition accuracy. Brightness adjustment enhances overall image visibility and sharpens details. Brightness adjustment applies a linear or nonlinear transformation to the brightness of each pixel in the image, bringing the overall brightness to an appropriate level and preventing the loss of important information due to overbrightness or underbrightness. Contrast enhancement enhances the brightness differences between pixels within an image, highlighting edges and details. This makes the outlines, textures, and potential defects of the photovoltaic panels more distinct, facilitating accurate recognition and analysis by subsequent object detection models. Through brightness adjustment and contrast enhancement processing, the quality of the initial image can be significantly improved, making the details in the image clearer, providing a better basis for subsequent defect detection.

[0056] In an optional embodiment, the second detection result also includes area information of the target defect, and the above method further includes: determining a target level of the target defect based on the defect type information and the area information; and recommending a corresponding target maintenance strategy based on the target level.

[0057] In the above embodiment, when using a drone for photovoltaic defect identification, the area information of the target defect can be obtained, the target level of the target defect can be determined based on the defect type information and area information, and the corresponding maintenance strategy can be recommended based on the target level, which helps to more accurately assess the defect condition, take appropriate maintenance measures, and improve the efficiency and effectiveness of photovoltaic panel maintenance.

[0058] In addition to defect type information, the second inspection result also includes the target defect's area information. This can be achieved by counting pixels or performing geometric calculations on the detected defect area to determine the defect's area size. Based on the defect type and area information, the target defect level is determined. This is typically achieved using preset rules or models. For example, defects are classified into different levels (such as minor, moderate, and severe) based on their type and area size. Defects of different types and sizes may have varying degrees of impact on PV system performance. For example, a small but severe crack may require more immediate attention than a large, minor stain. Based on the determined target defect level (or target defect level), a corresponding targeted maintenance strategy is recommended. For example, for low-level defects, regular inspections are conducted to monitor whether the defects are expanding or worsening. For low-priority defects (such as small areas of dust), regular cleaning can be arranged. For medium-level defects, professionals conduct on-site assessments to determine whether repairs are necessary. For medium-level defects (such as large hot spots or cracks), immediate repairs may be recommended. For high-level defects, maintenance personnel are immediately dispatched to replace the panels or perform repairs to minimize the impact on the PV system's power generation efficiency. This strategy can be customized based on the actual application scenario and maintenance standards.

[0059] Taking hot spot defects (local aging caused by high temperature) as an example, when the hot spot area accounts for less than 5% of the total area of ​​a single photovoltaic panel, it is considered a mild defect. When the proportion is 5%-15%, it is considered a moderate defect. When the proportion is greater than 15%, it is considered a moderate defect. The proportion range here is only an example and can be set as needed in actual applications.

[0060] In an optional embodiment, when the yolov11-obb model is used to detect the target image, multi-scale feature extraction is performed on the target image, and convolution operations are performed on the target image using convolution kernels of different scales to obtain feature information of each photovoltaic panel and defect at different scales.

[0061] In the above embodiment, when using the YOLOv11-OBB model to inspect a target image, multi-scale feature extraction is performed on the target image. This means that the model convolves the target image with convolution kernels of different scales to obtain feature information of each photovoltaic panel and defect at different scales. The size of the convolution kernel determines the scale of the image features that the model can capture. Smaller convolution kernels can capture detailed information in the image and are very effective for detecting small photovoltaic panel defects (such as tiny cracks and localized damage). Larger convolution kernels, on the other hand, can extract overall image features and contextual information, helping to identify large photovoltaic panels or defects with larger areas (such as large-area occlusion and severe deformation). By integrating feature information extracted at different scales, the model can more comprehensively and accurately identify individual photovoltaic panels and defects within them. By using convolution kernels of different scales, the model can simultaneously capture local details and global structure in the image, thereby achieving a more comprehensive understanding of the image content. Through multi-scale feature extraction, the model can simultaneously capture local details and global structure in the image, thereby more accurately detecting photovoltaic panels and defects of different sizes and improving detection accuracy.

[0062] In an optional embodiment, target row and column information of a target photovoltaic panel is determined based on a set of position information, including: determining the center coordinates of each rotation box based on a set of position information, wherein each photovoltaic panel in a group of photovoltaic panels corresponds to a rotation box; grouping the center coordinates of each rotation box by Y coordinate and sorting them by X coordinate to obtain a row and column distribution result, wherein the row and column distribution result includes the target row and column information.

[0063] In the above embodiment, a set of position information is used to determine the center coordinates of each rotating frame, and the center coordinates of each rotating frame are grouped by Y coordinate and sorted by X coordinate to obtain a row and column distribution result containing target row and column information. When there is a target defect in the target photovoltaic string, the target row and column information, defect type information and the first position information of the target photovoltaic panel can be accurately output, thereby solving the problems of low efficiency of manual inspection, incomplete monitoring coverage of fixed cameras and low accuracy and efficiency of existing drone detection, and improving the accuracy and efficiency of photovoltaic defect identification under complex backgrounds.

[0064] First, the center coordinates of each rotating frame are determined based on a set of position information. Each photovoltaic panel corresponds to a rotating frame, so the center coordinates of each rotating frame can represent the center position of the corresponding photovoltaic panel. The center coordinates of each rotating frame are grouped according to the Y coordinate, that is, the center coordinates with similar Y coordinates are grouped together, which can determine the row information of the photovoltaic panel. Within each group, the center coordinates are sorted by the X coordinate, which can determine the column information of the photovoltaic panel. In this way, the row and column distribution results of the photovoltaic panels are finally obtained, which includes the target row and column information of the target photovoltaic panel.

[0065] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application will be described in detail below with reference to specific embodiments.

[0066] The present invention provides a method for photovoltaic defect identification based on the YOLOv11 and YOLOv11-OBB models, which relates to the field of artificial intelligence. The method can realize intelligent analysis of photovoltaic defects. The present invention is described below.

[0067] The method of this embodiment mainly adopts target detection and rotation frame to locate defects; Use yolov11 and yolov11-obb models for photovoltaic string detection and photovoltaic panel defect identification respectively; Data synthesis and image fusion. For example, images of sand, snow, and rocks can be downloaded from the Internet and synthesized in a specified area, i.e., synthesizing samples with few defects. GAN can also be used to generate defective samples. Multi-sampling technology can also be used to collect a small number of samples to solve the problem of few defective samples in related technologies. The YOLOv11-OBB model backbone was modified to use IncepitonNeXt, achieving high throughput while maintaining competitive performance. A channel-enhanced feature pyramid network (CE-FPN) was used to mitigate feature loss caused by upsampling. Finally, a coordinate attention mechanism (CoordAttention) was introduced into the detection head to establish a one-to-one correspondence between image features and location information, improving the algorithm's positioning accuracy by 1.1%, 1.5%, and 1.9% compared to YOLOv11-OBB-M. KFIoU replaces the RotatedBboxLoss of yolov11-obb. KFIoU effectively approximates the SkewIoU loss based on Gaussian modeling and Gaussian product. It consists of two components: a scale-insensitive center point loss that quickly reduces the distance between two bounding box centers; and a distance-independent term that uses the product of Gaussian distributions to mimic the SkewIoU mechanism. Within a certain distance (for example, within 9 pixels), KFIoU aligns with the SkewIoU loss in terms of trend, improving the recognition map by 0.7%.

[0068] The overall steps of the photovoltaic defect identification method of the embodiment of the present application include: S1, image acquisition, uses drones to collect and annotate images; the image acquisition module can also perform some simple processing on the collected images, such as adjusting the brightness and contrast of the images to improve the quality and clarity of the images; S2, object detection and rotation box recognition; S3, outputting the location information of the photovoltaic string detection frame, the location information of the photovoltaic panel, and the defect information; S4, the row and column information of the photovoltaic panel position, uses the center position of the rotating frame as the basis for sorting, and calculates the tilt angle of each frame. For example, grouping the Y coordinate (row): group by vertical coordinate (Y value), with each group representing a row, and divide into two rows. Based on the center of the minimum coordinate value frame, the Y distance threshold is determined and divided into two rows. Sorting by X coordinate (column) within the group: sort the points in each row in ascending order by horizontal coordinate (X value).

[0069] Considering that the defect sample data in related technologies are somewhat insufficient, the image data collected by drones can be labeled for further training of subsequent models.

[0070] Figure 2 This is an example diagram of photovoltaic defect detection provided by an embodiment of the present application. Figure 2 It includes the detection frame of a single photovoltaic string and the rotation frame of a single photovoltaic panel. Figure 2 The single photovoltaic panel in the rotating frame shown in FIG is defective. Figure 3 、 Figure 4 Examples of photovoltaic defect recognition results Figure 1 、 two , Figure 3 The defect recognition result in the test is bird droppings, with a confidence level of 96.1%. Figure 4 The defect recognition result in the image is infrared hot spot, with a confidence level of 95% and 92.03%. It should be noted that Figure 3 The corresponding original image is a visible light image. Figure 2 、 Figure 4 The corresponding original image is infrared thermal imaging.

[0071] This application also provides a photovoltaic defect recognition system based on drones, such as Figure 5 As shown, Figure 5 This is a block diagram of a photovoltaic defect identification system based on a drone provided in an embodiment of the present application. The system includes: An acquisition module 51 is configured to acquire an initial image of the photovoltaic array using a drone; The processing module 52 is used to pre-process the initial image to obtain a processed image; a first detection module 53 for performing target detection on the processed image using a YOLOv11 model to obtain a first detection result, wherein the first detection result includes first position information of a target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; an obtaining module 54 for cropping the processed image according to the first position information to obtain a target image, wherein the target image includes a target photovoltaic string; The second detection module 55 is used to detect the target image using the yolov11-obb model to obtain a second detection result. When the second detection result indicates that a target defect exists in the target photovoltaic string, the second detection module 55 outputs the first position information, the defect type information of the target photovoltaic panel, and the target row and column information, wherein the target photovoltaic panel indicates the photovoltaic panel with the target defect in the target photovoltaic string.

[0072] It should be noted that the above embodiments provide systems that implement their functions using only the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0073] The present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed, any one of the above-mentioned method steps is executed.

[0074] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0075] This application also discloses an electronic device. Figure 6 As shown, Figure 6 6. This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. The electronic device 600 may include: at least one processor 601, at least one communication bus 602, a user interface 603, at least one network interface 604, and a memory 605.

[0076] The communication bus 602 is used to implement the connection and communication between these components.

[0077] The user interface 603 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 603 may also include a standard wired interface and a wireless interface.

[0078] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0079] The processor 601 may include one or more processing cores. Using various interfaces and circuits, the processor 601 connects various components within the electronic device (e.g., a server). By running or executing instructions, programs, code sets, or instruction sets stored in the memory 605 and accessing data stored in the memory 605, the processor 601 performs various server functions and processes data. Optionally, the processor 601 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 601 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 601.

[0080] Among them, the memory 605 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 605 includes a non-transitory computer-readable storage medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also optionally be at least one storage device located away from the aforementioned processor 601. Refer to Figure 6 , as a computer storage medium, the memory 605 may include an operating system, a network communication module, a user interface module, and an application program for a photovoltaic defect identification method based on a drone.

[0081] exist Figure 6In the electronic device 600 shown, the user interface 603 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 601 can be used to call an application of a photovoltaic defect identification method based on a drone stored in the memory 605. When executed by one or more processors 601, the electronic device 600 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0082] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed device or system can be implemented in other ways. For example, the device or system embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0084] The foregoing is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. In other words, any equivalent variations and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the disclosure herein.

[0085] This application is intended to cover any modifications, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field not described in the present disclosure.

Claims

1. A photovoltaic defect identification method based on drone, characterized in that: include: Use a drone to collect initial images of the photovoltaic array; Preprocessing the initial image to obtain a processed image; Performing target detection on the processed image using a yolov11 model to obtain a first detection result, where the first detection result includes first position information of a target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; cropping the processed image according to the first position information to obtain a target image, wherein the target image includes the target photovoltaic string; The target image is detected using the yolov11-obb model to obtain a second detection result. When the second detection result indicates that a target defect exists in the target photovoltaic string, the first position information, the defect type information of the target photovoltaic panel, and the target row and column information are output, wherein the target photovoltaic panel represents the photovoltaic panel in the target photovoltaic string that has the target defect.

2. The method according to claim 1, characterized in that The target image is detected using the yolov11-obb model to obtain a second detection result. When the second detection result indicates that a target defect exists in the target photovoltaic string, the first position information, the defect type information of the target photovoltaic panel, and the target row and column information are output, including: The yolov11-obb model uses a rotating frame to detect a group of photovoltaic panels in the target image to obtain a set of position information, and recognizes the local image corresponding to each rotating frame to obtain a set of recognition results, wherein the target photovoltaic string includes the group of photovoltaic panels, and the second detection result includes the set of position information and the set of recognition results; When the set of identification results indicates that the target photovoltaic panel has the target defect, the target row and column information of the target photovoltaic panel is determined based on the set of position information, and the first position information, the defect type information and the target row and column information are output, wherein the set of identification results includes the defect type information.

3. The method according to claim 2, characterized in that Determining target row and column information of the target photovoltaic panel according to the set of position information includes: Determining the center coordinates of each rotation frame according to the set of position information, wherein each photovoltaic panel in the set of photovoltaic panels corresponds to a rotation frame respectively; determining an arrangement direction of each photovoltaic panel in the group of photovoltaic panels according to the long side direction of each rotating frame; The rotation frames are sorted according to the x or y values ​​of the center coordinates to obtain a row and column distribution result of the group of photovoltaic panels, wherein the row and column distribution result includes the target row and column information.

4. The method according to claim 2, characterized in that The first position information includes first center coordinate information, first length information and first width information; the set of position information includes multiple second position information, each photovoltaic panel in the set of photovoltaic panels corresponds to a second position information, and the second position information includes second center coordinate information, second length information, second width information and rotation angle information.

5. The method according to claim 1, wherein The yolov11-obb model adopts IncepitonNeXt as the backbone structure, and the yolov11-obb model adopts a channel-enhanced feature pyramid network. The channel-enhanced feature pyramid network dynamically adjusts the weights of different channels during feature fusion through an adaptive channel weighting mechanism to enhance defect-related features.

6. The method according to claim 1, characterized in that The yolov11-obb model introduces a coordinate attention mechanism in the detection head so that image features correspond one-to-one with position information, and the yolov11-obb model uses the KFIoU loss function instead of RotatedBboxLoss, and the KFIoU loss function includes scale-insensitive center point loss and distance-independent terms.

7. The method according to claim 1, characterized in that The yolov11-obb model is trained using defect sample data, wherein the defect sample data is obtained by one of the following methods: Generate the defect sample data using a generative adversarial network (GAN); The defect sample data is obtained by simulating wind-blown sand, snow accumulation, and crack defects through image synthesis; Transfer learning is used to perform data expansion on a small number of sample defects to obtain the defect sample data.

8. A photovoltaic defect identification system based on drones, characterized in that: include: an acquisition module, for acquiring an initial image of the photovoltaic array using a drone; A processing module, configured to pre-process the initial image to obtain a processed image; a first detection module, configured to perform target detection on the processed image using a yolov11 model to obtain a first detection result, wherein the first detection result includes first position information of a target photovoltaic string, wherein the target photovoltaic string is any photovoltaic string in the photovoltaic array; an obtaining module, configured to crop the processed image according to the first position information to obtain a target image, wherein the target image includes the target photovoltaic string; The second detection module is used to detect the target image using the yolov11-obb model to obtain a second detection result. When the second detection result indicates that a target defect exists in the target photovoltaic string, the second detection module outputs the first position information, the defect type information of the target photovoltaic panel, and the target row and column information, wherein the target photovoltaic panel represents the photovoltaic panel in the target photovoltaic string that has the target defect.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images