A photovoltaic module identification model training method, an identification method, and related equipment

Abstract

The application discloses a photovoltaic module identification model training method, an identification method and related equipment, and belongs to the technical field of photovoltaic technology. The application performs photovoltaic module identification based on unmanned aerial vehicle aerial visible light images, constructs a photovoltaic module identification model in combination with an attention mechanism and a rotating target detection algorithm based on a YOLO algorithm, and performs training of the photovoltaic module identification model based on unmanned aerial vehicle aerial images of different photovoltaic power stations. The photovoltaic module identification model obtained through the training can effectively reduce false detection and missed detection of photovoltaic modules, realize efficient and accurate end-to-end photovoltaic module identification with strong scene generalization, and can perform photovoltaic module identification on various scenes such as distributed photovoltaic power stations and centralized photovoltaic power stations through model deployment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photovoltaic, in particular to a photovoltaic module identification model training method, a photovoltaic module identification method, a photovoltaic module identification model training system, a photovoltaic module identification system, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Photovoltaic power stations are mainly divided into distributed photovoltaic power stations and centralized photovoltaic power stations. Current research on photovoltaic module identification in photovoltaic power stations mainly focuses on visible light images, infrared images, satellite images and aerial orthographic images. Due to the different characteristics of different images, different identification methods are used. In visible light images, photovoltaic strings and modules are mainly identified based on prior knowledge and image preprocessing, such as threshold segmentation, image filtering and segmentation, and contour extraction. In infrared images, photovoltaic module segmentation results are mainly extracted by threshold segmentation and denoising. Satellite images and aerial orthographic images often use deep learning for photovoltaic target identification. Due to the complex background of satellite images and aerial orthographic images, deep learning image segmentation algorithms are often used for photovoltaic string segmentation and identification.

[0003] Although the traditional image processing algorithm based on prior knowledge is good in pertinence, it is usually based on a specific scene (distributed photovoltaic power station or centralized photovoltaic power station) and has low generalization. The deep learning photovoltaic string identification algorithm based on satellite images and aerial orthographic images can only identify at the string level due to low image resolution, and cannot achieve component-level identification. After segmentation and identification, some post-processing is required. Different algorithms are used in the two-step processing process, which needs to be connected, and end-to-end identification cannot be achieved. Therefore, how to achieve efficient and accurate end-to-end photovoltaic module identification with strong scene generalization is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] The purpose of the present application is to provide a photovoltaic module identification model training method, a photovoltaic module identification method, a photovoltaic module identification model training system, a photovoltaic module identification system, an electronic device and a computer readable storage medium, so as to realize efficient and accurate end-to-end photovoltaic module identification with strong scene generalization.

[0005] To achieve the above purpose, the present application provides a photovoltaic module identification model training method, comprising:

[0006] Obtaining image data of photovoltaic modules in a first photovoltaic power station and a second photovoltaic power station; the image data is visible light image data obtained by a UAV aerial photography;

[0007] Labeling the image data with a component frame to obtain the image data with the component frame; the component frame is a rotating frame containing the photovoltaic module.

[0008] constructing a photovoltaic module recognition model; the photovoltaic module recognition model comprises an attention mechanism module and a rotating target detection algorithm based on a YOLO algorithm; the rotating target detection algorithm based on the YOLO algorithm comprises a backbone network, a neck network and a head network connected in turn; the attention mechanism module comprises a channel attention mechanism and a spatial attention mechanism connected in turn; the attention mechanism module is located between the backbone network and the neck network; the output of a spatial pyramid pooling feature module in the backbone network is taken as the input of the channel attention mechanism; the output of the spatial attention mechanism is taken as the input of an up-sampling layer and the input of a connection layer in the neck network;

[0009] training the photovoltaic module recognition model based on the image data with the component frame to obtain a trained photovoltaic module recognition model.

[0010] Optionally, the rotating target detection algorithm based on the YOLO algorithm comprises a YOLOv5 rotating target detection algorithm, a YOLOv8 rotating target detection algorithm and a YOLOv11 rotating target detection algorithm.

[0011] Optionally, after the image data is annotated with the component frame to obtain the image data with the component frame, the method further comprises:

[0012] cropping the image data with the component frame and converting the coordinates of the component frame to the cropped image data;

[0013] after the cropped image data is classified according to the first kind of photovoltaic power station and the second kind of photovoltaic power station, a training set and a test set are respectively divided out; the photovoltaic power stations in the training set and the test set are not repeated;

[0014] Correspondingly, the training of the photovoltaic module recognition model based on the image data with the component frame to obtain a trained photovoltaic module recognition model comprises:

[0015] training the photovoltaic module recognition model based on the training set and the test set to obtain a trained photovoltaic module recognition model.

[0016] Optionally, the training of the photovoltaic module recognition model based on the training set and the test set to obtain a trained photovoltaic module recognition model comprises:

[0017] Step 1: after setting the model training hyperparameters, the photovoltaic module recognition model is trained based on the training set, and the photovoltaic module recognition model is iteratively trained and optimized to obtain an optimized photovoltaic module recognition model.

[0018] Step 2: determining whether the optimized photovoltaic module recognition model converges through the trend of the loss function;

[0019] Step 3: when it is detected that the optimized photovoltaic module recognition model converges, evaluating the converged photovoltaic module recognition model according to a model evaluation index by using the test set to determine whether the converged photovoltaic module recognition model meets an application accuracy requirement;

[0020] Step 4: when it is detected that the converged photovoltaic module recognition model does not meet the application accuracy requirement, returning to the step 1 until the converged photovoltaic module recognition model meets the application accuracy requirement, and obtaining the trained photovoltaic module recognition model.

[0021] Optionally, the loss function comprises a classification loss and a bounding box regression loss; the bounding box regression loss comprises a DFL loss and a CIoU loss.

[0022] The expression of the classification loss is:

[0023] ;

[0024] In the formula, represents the classification loss; represents an intersection over union of a predicted module box and a real module box when the predicted module box is a positive sample, the predicted module box being a module box predicted by the photovoltaic module recognition model; represents a confidence of the predicted module box, and are both manually set parameters;

[0025] The expression of the DFL loss is:

[0026] ;

[0027] In the formula, represents the DFL loss; represents a position of a pixel point; represents a coordinate value of the predicted module box in which a center point is at the i-th pixel point; represents a coordinate value of the predicted module box in which a center point is at the i-th pixel point; represents a coordinate value of the predicted module box in which a center point is at the i-th pixel point; represents a coordinate value of the predicted module box in which a center point is at the i-th pixel point; represents a coordinate value of the real module box; represents a coordinate integral value of the real module box; represents a coordinate proximity integral value of the real module box;

[0028] The expression of the CIoU loss is:

[0029] ;

[0030] wherein, denotes the CIoU loss; denotes the center point of the real component frame; denotes the center point of the predicted component frame; denotes the Euclidean distance between the center point of the real component frame and the center point of the predicted component frame; is the diagonal distance of the minimum enclosing box of the real component frame and the predicted component frame, is a weight function, is a weight term;

[0031] The expression of is:

[0032] ;

[0033] wherein, denotes the intersection over union;

[0034] The expression of is:

[0035] ;

[0036] wherein, denotes the width of the real component frame; denotes the height of the real component frame; denotes the width of the predicted component frame; denotes the height of the predicted component frame;

[0037] The expression of the loss function is:

[0038] ;

[0039] wherein, denotes the classification loss; denotes the DFL loss; denotes the CIoU loss.

[0040] Optionally, the backbone network comprises a first convolutional block, four first module groups and a spatial pyramid pooling feature module connected in sequence; the first module group comprises a convolutional block and a C2f module connected in sequence;

[0041] The input of the first convolutional block is taken as the input of the backbone network;

[0042] The output of the C2f module of the third first module group is taken as a first output of the backbone network, and is used for outputting a first-level feature;

[0043] The output of the spatial pyramid pooling feature module is taken as a second output of the backbone network, and is used for outputting a second-level feature;

[0044] The second output of the backbone network is taken as an input of the channel attention mechanism.

[0045] Optionally, the neck network comprises a first neck branch and a second neck branch; the first neck branch comprises two second module groups connected in sequence; the second module group comprises an upsampling layer, a connection layer and a C2f module connected in sequence; the second neck branch comprises two third module groups connected in sequence; the third module group comprises a convolution block, a connection layer and a C2f module connected in sequence;

[0046] The first output of the backbone network is taken as an input of the connection layer of the first second module group;

[0047] The output of the spatial attention mechanism is taken as an input of the upsampling layer of the first second module group, and is taken as an input of the connection layer of the second third module group;

[0048] The output of the C2f module of the second second module group is taken as an input of the convolution block of the first third module group; the output of the convolution block of the first third module group is taken as a first output of the neck network, and is used for outputting a first scale fusion feature;

[0049] The output of the C2f module of the first third module group is taken as a second output of the neck network, and is used for outputting a second scale fusion feature;

[0050] The output of the C2f module of the second third module group is taken as a third output of the neck network, and is used for outputting a third scale fusion feature.

[0051] Optionally, the head network comprises three detection heads; the first output, the second output and the third output of the neck network are taken as inputs of the three detection heads respectively;

[0052] The detection head comprises a first convolution layer, a first head branch and a second head branch; the input of the first convolution layer is taken as an input of the detection head; the output of the first convolution layer is taken as an input of the first head branch, and is taken as an input of the second head branch;

[0053] The first head branch comprises three convolutional layers connected in sequence; an input of a first convolutional layer in the three convolutional layers is taken as an input of the first head branch; an output of a third convolutional layer in the three convolutional layers is taken as an output of the first head branch, and is used for outputting whether the component frame is a photovoltaic component.

[0054] The second head branch comprises a fourth module group, a first output branch and a second output branch; an input of the fourth module group is taken as an input of the second head branch; an output of the fourth module group is taken as an input of the first output branch and an input of the second output branch; an output of the first output branch is taken as a first output of the second head branch, and is used for outputting coordinates of the component frame; an output of the second output branch is taken as a second output of the second head branch, and is used for outputting confidence of the component frame; the fourth module group comprises two convolutional layers connected in sequence; the first output branch and the second output branch each comprise a convolutional layer.

[0055] To achieve the above object, the present application provides a photovoltaic component recognition method, comprising:

[0056] Obtaining image data to be tested of a photovoltaic component in a photovoltaic power station to be tested; the image data to be tested is visible light image data to be tested obtained by aerial photography of a UAV;

[0057] Inputting the image data to be tested into a photovoltaic component recognition model to obtain coordinates of a component frame; the photovoltaic component recognition model is a model trained according to the photovoltaic component recognition model training method in any one of the above.

[0058] Optionally, after the image data to be tested is inputted into the photovoltaic component recognition model to obtain the coordinates of the component frame, the method further comprises:

[0059] Converting the coordinates of the component frame into geographic coordinates.

[0060] Optionally, after the image data to be tested is inputted into the photovoltaic component recognition model to obtain the coordinates of the component frame of the photovoltaic component, the method further comprises:

[0061] Processing the coordinates of the component frame by using an NMS algorithm to obtain final coordinates of the component frame.

[0062] Optionally, the inputting the image data to be tested into the photovoltaic component recognition model to obtain the coordinates of the component frame comprises:

[0063] Cutting the image data to be tested;

[0064] Inputting the cut image data to be tested into the photovoltaic component recognition model to obtain coordinates of a component frame of the photovoltaic component in the cut image data to be tested.

[0065] restore the coordinates of the component frame of the photovoltaic module in the to-be-tested image data after cutting to the coordinates of the component frame of the photovoltaic module in the to-be-tested image data before cutting.

[0066] To achieve the above object, the present application provides a photovoltaic module identification model training system, comprising:

[0067] a training data acquisition module configured to acquire image data of photovoltaic modules in a first photovoltaic power station and a second photovoltaic power station; the image data is visible light image data obtained by aerial photography of a UAV;

[0068] a labeling module configured to perform component frame labeling on the image data to obtain the image data with the component frame; the component frame is a rotating frame containing the photovoltaic module;

[0069] a model construction module configured to construct a photovoltaic module identification model; the photovoltaic module identification model comprises an attention mechanism module and a rotating target detection algorithm based on a YOLO algorithm; the rotating target detection algorithm based on the YOLO algorithm comprises a backbone network, a head network and a neck network connected in sequence; the attention mechanism module is located between the backbone network and the head network; the attention mechanism module comprises a channel attention mechanism and a spatial attention mechanism connected in sequence;

[0070] a training module configured to train the photovoltaic module identification model based on the image data with the component frame to obtain a trained photovoltaic module identification model.

[0071] To achieve the above object, the present application provides a photovoltaic module identification system, comprising:

[0072] a to-be-tested data acquisition module configured to acquire to-be-tested image data of photovoltaic modules in a to-be-tested photovoltaic power station; the to-be-tested image data is to-be-tested visible light image data obtained by aerial photography of a UAV;

[0073] an identification module configured to input the to-be-tested image data into a photovoltaic module identification model to obtain the coordinates of the component frame; the photovoltaic module identification model is a model trained according to the photovoltaic module identification model training method of any one of the above.

[0074] To achieve the above object, the present application provides an electronic device, comprising:

[0075] a memory configured to store a computer program;

[0076] a processor configured to implement the steps of the photovoltaic module identification model training method of any one of the above and / or the steps of the photovoltaic module identification method of any one of the above when the computer program is executed.

[0077] To achieve the above object, the application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the photovoltaic module identification model training method according to any one of the above, and / or the steps of the photovoltaic module identification method according to any one of the above.

[0078] Obviously, the application provides a photovoltaic module identification model training method. The photovoltaic module identification is based on unmanned aerial vehicle aerial visible light images. The photovoltaic module identification model is constructed by combining the attention mechanism and the rotating target detection algorithm based on the YOLO algorithm. The photovoltaic module identification model is trained based on the unmanned aerial vehicle aerial images of different photovoltaic power stations. The trained photovoltaic module identification model can effectively reduce the false detection and missed detection of photovoltaic modules, realize efficient and accurate end-to-end photovoltaic module identification with strong scene generalization, and perform photovoltaic module identification on various scenes such as distributed photovoltaic power stations and centralized photovoltaic power stations through model deployment.

[0079] The application provides a photovoltaic module identification method. The trained photovoltaic module identification model is applied to photovoltaic module identification. Compared with the existing photovoltaic identification method, the photovoltaic module identification method has strong scene generalization. The photovoltaic module identification method can realize high-precision photovoltaic module identification for different photovoltaic power stations such as distributed photovoltaic power stations and centralized photovoltaic power stations. At the same time, the photovoltaic module identification method can realize end-to-end target identification at the module level, and further meet the demand for photovoltaic module level identification in photovoltaic operation and maintenance.

[0080] The application also provides a photovoltaic module identification model training system, a photovoltaic module identification system, an electronic device, and a computer readable storage medium, which have the above beneficial effects. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only embodiments of the application. Those skilled in the art can obtain other drawings without creative labor based on the provided drawings.

[0082] Figure 1 A flowchart of a photovoltaic module identification model training method provided by the embodiment of the application is shown in the figure.

[0083] Figure 2 A structural schematic diagram of a photovoltaic module identification model provided by the embodiment of the application is shown in the figure.

[0084] Figure 3 A structural schematic diagram of a C2f module provided by the embodiment of the application is shown in the figure.

[0085] Figure 4 A structural schematic diagram of a spatial pyramid pooling feature module provided for an embodiment of the present application;

[0086] Figure 5 A structural schematic diagram of a bottleneck module provided for an embodiment of the present application;

[0087] Figure 6 A structural schematic diagram of a CBS module provided for an embodiment of the present application;

[0088] Figure 7 A structural schematic diagram of a detection head provided for an embodiment of the present application;

[0089] Figure 8 A structural schematic diagram of an attention mechanism module provided for an embodiment of the present application;

[0090] Figure 9 A flowchart of a photovoltaic module recognition method provided for an embodiment of the present application;

[0091] Figure 10 A photovoltaic module recognition flowchart provided for an embodiment of the present application;

[0092] Figure 11 A photovoltaic power station image data cropping manner provided for an embodiment of the present application;

[0093] Figure 12 A P-R curve diagram of a photovoltaic module recognition model provided for an embodiment of the present application;

[0094] Figure 13 A schematic diagram of a mountain concentrated photovoltaic recognition result provided for an embodiment of the present application;

[0095] Figure 14 A schematic diagram of a roof distributed photovoltaic power station recognition result provided for an embodiment of the present application;

[0096] Figure 15 A first image of a photovoltaic power station to be tested provided for an embodiment of the present application;

[0097] Figure 16 A schematic diagram of a first attention mechanism optimized component recognition result provided for an embodiment of the present application;

[0098] Figure 17 A schematic diagram of a first attention mechanism pre-optimized component recognition result provided for an embodiment of the present application;

[0099] Figure 18 A schematic diagram of a second attention mechanism optimized component recognition result provided for an embodiment of the present application;

[0100] Figure 19 A schematic diagram of a second attention mechanism for optimizing a component recognition result provided by an embodiment of the present application;

[0101] Figure 20 A structural block diagram of a photovoltaic component recognition model training system provided by an embodiment of the present application;

[0102] Figure 21 A structural block diagram of a photovoltaic component recognition system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0103] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0104] With the increasing demand for renewable energy worldwide, the market size of the photovoltaic industry is expanding, and intelligent recognition of photovoltaic components (also known as photovoltaic panel components) has become an indispensable part of photovoltaic power station operation and maintenance, which is of great significance to photovoltaic intelligent operation and maintenance. At present, the scale of photovoltaic power stations is increasing year by year, and the construction of highway distributed photovoltaics is in a stage of rapid development. Photovoltaic component intelligent recognition technology will bring new vitality to the photovoltaic new energy industry, accelerate the deep integration of photovoltaic industry digitalization, intelligentization, and greenization, and promote the construction of photovoltaic and spatiotemporal information empowerment application.

[0105] Photovoltaic power stations are mainly divided into distributed photovoltaic power stations and centralized photovoltaic power stations. Distributed photovoltaic power stations are mostly based on building surfaces, such as residential building rooftops, commercial building rooftops, etc., to solve the power supply problem of users nearby, achieve compensation and external transmission of power supply differences through grid connection, and have the characteristics of sparse distribution in space. Centralized photovoltaic power stations are often built in areas with open terrain and sufficient sunlight, such as high-altitude deserts, abandoned mountainous areas, and some coastal beaches, to utilize the relatively stable and abundant solar energy resources to build large centralized photovoltaic power stations, which are connected to high-voltage transmission systems to supply remote loads. These places are often remote and have harsh environments, making the cost of human maintenance and repair extremely high. Therefore, using unmanned aerial vehicles (UAVs) to replace human labor for photovoltaic inspection has become the current development direction, and accurately obtaining the position information of photovoltaic components is a necessary step for subsequent high-precision UAV inspection. The current way of obtaining a large amount of photovoltaic component string and component position information using manual marking is extremely time-consuming and labor-intensive. Therefore, how to quickly and accurately obtain the position information of photovoltaic component strings and components automatically is an urgent problem to be solved.

[0106] Current research on photovoltaic module recognition mainly focuses on visible light images, infrared images, satellite images and aerial orthographic images. Due to the different characteristics of different images, different recognition methods are used.

[0107] In visible light images and infrared images, image processing techniques are mainly used for photovoltaic module recognition. Among them, in visible light images, photovoltaic module strings and modules are mainly recognized based on prior knowledge and image preprocessing, using threshold segmentation, image filtering and segmentation, and contour extraction; in infrared images, threshold segmentation and denoising are mainly used to extract photovoltaic module segmentation results.

[0108] In satellite images and aerial orthographic images, deep learning is often used for photovoltaic target recognition. Due to the complexity of the background of satellite images and aerial orthographic images, deep learning image segmentation algorithms are usually used for photovoltaic string segmentation and recognition.

[0109] Defects and deficiencies of the prior art:

[0110] (1) Although the traditional image processing algorithm based on prior knowledge is good in pertinence, it is usually based on a specific scene (distributed photovoltaic power station or centralized photovoltaic power station), and has low generalization;

[0111] (2) The deep learning photovoltaic string recognition algorithm based on satellite images can only recognize at the string level due to low image resolution, and cannot achieve component-level recognition; the existing public satellite image and aerial orthographic image data sets for photovoltaic string recognition are pixel-level string segmentation data sets, which cannot meet the current demand for photovoltaic component-level recognition in photovoltaic operation and maintenance;

[0112] (3) The deep learning image segmentation algorithm used for photovoltaic string segmentation and recognition needs certain post-processing to achieve good accuracy, and the two-step processing process uses different algorithms, which need to be connected, and cannot realize end-to-end recognition.

[0113] Therefore, the present application provides a photovoltaic module recognition model training method, which is based on unmanned aerial vehicle aerial visible light images for photovoltaic module recognition, combines attention mechanism and a rotating target detection algorithm based on YOLO algorithm to construct a photovoltaic module recognition model, and trains the photovoltaic module recognition model based on unmanned aerial vehicle aerial images of different photovoltaic power stations, thereby realizing efficient and accurate end-to-end photovoltaic module recognition with strong scene generalization.

[0114] Please refer to Figure 1 , Figure 1 A flowchart of a photovoltaic module recognition model training method provided by the embodiments of the present application, which can include:

[0115] S101: Obtain image data of the photovoltaic components in the first photovoltaic power station and the second photovoltaic power station; the image data is visible light image data obtained by aerial photography of a UAV.

[0116] The first photovoltaic power station in this embodiment can be a distributed photovoltaic power station, which can be but is not limited to a distributed photovoltaic power station installed on a roof. The second photovoltaic power station in this embodiment can be a centralized photovoltaic power station, which can be but is not limited to a centralized photovoltaic power station installed on a mountain or in a forest. The first photovoltaic power station and the second photovoltaic power station in this embodiment can be two different photovoltaic power stations divided according to installation locations of the photovoltaic power stations, or can be two different photovoltaic power stations divided according to other standards.

[0117] It should be noted that the photovoltaic power station data obtained by aerial photography of a photovoltaic power station by a UAV generally includes image data of the photovoltaic components and geographic information, wherein the image data obtained by aerial photography of a UAV is visible light image data, i.e., R (Red), G (Green), and B (Blue) data. In this embodiment, only the image data of the photovoltaic components is extracted to train the photovoltaic component recognition model.

[0118] It should be noted that if a satellite image can achieve a very high resolution, the method of this embodiment can also be extended to the photovoltaic component recognition scenario of a satellite image.

[0119] S102: Perform component bounding box labeling on the image data to obtain image data with a component bounding box; the component bounding box is a rotating bounding box containing the photovoltaic components.

[0120] It should be noted that the process of performing component bounding box labeling on the image data in this embodiment forms a data label on the image data, which is a component bounding box. A rotating bounding box refers to a rectangular bounding box that can rotate around its center point. In this embodiment, the photovoltaic components are labeled using a rotating bounding box, which can more accurately represent the shape of the photovoltaic components. In this embodiment, the component bounding box can be the smallest rotating bounding box containing the photovoltaic components, i.e., the smallest rectangular bounding box among all the rectangular bounding boxes wrapping the photovoltaic components.

[0121] This embodiment does not limit the specific way of performing component bounding box labeling, as long as a component bounding box wrapping the photovoltaic components can be labeled in the image data, for example, the image data can be manually labeled with a component bounding box.

[0122] Further, after step S102, this embodiment can further include:

[0123] cropping the image data with the component bounding box and converting the coordinates of the component bounding box to the cropped image data;

[0124] After the cropped image data is classified according to the first type of photovoltaic power station and the second type of photovoltaic power station, a training set and a test set are respectively divided; the photovoltaic power stations in the training set and the test set are not repeated.

[0125] In the formula, the cropping process can be performed according to fixed-length rectangular non-overlapping cropping.

[0126] The embodiment is not limited to the specific size of the cropped image data, and the specific size can be determined according to actual conditions. The specific size can be, but is not limited to, 512 pixels in width and 512 pixels in height, and 3 color channels.

[0127] It should be noted that, before training the photovoltaic module recognition model, the image data is pre-cropped, which can reduce the calculation amount of the photovoltaic module recognition model and improve the training speed. Meanwhile, the collected image data is divided into a training set and a test set, wherein the training set is used for subsequent model training iteration; the test set is used for model performance testing to select the optimal model, so that a high-precision photovoltaic module recognition model can be obtained.

[0128] Further, the embodiment can further include: performing data augmentation on the divided training set, and processing the parameters of the component frame according to the same data augmentation manner; the data augmentation can include, but is not limited to, scaling and / or splicing. It should be noted that, by performing data augmentation on the training set, the training set is expanded, and model training is performed based on the expanded training set, which can improve the generalization of the photovoltaic module recognition model.

[0129] S103: Construct a photovoltaic module recognition model; the photovoltaic module recognition model includes an attention mechanism module and a rotation target detection algorithm based on a YOLO (You Only Look Once, a real-time target detection algorithm); the rotation target detection algorithm based on the YOLO algorithm includes a sequentially connected backbone network, a neck network and a head network; the attention mechanism module includes a sequentially connected channel attention mechanism and a spatial attention mechanism; the attention mechanism module is located between the backbone network and the neck network; the output of the spatial pyramid pooling feature module in the backbone network is used as the input of the channel attention mechanism; the output of the spatial attention mechanism is used as the input of the up-sampling layer and the input of the connection layer in the neck network.

[0130] ​The embodiment is not limited to the specific type of rotating target detection algorithm, and can be any YOLO algorithm that can realize rotating target detection. For example, the rotating target detection algorithm based on the YOLO algorithm includes a YOLOv5 rotating target detection algorithm, a YOLOv8 rotating target detection algorithm, and a YOLOv11 rotating target detection algorithm. It should be noted that the YOLOv5 rotating target detection algorithm, the YOLOv8 rotating target detection algorithm, and the YOLOv11 rotating target detection algorithm each include a backbone network, a neck network, and a head network. The specific network structures of the three parts of different algorithms are different.

[0131] It should be noted that the backbone network in the embodiment is the basis of the photovoltaic module recognition model and is responsible for extracting features from the input image data. These features are the basis for target detection by subsequent network layers.

[0132] The embodiment is not limited to the specific structure of the backbone network, as long as it can extract features from the input image data. For example, taking the YOLOv8 rotating target detection algorithm as an example, as shown in Figure 2 , the backbone network can include a first convolutional block, four first module groups, and a spatial pyramid pooling feature module connected in sequence. The first module group includes a convolutional block and a C2f (Convolution to Feature) module connected in sequence.

[0133] The input of the first convolutional block is used as the input of the backbone network.

[0134] The output of the C2f module of the third first module group is used as the first output of the backbone network, which is used to output the first-level features.

[0135] The output of the spatial pyramid pooling feature module is used as the second output of the backbone network, which is used to output the second-level features.

[0136] The second output of the backbone network is used as the input of the channel attention mechanism.

[0137] As shown in Figure 3 , the C2f module in the embodiment can include a first CBS module, a Split layer, three Bottleneck modules, and a second CBS module connected in sequence. The input of the first CBS module is used as the input of the C2f module. The outputs of the three Bottleneck modules are used as the inputs of the second CBS module. The output of the second CBS module is used as the output of the C2f module.

[0138] As shown in Figure 4As shown in the figure, the spatial pyramid pooling feature module in the embodiment can include a first CBS module, three pooling layers, a connection layer and a second CBS module connected in sequence; the input of the first CBS module serves as the input of the spatial pyramid pooling feature module; the outputs of the three pooling layers all serve as the input of the connection layer; the output of the second CBS module serves as the output of the spatial pyramid pooling feature module; and the connection layer can be a Concat (concatenation) connection layer.

[0139] As shown in the figure, the Bottleneck module in the embodiment can include two CBS modules connected in sequence; the input of the first CBS module serves as the input of the Bottleneck module; and the output of the second CBS module and the input of the first CBS module are concatenated to serve as the output of the Bottleneck module. Figure 5 As shown in the figure, the CBS module in the embodiment can include a Convolution layer, a Batch Normalization (BN) layer and a Sigmoid Linear Unit (SiLU) layer connected in sequence.

[0140] Figure 6 It should be noted that the convolution block in the embodiment can change the resolution and the number of channels of the input image data to achieve better feature extraction; the C2f module can better understand the image by combining global semantic information and local target information, and make the entire photovoltaic module recognition model pay more attention to meaningful areas; and the spatial pyramid pooling feature module can pool the features at different scales, thereby allowing the photovoltaic module recognition model to perform target recognition on different sizes of input image data, and improving the receptive field and robustness of the photovoltaic module recognition model.

[0141] It should be noted that the features output by the C2f module in the embodiment are low-level features, and the pyramid pooling feature module outputs high-level features. The embodiment does not limit the specific number of features output by the backbone network, and in addition to outputting the above two levels of features, more levels of features can also be output.

[0142] It should be noted that the neck network in the embodiment is located between the backbone network and the head network, and its function is to perform feature fusion and enhancement, fuse the low-level features and the high-level features in the backbone network to provide more overall and local feature information, so as to allow the photovoltaic module recognition model to process targets at different scales, enhance the receptive field and semantic information of the photovoltaic module recognition model, and improve the accuracy and robustness of the photovoltaic module recognition model.

[0143] It should be noted that the neck network in the embodiment is located between the backbone network and the head network, and its function is to perform feature fusion and enhancement, fuse the low-level features and the high-level features in the backbone network to provide more overall and local feature information, so as to allow the photovoltaic module recognition model to process targets at different scales, enhance the receptive field and semantic information of the photovoltaic module recognition model, and improve the accuracy and robustness of the photovoltaic module recognition model.

[0144] ​The embodiment does not limit the specific structure of the neck network, as long as it can ensure that the different levels of features output by the backbone network can be fused and enhanced, for example, taking the YOLOv8 rotating target detection algorithm as an example, as shown in Figure 2 , the neck network can include a first neck branch and a second neck branch; the first neck branch includes two second module groups connected in turn; the second module group includes an upsampling layer, a connection layer and a C2f module connected in turn; the second neck branch includes two third module groups connected in turn; the third module group includes a convolution block, a connection layer and a C2f module connected in turn;

[0145] The first output of the backbone network is input to the connection layer of the first second module group;

[0146] The output of the spatial attention mechanism is input to the upsampling layer of the first second module group, and is input to the connection layer of the second third module group;

[0147] The output of the C2f module of the second second module group is input to the convolution block of the first third module group; the output of the convolution block of the first third module group is the first output of the neck network, used to output the first scale fusion feature;

[0148] The output of the C2f module of the first third module group is the second output of the neck network, used to output the second scale fusion feature;

[0149] The output of the C2f module of the second third module group is the third output of the neck network, used to output the third scale fusion feature.

[0150] In the embodiment, the neck network also uses the C2f module as shown in Figure 3 ; the connection layer in the embodiment can be a Concat connection layer.

[0151] It should be noted that in the embodiment, the first scale fusion feature is a fusion feature with a first scale, the second scale fusion feature is a fusion feature with a second scale, and the third scale fusion feature is a fusion feature with a third scale; wherein the first scale is greater than the second scale, and the second scale is greater than the third scale. The embodiment does not limit the specific scale of the three fusion features, which can be determined according to actual conditions, for example, the first scale can be , the second scale can be , and the third scale can be .

[0152] The embodiment is not limited to the specific number of fusion features output by the neck network. In addition to outputting the three scales of fusion features described above, two scales of fusion features or more scales of features can also be output. In addition, the number of fusion features output is the same as the number of detection heads in the head network to ensure that each scale of fusion feature output is input into a detection head for target recognition.

[0153] It should be noted that the head network in this embodiment is the decision part of the photovoltaic module recognition model, responsible for generating the final recognition result.

[0154] The embodiment is not limited to the specific structure of the head network, as long as it can recognize the fusion features of different scales output by the neck network and output the recognition result. For example, taking the YOLOv8 rotating target detection algorithm as an example, as shown in Figure 2 and Figure 7 The head network can include three detection heads; the first output, the second output and the third output of the neck network are respectively input into the three detection heads;

[0155] The detection head includes a first convolutional layer, a first head branch and a second head branch; the input of the first convolutional layer is input into the detection head; the output of the first convolutional layer is input into the first head branch and the second head branch;

[0156] The first head branch includes three convolutional layers connected in sequence; the input of the first convolutional layer in the three convolutional layers is input into the first head branch; the output of the third convolutional layer in the three convolutional layers is output by the first head branch, used to output whether the component frame is a photovoltaic module;

[0157] The second head branch includes a fourth module group, a first output branch and a second output branch; the input of the fourth module group is input into the second head branch; the output of the fourth module group is input into the first output branch and the second output branch; the output of the first output branch is the first output of the second head branch, used to output the coordinates of the component frame; the output of the second output branch is the second output of the second head branch, used to output the confidence of the component frame; the fourth module group includes two convolutional layers connected in sequence; the first output branch and the second output branch each include a convolutional layer.

[0158] It should be noted that the head network in this embodiment adopts an Anchor-Free (anchor-free) detection method. The head network includes two branches, one branch is responsible for classifying and predicting the target (i.e., the rotating frame containing the photovoltaic module), and the other branch is responsible for predicting the position of the target (i.e., the coordinates of the rotating frame) and the target confidence; the two branches share a convolution layer for feature fusion; each branch includes two convolution layers connected in turn for feature extraction; wherein one branch further includes a convolution layer for outputting whether the target is a photovoltaic module; one branch further includes two independent convolution layers for outputting the position and confidence of the target, respectively.

[0159] It should be noted that the photovoltaic module recognition model in this embodiment is built on the basis of the framework of the rotating target detection algorithm based on the YOLO algorithm. Since the attention mechanism can make the model selectively focus on the region or feature of interest and help the model suppress irrelevant information, for photovoltaic module recognition, the background is complex and has a great impact, and the use of the attention mechanism can make the photovoltaic module recognition model selectively focus on the photovoltaic module and suppress background interference, thereby improving the generalization and robustness of the photovoltaic module recognition model, and then improving the photovoltaic module recognition model precision. Therefore, the attention mechanism module is added to the basis of the rotating target detection algorithm based on the YOLO algorithm in this embodiment to improve the photovoltaic module recognition precision, and the attention mechanism module is added after the spatial pyramid pooling feature module in the backbone network, that is, between the backbone network and the neck network.

[0160] As shown in Figure 8 , the attention mechanism module in this embodiment adopts a combination of channel attention mechanism and spatial attention mechanism, and combines the global features of channels and spaces, which can further improve the performance of the photovoltaic module recognition model. Figure 8 The input feature map in the channel attention mechanism and the spatial attention mechanism is the second level feature output by the spatial pyramid pooling feature module in this embodiment; for this input feature map, after sequentially passing through the channel attention mechanism and the spatial attention mechanism, an output feature map is obtained, and the expression of the output feature map is:

[0161] ;

[0162] In the formula, , the input feature map is a tensor with a dimension of , and each element is a real number (i.e. ; wherein R represents a real number, represents the number of channels, represents the width, represents the height); , the output feature map is a tensor with a dimension of , and each element is a real number (i.e. ; wherein represents a real number, represents a number of channels, represents a width, represents a height); represents a channel attention weight; represents a spatial attention weight; represents a multiplication operation on elements.

[0163] It should be noted that the channel attention mechanism in this embodiment can include a dimension conversion module, a multi-layer perception and an activation function layer connected in turn; the channel attention mechanism first converts the input feature in dimension; then processes through a multi-layer perception to amplify the dependence of the dimension feature on the channel; the output of the multi-layer perception is processed through the activation function layer to obtain the channel attention weight, and the channel attention weight is multiplied with the input feature map according to elements to emphasize important information.

[0164] It should be noted that the spatial attention in this embodiment can include two convolution layers and an activation function layer connected in turn; the spatial attention mechanism uses the convolution layer to realize the reduction and increase of the number of channels, first reduces the number of channels through the first convolution layer, while reducing the amount of calculation and retaining important feature information; then increases the number of channels through the second convolution layer to maintain the consistency of the number of channels; then processes through the activation function layer to obtain the spatial attention weight, and the spatial attention weight is multiplied with the input feature map according to elements to emphasize important information.

[0165] S104: training the photovoltaic component recognition model based on the image data with the component frame to obtain a trained photovoltaic component recognition model.

[0166] This embodiment does not limit the specific way of training the photovoltaic component recognition model, as long as the target photovoltaic component recognition model can be obtained. In order to improve the accuracy of the photovoltaic component recognition model, step S104 in this embodiment can include:

[0167] training the photovoltaic component recognition model based on the training set and the test set to obtain a trained photovoltaic component recognition model.

[0168] Further, training the photovoltaic component recognition model based on the training set and the test set to obtain a trained photovoltaic component recognition model can include:

[0169] Step 1: after setting the model training hyperparameters, training the photovoltaic component recognition model based on the training set, optimizing the photovoltaic component recognition model, and obtaining an optimized photovoltaic component recognition model;

[0170] Step 2: determining whether the optimized photovoltaic module recognition model converges through the trend of the loss function;

[0171] Step 3: when it is detected that the optimized photovoltaic module recognition model converges, the converged photovoltaic module recognition model is evaluated according to the model evaluation index by using the test set to determine whether the converged photovoltaic module recognition model meets the application precision requirement;

[0172] Step 4: when it is detected that the converged photovoltaic module recognition model does not meet the application precision requirement, returning to Step 1 until the converged photovoltaic module recognition model meets the application precision requirement, and obtaining the trained photovoltaic module recognition model.

[0173] It should be noted that the process of optimizing the photovoltaic module recognition model in this embodiment is to optimize the parameters of each module in the backbone network, and / or the parameters of each module in the neck network, and / or the parameters of each module in the head network, and / or the parameters of the channel attention mechanism, and / or the parameters of the spatial attention mechanism in the photovoltaic module recognition model.

[0174] This embodiment does not limit the specific types of model training hyperparameters, which can be determined according to actual conditions. For example, the model training hyperparameters can include, but are not limited to, the number of iterations, the weight of the loss function, and the artificially set parameters (such as the following and ).

[0175] This embodiment does not limit the specific types of loss functions, as long as it can determine whether the optimized photovoltaic module recognition model converges. For example, the loss function can include a classification loss and a bounding box regression loss; the bounding box regression loss can include a DFL (Distribution Focal Loss) loss and a CIoU (Complete Intersection over Union) loss;

[0176] The expression of the classification loss is:

[0177] ;

[0178] In the formula, represents the classification loss; represents the intersection over union of the predicted component box and the real component box when the predicted component box is a positive sample, and the predicted component box is the component box predicted by the photovoltaic module recognition model; represents the confidence of the predicted component box, and are artificially set parameters; it should be noted that the positive sample refers to the predicted component box containing the photovoltaic module;

[0179] The expression of the DFL loss is:

[0180] ;

[0181] In the formula, DFL loss is represented; represents the position of the pixel point; represents the coordinate value of the predicted component frame of the center point at the pixel point; represents the coordinate value of the predicted component frame of the center point at the pixel point; represents the coordinate value of the real component frame; represents the coordinate integral value of the real component frame; represents the coordinate adjacent integral value of the real component frame; It should be noted that the DFL loss in the embodiment is optimized by taking the form of cross-entropy to optimize the probability of the position closest to the real component frame, so that the network focuses on the target position and the distribution of the adjacent area faster;

[0182] The expression of the CIoU loss is:

[0183] ;

[0184] In the formula, CIoU loss is represented; represents the center point of the real component frame; represents the center point of the predicted component frame; represents the Euclidean distance between the center point of the real component frame and the center point of the predicted component frame; is the diagonal distance of the minimum enclosing box of the real component frame and the predicted component frame, is a weight function, is a weight term; It should be noted that when the aspect ratio of the predicted component frame and the real component frame is closer, is smaller; The CIoU loss in the embodiment considers the distance between the center points of the real component frame and the predicted component frame and the diagonal distance of the minimum enclosing box, so that the regression of the component frame of the target is more stable;

[0185] The expression of the IoU loss is:

[0186] ;

[0187] In the formula, IoU is represented;

[0188] The expression of the SIOU loss is:

[0189] ;

[0190] wherein, represents the width of the real component frame; represents the height of the real component frame; represents the width of the predicted component frame; represents the height of the predicted component frame;

[0191] The expression of the loss function is:

[0192] ;

[0193] wherein, represents the classification loss; represents the DFL loss; represents the CIoU loss.

[0194] The present embodiment is not limited to the specific type of model evaluation index, as long as it can determine whether the converged photovoltaic component recognition model meets the application accuracy requirement. For example, the model evaluation index can include, but is not limited to, recall rate and precision.

[0195] The expression of the recall rate is:

[0196] ;

[0197] The expression of the precision is:

[0198] ;

[0199] wherein, represents the number of photovoltaic components correctly recognized in the component recognition result; represents the number of missed photovoltaic components; represents the number of misrecognized photovoltaic components; that is, the recall rate is the proportion of the number of recognized photovoltaic components in the total number of photovoltaic components, and the precision is the proportion of the number of correctly recognized photovoltaic components in the total number of recognized photovoltaic components; it should be noted that the greater than 0.5 indicates correct recognition; and the less than 0.5 indicates misrecognition.

[0200] Based on the above embodiments, this application uses drone aerial images of visible light to identify photovoltaic modules. It combines an attention mechanism and a rotating target detection algorithm based on the YOLO algorithm to construct a photovoltaic module identification model. The photovoltaic module identification model is trained based on drone aerial images of different photovoltaic power plants. The trained photovoltaic module identification model can effectively reduce false detections and false negatives of photovoltaic modules, and achieve efficient, accurate and versatile end-to-end photovoltaic module identification. Through model deployment, photovoltaic module identification can be performed in various scenarios such as distributed photovoltaic power plants and centralized photovoltaic power plants.

[0201] Please refer to Figure 9 , Figure 9 A flowchart of a photovoltaic module identification method provided in this application embodiment, the method may include:

[0202] S201: Acquire the image data of the photovoltaic modules in the photovoltaic power station under test; the image data is the visible light image data obtained by drone aerial photography.

[0203] It should be noted that the data obtained from drone aerial photography of the photovoltaic power station under test typically includes image data of the photovoltaic modules and geographical information, where the image data is RGB data. In this embodiment, only the image data of the photovoltaic modules is extracted and input into the photovoltaic module recognition model.

[0204] S202: Input the image data to be tested into the photovoltaic module recognition model to obtain the coordinates of the module frame; the photovoltaic module recognition model is a model trained according to the photovoltaic module recognition model training method described above.

[0205] The photovoltaic module identification model in this embodiment can be deployed on a server or workstation.

[0206] This embodiment does not limit the specific method of obtaining the coordinates of the component frame, as long as it ensures that the image data to be tested can be input into the photovoltaic module recognition model. For example, the following methods can be used:

[0207] Crop the image data to be tested;

[0208] The cropped image data to be tested is input into the photovoltaic module recognition model to obtain the coordinates of the component frame of the photovoltaic module in the cropped image data to be tested;

[0209] The coordinates of the photovoltaic module's frame in the cropped image data are restored to the coordinates of the frame in the original image data.

[0210] The cropping process can be performed by cropping rectangles of fixed length and width without overlap.

[0211] It should be noted that the embodiment pre-crops the image data before inputting the to-be-tested image data into the photovoltaic module recognition model, so as to reduce the calculation amount of the photovoltaic module recognition model and improve the recognition speed.

[0212] Further, the embodiment after step S202 can further include:

[0213] Converting the coordinates of the module frame into geographical coordinates.

[0214] It should be noted that the coordinates of the module frame in the embodiment are the positions of the photovoltaic module in the image data, and the geographical coordinates are the actual geographical positions of the photovoltaic module. Converting the coordinates of the module frame into geographical coordinates can accurately locate the geographical position of the photovoltaic module.

[0215] The embodiment does not limit converting the coordinates of the module frame into geographical coordinates, as long as the geographical coordinates corresponding to the coordinates of the module frame can be obtained, for example, the geographical information of the photovoltaic module in the to-be-tested photovoltaic power station obtained by the unmanned aerial vehicle aerially photographing the to-be-tested photovoltaic power station can be used to determine the geographical coordinates corresponding to the coordinates of the module frame.

[0216] Further, the embodiment after step S202 can further include:

[0217] The coordinates of the module frame are processed by using the NMS (Non-Maximum Suppression) algorithm to obtain the final coordinates of the module frame.

[0218] It should be noted that the embodiment processes the coordinates of the module frame by using the NMS algorithm, which can remove the repeated module frames, so as to accurately locate the position of the photovoltaic module.

[0219] Further, according to the specific type of the photovoltaic module recognition model used, the embodiment can obtain whether the module frame is a photovoltaic module and the confidence of the module frame by inputting the to-be-tested image data into the photovoltaic module recognition model in step S202.

[0220] Based on the above embodiment, the photovoltaic module recognition model trained by the application is applied to photovoltaic module recognition, which has strong scene generalization compared with the existing photovoltaic recognition method, can realize high-precision photovoltaic module recognition for different photovoltaic power stations such as distributed photovoltaic power stations and centralized photovoltaic power stations, can realize component-level end-to-end target recognition, and further meets the demand for photovoltaic module level recognition in photovoltaic operation and maintenance.

[0221] The above photovoltaic module recognition process will be described in combination with specific examples. Please refer to Figure 10 , Figure 10 A photovoltaic module recognition process schematic diagram provided by the embodiment of the application is shown in the following figure. The process is as follows:

[0222] 1. Data Acquisition and Labeling: Drones are used to take aerial photos of rooftop distributed photovoltaic power stations and mountain centralized photovoltaic power stations. Labelers label the aerial image data with component frames, which are the smallest rotating frames containing photovoltaic components.

[0223] 2. Data Processing and Dataset Organization: The photovoltaic power station component data from Step 1 is organized and categorized into two scenarios: rooftop distributed photovoltaic power stations and mountain-based centralized photovoltaic power stations. Training and testing sets are then created based on the different photovoltaic power stations. The photovoltaic power station image data is cropped using fixed-length, non-overlapping rectangles, such as... Figure 11 As shown, the coordinates of the corresponding component boxes are transformed onto the cropped image.

[0224] 3. Model building and training: Build a photovoltaic module recognition model according to the selected network structure, and perform data augmentation such as scaling and stitching on the training set divided in step 2. At the same time, process the parameters of the component boxes in the same data augmentation method. After setting the model training hyperparameters, perform model training iterations and optimize the model in this step.

[0225] 4. Model Evaluation and Selection: Determine whether the photovoltaic module identification model has converged by analyzing the trend of the loss function. At the same time, evaluate the converged photovoltaic module identification model using the test set in step 2 according to the model evaluation index to determine whether the photovoltaic module identification model meets the application accuracy requirements. If it does not meet the requirements, continue to step 3 to seek model optimization until the application accuracy requirements are met.

[0226] The final curve showing the change in precision versus recall (PR curve) of the optimal photovoltaic module identification model is as follows: Figure 12 As shown, the closer the curve is to the upper right corner (higher recall and higher precision), the better the model performance. Indicates in The mean average precision at a threshold of 0.5, in this embodiment... The accuracy is 0.984, indicating that the photovoltaic module recognition model has high precision.

[0227] 5. Model Deployment and Application: The photovoltaic module recognition model can be deployed to a server or workstation. Aerial data from drones is used to capture photovoltaic power station images. Image data (RGB data) is extracted from the photovoltaic power station data. For unlabeled image data, it is cropped according to fixed-width, non-overlapping rectangles and input into the optimal photovoltaic module recognition model selected in step 4 for photovoltaic module recognition. Then, the coordinates of the obtained module boxes are restored to the large photovoltaic power station image, and NMS post-processing is used to remove duplicate module boxes, resulting in the final photovoltaic power station module recognition result.Figure 13 and Figure 14 As shown in

[0228] 6. Comparative experiment: (1) select the first to-be-detected photovoltaic power station, the image of the to-be-detected photovoltaic power station is as shown in Figure 15 The photovoltaic component identification result obtained by using the optimal photovoltaic component identification model selected in step 4 for photovoltaic component identification is as shown in Figure 16 After removing the attention mechanism module in the optimal photovoltaic component identification model selected in step 4, the photovoltaic component identification result obtained by re-performing photovoltaic component identification is as shown in Figure 17 From the comparison of the component identification results before and after the optimization of the attention mechanism, it can be seen that the optimization of the attention mechanism in this embodiment can effectively reduce the false detection of photovoltaic components; (2) select the second to-be-detected photovoltaic power station; the photovoltaic component identification result obtained by using the optimal photovoltaic component identification model selected in step 4 for photovoltaic component identification is as shown in Figure 18 After removing the attention mechanism module in the optimal photovoltaic component identification model selected in step 4, the photovoltaic component identification result obtained by re-performing photovoltaic component identification is as shown in Figure 19 From the comparison of the component identification results before and after the optimization of the attention mechanism, it can be seen that the optimization of the attention mechanism in this embodiment can effectively reduce the false detection of photovoltaic components; Through the above comparative experiment, it can be seen that the method and system of this embodiment have strong robustness.

[0229] Next, a photovoltaic component identification model training system, a photovoltaic component identification system, an electronic device, and a computer readable storage medium provided by an embodiment of the present application are introduced. The photovoltaic component identification model training system, the photovoltaic component identification system, the electronic device, and the computer readable storage medium described below can be mutually corresponding and referred to with the photovoltaic component identification model training method and the photovoltaic component identification method described above.

[0230] Please refer to Figure 20 , Figure 20 A structural block diagram of a photovoltaic component identification model training system provided by an embodiment of the present application, which can include:

[0231] The training data acquisition module 100 is configured to acquire image data of photovoltaic components in the first photovoltaic power station and the second photovoltaic power station; the image data is visible light image data obtained by aerial photography of a UAV;

[0232] The labeling module 200 is configured to perform component frame labeling on the image data to obtain image data with a component frame; the component frame is a rotating frame containing a photovoltaic component;

[0233] The model construction module 300 is configured to construct a photovoltaic component recognition model; the photovoltaic component recognition model comprises an attention mechanism module and a rotating target detection algorithm based on a YOLO algorithm; the rotating target detection algorithm based on the YOLO algorithm comprises a backbone network, a head network and a neck network connected in sequence; the attention mechanism module is located between the backbone network and the head network; the attention mechanism module comprises channel attention and spatial attention connected in sequence;

[0234] The training module 400 is configured to train the photovoltaic component recognition model based on the image data with the component frame to obtain a trained photovoltaic component recognition model.

[0235] Based on the above embodiments, the photovoltaic component recognition model is constructed based on the attention mechanism and the rotating target detection algorithm based on the YOLO algorithm, and the photovoltaic component recognition model is trained based on the unmanned aerial vehicle aerial visible light images of different photovoltaic power stations. The trained photovoltaic component recognition model can effectively reduce the false detection and missed detection of photovoltaic components, realize efficient and accurate end-to-end photovoltaic component recognition with strong scene generalization, and perform photovoltaic component recognition on various scenes such as distributed photovoltaic power stations and centralized photovoltaic power stations through model deployment.

[0236] Based on the above embodiments, the rotating target detection algorithm based on the YOLO algorithm comprises a YOLOv5 rotating target detection algorithm, a YOLOv8 rotating target detection algorithm and a YOLOv11 rotating target detection algorithm.

[0237] Based on the above embodiments, the labeling module 200 further comprises:

[0238] The first image processing module is configured to crop the image data with the component frame and convert the coordinates of the component frame to the cropped image data;

[0239] The second image processing module is configured to classify the cropped image data into a first photovoltaic power station and a second photovoltaic power station, and then divide the cropped image data into a training set and a test set; the photovoltaic power stations in the training set and the test set are not repeated;

[0240] Correspondingly, the training module 400 is configured to train the photovoltaic component recognition model based on the training set and the test set to obtain a trained photovoltaic component recognition model.

[0241] Based on the above embodiments, the training module 400 is configured to:

[0242] Step 1: After setting the model training hyperparameters, train and iterate the photovoltaic module recognition model based on the training set, optimize the photovoltaic module recognition model, and obtain the optimized photovoltaic module recognition model;

[0243] Step 2: Determine whether the optimized photovoltaic module identification model has converged by analyzing the trend of the loss function;

[0244] Step 3: When the convergence of the optimized photovoltaic module identification model is detected, the converged photovoltaic module identification model is evaluated using the test set according to the model evaluation index to determine whether the converged photovoltaic module identification model meets the application accuracy requirements.

[0245] Step 4: When it is detected that the converged photovoltaic module recognition model does not meet the application accuracy requirements, return to step 1 until the converged photovoltaic module recognition model meets the application accuracy requirements, and obtain the trained photovoltaic module recognition model.

[0246] Based on the above embodiments, the loss function includes classification loss and bounding box regression loss; the bounding box regression loss includes DFL loss and CIoU loss;

[0247] The expression for classification loss is:

[0248] ;

[0249] In the formula, Indicates classification loss; This represents the intersection-over-union ratio (IoU) between the predicted component box and the actual component box when the predicted component box is a positive sample. The predicted component box is the component box predicted by the photovoltaic component recognition model. This indicates the confidence level of the predicted component box. and All parameters are manually set;

[0250] The expression for DFL loss is:

[0251] ;

[0252] In the formula, Indicates DFL loss; Indicates the position of a pixel; Indicates the center point is at the th The coordinates of the predicted component box for each pixel; Indicates the center point is at the th The coordinates of the predicted component box for each pixel; Represents the coordinates of the actual component box; Represents the integral value of the coordinates of the actual component box; The coordinates of the actual component box are adjacent to the integral value;

[0253] The expression of the CIoU loss is:

[0254]

[0255] In the formula, CIoU loss is represented by; The center point of the real component frame is represented by; The center point of the predicted component frame is represented by; The Euclidean distance between the center point of the real component frame and the center point of the predicted component frame is represented by; The diagonal distance of the minimum bounding box of the real component frame and the predicted component frame is represented by, The weight function is represented by, The weight term is represented by;

[0256] The expression of the CIoU loss is:

[0257]

[0258] In the formula, The intersection over union is represented by;

[0259] The expression of the CIoU loss is:

[0260]

[0261] In the formula, The width of the real component frame is represented by; The height of the real component frame is represented by; The width of the predicted component frame is represented by; The height of the predicted component frame is represented by;

[0262] The expression of the loss function is:

[0263]

[0264] In the formula, The classification loss is represented by; The DFL loss is represented by; The CIoU loss is represented by.

[0265] Based on the above embodiments, the backbone network comprises a first convolutional block, four first module groups and a spatial pyramid pooling feature module connected in sequence; the first module group comprises a convolutional block and a C2f module connected in sequence;

[0266] The input of the first convolutional block is taken as the input of the backbone network;

[0267] The output of the C2f module of the third first module group is taken as the first output of the backbone network, which is used to output the first level feature;​​​​

[0268] The output of the spatial pyramid pooling feature module is taken as a second output of the backbone network, and is used for outputting a second-level feature;

[0269] The second output of the backbone network is taken as an input of the channel attention mechanism.

[0270] Based on the above embodiments, the neck network comprises a first neck branch and a second neck branch; the first neck branch comprises two second module groups connected in sequence; the second module group comprises an upsampling layer, a connection layer and a C2f module connected in sequence; the second neck branch comprises two third module groups connected in sequence; the third module group comprises a convolution block, a connection layer and a C2f module connected in sequence;

[0271] The first output of the backbone network is taken as an input of the connection layer of the first second module group;

[0272] The output of the spatial attention mechanism is taken as an input of the upsampling layer of the first second module group, and is taken as an input of the connection layer of the second third module group;

[0273] The output of the C2f module of the second second module group is taken as an input of the convolution block of the first third module group; the output of the convolution block of the first third module group is taken as a first output of the neck network, and is used for outputting a first scale fusion feature;

[0274] The output of the C2f module of the first third module group is taken as a second output of the neck network, and is used for outputting a second scale fusion feature;

[0275] The output of the C2f module of the second third module group is taken as a third output of the neck network, and is used for outputting a third scale fusion feature.

[0276] Based on the above embodiments, the head network comprises three detection heads; the first output, the second output and the third output of the neck network are respectively taken as inputs of the three detection heads;

[0277] The detection head comprises a first convolution layer, a first head branch and a second head branch; the input of the first convolution layer is taken as an input of the detection head; the output of the first convolution layer is taken as an input of the first head branch, and is taken as an input of the second head branch;

[0278] The first head branch comprises three convolution layers connected in sequence; the input of the first convolution layer in the three convolution layers is taken as an input of the first head branch; the output of the third convolution layer in the three convolution layers is taken as an output of the first head branch, and is used for outputting whether the component frame is a photovoltaic component;

[0279] The second head branch includes a fourth module group, a first output branch and a second output branch; an input of the fourth module group is an input of the second head branch; an output of the fourth module group is an input of the first output branch and an input of the second output branch; an output of the first output branch is a first output of the second head branch, used for outputting coordinates of the component frame; an output of the second output branch is a second output of the second head branch, used for outputting confidence of the component frame; the fourth module group includes two convolutional layers connected in sequence; and the first output branch and the second output branch each include a convolutional layer.

[0280] Please refer to Figure 21 , Figure 21 A structural block diagram of a photovoltaic component recognition system provided by an embodiment of the present application is provided, and the system can include:

[0281] A to-be-tested data acquisition module 500 is configured to acquire to-be-tested image data of photovoltaic components in a to-be-tested photovoltaic power station; the to-be-tested image data is to-be-tested visible light image data obtained by aerial photography of a UAV;

[0282] An identification module 600 is configured to input the to-be-tested image data into a photovoltaic component recognition model to obtain coordinates of a component frame; the photovoltaic component recognition model is a model trained according to the photovoltaic component recognition model training method described above.

[0283] Based on the above embodiments, the photovoltaic component recognition model trained by the present application is applied to photovoltaic component recognition, and compared with existing photovoltaic recognition methods, the present application has strong scene generalization, can realize high-precision photovoltaic component recognition for different photovoltaic power stations such as distributed photovoltaic power stations and centralized photovoltaic power stations, can realize component-level end-to-end target recognition, and further meets the needs of photovoltaic component-level recognition in photovoltaic operation and maintenance.

[0284] Based on the above embodiments, the identification module 600 can further include:

[0285] A geographic coordinate conversion module is configured to convert the coordinates of the component frame into geographic coordinates.

[0286] Based on the above embodiments, the identification module 600 can further include:

[0287] A component frame processing module is configured to process the coordinates of the component frame by using an NMS algorithm to obtain final coordinates of the component frame.

[0288] Based on the above embodiments, the identification module 600 is configured to:

[0289] Crop the to-be-tested image data;

[0290] input the cropped to-be-tested image data into the photovoltaic module recognition model to obtain coordinates of a module frame of the photovoltaic module in the cropped to-be-tested image data;

[0291] restore the coordinates of the module frame of the photovoltaic module in the cropped to-be-tested image data to coordinates of a module frame in the to-be-tested image data before cropping.

[0292] Based on the above embodiments, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory is configured to store a computer program; and the processor is configured to execute the computer program to implement the steps of the photovoltaic module recognition model training method according to the above embodiments and / or the steps of the photovoltaic module recognition method according to the above embodiments. Of course, the electronic device can further comprise various necessary network interfaces, power supplies and other components, etc.

[0293] The present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the photovoltaic module recognition model training method according to the above embodiments and / or the steps of the photovoltaic module recognition method according to the above embodiments. The storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0294] The principles and implementation manners of the present application are described by using specific examples in the present application, and each embodiment is in a progressive relationship, and each embodiment mainly describes the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiments, the corresponding method part can be referred to. The above embodiment description is only used to help understand the method of the present application and its core idea. For ordinary skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the present application.

[0295] It also needs to be explained that in the present specification, the relational terms such as first and second and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

Claims

1. A method for training a photovoltaic module recognition model, characterized in that, The application relates to a method for training a photovoltaic module recognition model. The method comprises the following steps: acquiring image data of photovoltaic modules in a first photovoltaic power station and a second photovoltaic power station; the image data is visible light image data obtained by aerial photography of a UAV; annotating the image data with a module frame to obtain the image data with the module frame; the module frame is a rotating frame containing the photovoltaic module; cropping the image data with the module frame and converting the coordinates of the module frame to the cropped image data; after the cropped image data is classified according to the first photovoltaic power station and the second photovoltaic power station, a training set and a test set are respectively divided out; the photovoltaic power stations in the training set and the test set are not repeated; constructing a photovoltaic module recognition model; the photovoltaic module recognition model comprises an attention mechanism module and a rotating target detection algorithm based on a YOLO algorithm; the rotating target detection algorithm based on the YOLO algorithm comprises a main network, a neck network and a head network connected in sequence; the attention mechanism module comprises a channel attention mechanism and a spatial attention mechanism connected in sequence; the attention mechanism module is located between the main network and the neck network; the output of a spatial pyramid pooling feature module in the main network is used as the input of the channel attention mechanism; the output of the spatial attention mechanism is used as the input of an up-sampling layer and the input of a connection layer in the neck network; training the photovoltaic module recognition model based on the training set and the test set to obtain a trained photovoltaic module recognition model, comprising: step 1: after the model training hyperparameters are set, the photovoltaic module recognition model is trained based on the training set, the photovoltaic module recognition model is iteratively optimized, and an optimized photovoltaic module recognition model is obtained; step 2: whether the optimized photovoltaic module recognition model converges is determined by the trend of a loss function; the loss function comprises a classification loss and a boundary box regression loss; the boundary box regression loss comprises a DFL loss and a CIoU loss; step 3: when it is detected that the optimized photovoltaic module recognition model converges, the test set is used to evaluate the converged photovoltaic module recognition model according to a model evaluation index, and whether the converged photovoltaic module recognition model meets an application precision requirement is determined; 2.The photovoltaic module identification model training method of claim 1, wherein, step 4: when it is detected that the converged photovoltaic module recognition model does not meet the application precision requirement, the step 1 is returned until the converged photovoltaic module recognition model meets the application precision requirement, and a trained photovoltaic module recognition model is obtained. 3.The photovoltaic module identification model training method of claim 1, wherein, The rotating target detection algorithm based on the YOLO algorithm comprises a YOLOv5 rotating target detection algorithm, a YOLOv8 rotating target detection algorithm and a YOLOv11 rotating target detection algorithm. ； In the formula, represents the classification loss; represents the intersection over union of the predicted component frame and the real component frame when the predicted component frame is a positive sample, and the predicted component frame is a component frame predicted by the photovoltaic component recognition model; represents the confidence of the predicted component frame, and are all artificial parameters; The expression of the classification loss is: ； In the formula, represents the DFL loss; represents the position of a pixel point; represents the coordinate value of the predicted component frame of the center point at the pixel point; represents the coordinate value of the predicted component frame of the center point at the pixel point; represents the coordinate value of the real component frame; represents the coordinate integral value of the real component frame; represents the coordinate adjacent integral value of the real component frame; The expression of the DFL loss is: ； wherein denotes the CIoU loss; denotes the center point of the real component box; denotes the center point of the predicted component box; denotes the Euclidean distance between the center point of the real component box and the center point of the predicted component box; is the diagonal distance of the minimum enclosing box of the real component box and the predicted component box, is a weight function, is a weight term; The expression is: ； In the formulae, represents the intersection over union; The expression is: ； wherein represents the width of the real component box; represents the height of the real component box; represents the width of the predicted component box; represents the height of the predicted component box; The expression of the CIoU loss is: The expression of the loss function is: ； In the formula, denotes the classification loss; denotes the DFL loss; denotes the CIoU loss. 4.The photovoltaic module identification model training method of any one of claims 1 to 3, wherein, The backbone network comprises a first convolutional block, four first module groups and a spatial pyramid pooling feature module connected in sequence; the first module group comprises a convolutional block and a C2f module connected in sequence; The input of the first convolutional block is taken as the input of the backbone network; The output of the C2f module of the third first module group is taken as the first output of the backbone network, for outputting first-level features; The output of the spatial pyramid pooling feature module is taken as the second output of the backbone network, for outputting second-level features; The second output of the backbone network is taken as the input of the channel attention mechanism. 5.The photovoltaic module identification model training method of claim 4, wherein, The neck network comprises a first neck branch and a second neck branch; the first neck branch comprises two second module groups connected in sequence; the second module group comprises an upsampling layer, a connection layer and a C2f module connected in sequence; the second neck branch comprises two third module groups connected in sequence; the third module group comprises a convolutional block, a connection layer and a C2f module connected in sequence; The first output of the backbone network is taken as the input of the connection layer of the first second module group; The output of the spatial attention mechanism is taken as the input of the upsampling layer of the first second module group, and as the input of the connection layer of the second third module group; The output of the C2f module of the second second module group is taken as the input of the convolutional block of the first third module group; the output of the convolutional block of the first third module group is taken as the first output of the neck network, for outputting first-scale fusion features; The output of the C2f module of the first third module group is taken as the second output of the neck network, for outputting second-scale fusion features; The output of the C2f module of the second third module group is taken as the third output of the neck network, for outputting third-scale fusion features. 6.The photovoltaic module identification model training method of claim 5, wherein, The head network comprises three detection heads; the first output, the second output and the third output of the neck network are taken as the inputs of the three detection heads respectively; The detection head comprises a first convolutional layer, a first head branch and a second head branch; The input of the first convolutional layer is taken as the input of the detection head; the output of the first convolutional layer is taken as the input of the first head branch, and as the input of the second head branch; The first head branch comprises three convolutional layers connected in sequence; the input of the first convolutional layer in the three convolutional layers is taken as the input of the first head branch; the output of the third convolutional layer in the three convolutional layers is taken as the output of the first head branch, for outputting whether the component frame is a photovoltaic component; The second head branch comprises a fourth module group, a first output branch and a second output branch; The input of the fourth module group is taken as the input of the second head branch; The output of the fourth module group is taken as the input of the first output branch, and as the input of the second output branch; The output of the first output branch is used as a first output of the second head branch for outputting coordinates of a component frame; the output of the second output branch is used as a second output of the second head branch for outputting confidence of the component frame; the fourth module group comprises two convolutional layers connected in sequence; and the first output branch and the second output branch each comprise a convolutional layer.

7. A photovoltaic module identification method characterized by, Comprise: Obtaining image data to be tested of photovoltaic components in a photovoltaic power station to be tested; the image data to be tested is visible light image data to be tested obtained by aerial photography of a UAV; Inputting the image data to be tested into a photovoltaic component recognition model to obtain coordinates of a component frame; the photovoltaic component recognition model is a model trained according to the photovoltaic component recognition model training method in any one of claims 1 to 6.

8. The photovoltaic module identification method of claim 7, wherein, After the step of inputting the image data to be tested into the photovoltaic component recognition model to obtain coordinates of a component frame, the method further comprises: Converting the coordinates of the component frame into geographic coordinates.

9. The photovoltaic module identification method of claim 7, wherein, After the step of inputting the image data to be tested into the photovoltaic component recognition model to obtain coordinates of a component frame, the method further comprises: Processing the coordinates of the component frame by using an NMS algorithm to obtain final coordinates of the component frame.

10. The photovoltaic module identification method of claim 7, wherein, The step of inputting the image data to be tested into the photovoltaic component recognition model to obtain coordinates of a component frame comprises: Cropping the image data to be tested; Inputting the cropped image data to be tested into the photovoltaic component recognition model to obtain coordinates of a component frame of the photovoltaic component in the cropped image data to be tested; Restoring the coordinates of the component frame of the photovoltaic component in the cropped image data to be tested to coordinates of a component frame in the image data to be tested before cropping. 11.A photovoltaic module identification model training system, comprising: Comprise: A training data acquisition module is configured to acquire image data of photovoltaic components in a first photovoltaic power station and a second photovoltaic power station; The image data is visible light image data obtained by aerial photography of a UAV; A labeling module is configured to perform component frame labeling on the image data to obtain the image data with the component frame; the component frame is a rotation frame containing the photovoltaic component; A first image processing module is configured to crop the image data with the component frame and convert coordinates of the component frame to the cropped image data; A second image processing module is configured to divide the cropped image data into a training set and a test set after classifying the cropped image data according to the first photovoltaic power station and the second photovoltaic power station; the photovoltaic power stations in the training set and the test set are not repeated; A model construction module is configured to construct a photovoltaic component recognition model; the photovoltaic component recognition model comprises an attention mechanism module and a rotation target detection algorithm based on a YOLO algorithm; the rotation target detection algorithm based on the YOLO algorithm comprises a backbone network, a head network and a neck network connected in sequence; the attention mechanism module is located between the backbone network and the head network; the attention mechanism module comprises a channel attention mechanism and a spatial attention mechanism connected in sequence; The training module is configured to train the photovoltaic module recognition model based on the training set and the test set to obtain a trained photovoltaic module recognition model, and includes: Step 1: After setting the model training hyperparameters, the photovoltaic module recognition model is trained based on the training set, and the photovoltaic module recognition model is optimized to obtain an optimized photovoltaic module recognition model; Step 2: Determine whether the optimized photovoltaic module recognition model converges by the trend of the loss function; the loss function includes a classification loss and a bounding box regression loss; the bounding box regression loss includes a DFL loss and a CIoU loss; Step 3: When it is detected that the optimized photovoltaic module recognition model converges, the test set is used to evaluate the converged photovoltaic module recognition model according to a model evaluation index to determine whether the converged photovoltaic module recognition model meets the application accuracy requirement; Step 4: When it is detected that the converged photovoltaic module recognition model does not meet the application accuracy requirement, return to Step 1 until the converged photovoltaic module recognition model meets the application accuracy requirement to obtain the trained photovoltaic module recognition model.

12. A photovoltaic module identification system, comprising: comprising: a to-be-measured data acquisition module configured to acquire to-be-measured image data of photovoltaic modules in a to-be-measured photovoltaic power station; the to-be-measured image data is to-be-measured visible light image data obtained by aerial photography of a UAV; an identification module configured to input the to-be-measured image data into a photovoltaic module recognition model to obtain coordinates of a module frame; the photovoltaic module recognition model is a model trained according to the photovoltaic module recognition model training method in any one of claims 1 to 6.

13. An electronic device, comprising: comprising: a memory configured to store a computer program; a processor configured to implement the steps of the photovoltaic module recognition model training method in any one of claims 1 to 6 and / or the steps of the photovoltaic module recognition method in any one of claims 7 to 10 when the computer program is executed.

14. A computer-readable storage medium, characterized in that: The computer program is stored on the computer-readable storage medium, and when executed by the processor, the computer program implements the steps of the photovoltaic module recognition model training method in any one of claims 1 to 6 and / or the steps of the photovoltaic module recognition method in any one of claims 7 to 10.

Images