Intelligent inspection system of centralized photovoltaic power station based on unmanned aerial vehicle

Abstract

The embodiment of the invention provides an intelligent inspection system for a centralized photovoltaic power station based on an unmanned aerial vehicle, and the system employs the unmanned aerial vehicle with a camera as a state collection device of a monitored photovoltaic module in the centralized photovoltaic power station, and carries out the analysis and processing of collected image data through the combination with a deep learning algorithm. Therefore, the surface defects of the photovoltaic module are identified. Therefore, automatic photovoltaic module surface defect detection can be realized, the detection efficiency is improved, and normal operation and continuous power generation of a photovoltaic power station can be ensured.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent inspection technology, and more specifically, to an intelligent inspection system for a centralized photovoltaic power station based on a drone. Background Art

[0002] A centralized photovoltaic power station is a facility that uses solar energy to generate electricity. It consists of a large number of photovoltaic modules that convert sunlight into electrical energy. However, photovoltaic modules may be affected by various factors during operation, resulting in surface defects such as cracks, stains, hot spots, etc. These defects will reduce the power generation efficiency of photovoltaic modules and even cause safety hazards. Therefore, regular inspection and maintenance of photovoltaic modules is an important measure to ensure the normal operation of centralized photovoltaic power stations.

[0003] Traditional inspection of photovoltaic modules is mainly carried out manually. The manual inspection method refers to professionals carrying corresponding instruments to inspect photovoltaic modules one by one. This method is complicated, time-consuming, labor-intensive, and costly, and is easily affected by human subjective factors, resulting in inaccurate test results.

[0004] Therefore, an optimized inspection scheme for centralized photovoltaic power stations is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present invention is proposed. An embodiment of the present invention provides an intelligent inspection system for a centralized photovoltaic power station based on a drone, which uses a drone equipped with a camera as a status acquisition device for the monitored photovoltaic components in the centralized photovoltaic power station, and combines a deep learning algorithm to analyze and process the collected image data to identify the surface defects of the photovoltaic components. In this way, automated photovoltaic component surface defect detection can be achieved and detection efficiency can be improved, thereby helping to ensure the normal operation and continuous power generation of the photovoltaic power station.

[0006] The embodiment of the present invention provides an intelligent inspection system for a centralized photovoltaic power station based on a drone, which includes:

[0007] A photovoltaic module image acquisition module, used to acquire a photovoltaic module image of a monitored photovoltaic module in a centralized photovoltaic power station collected by a camera of a drone;

[0008] An image semantic feature extraction module is used to extract image semantic features of a region of interest of the photovoltaic module image to obtain a plurality of image semantic feature vectors of the region of interest of the photovoltaic module;

[0009] A consistent topological feature extraction module is used to extract consistent topological features from the semantic feature vectors of the images of the plurality of photovoltaic component interest regions to obtain a local consistent topological feature matrix of the photovoltaic component;

[0010] A feature fusion module, used for fusing the plurality of photovoltaic component region of interest image semantic feature vectors and the photovoltaic component local consistent topological feature matrix to obtain a consistent topological global photovoltaic component image feature matrix; and

[0011] The surface defect determination module of the monitored photovoltaic assembly is used to determine whether the monitored photovoltaic assembly has surface defects based on the consistent topology global photovoltaic assembly image feature matrix.

[0012] In some possible embodiments, the image semantic feature extraction module includes:

[0013] a preprocessing unit, configured to preprocess the photovoltaic assembly image to obtain a preprocessed photovoltaic assembly image; and

[0014] The regional feature extraction unit is used to pass the preprocessed photovoltaic component image through a regional feature extractor based on the RCNN model to obtain semantic feature vectors of the plurality of photovoltaic component regions of interest.

[0015] In some possible embodiments, the region feature extraction unit is used to:

[0016] Using a selective search algorithm to segment and merge the preprocessed photovoltaic module image to obtain multiple regions of interest; and

[0017] The CNN model is used to extract features from the multiple candidate regions to obtain image semantic feature vectors of the multiple photovoltaic component regions of interest.

[0018] In some possible embodiments, the consistent topological feature extraction module includes:

[0019] a similarity association unit, configured to generate a photovoltaic assembly local consistency topology matrix based on a similarity association relationship between any two photovoltaic assembly region of interest image semantic feature vectors among the plurality of photovoltaic assembly region of interest image semantic feature vectors; and

[0020] The consistency feature extraction unit is used to extract features from the local consistency topology matrix of the photovoltaic assembly by using a deep learning network model to obtain the local consistency topology feature matrix of the photovoltaic assembly.

[0021] In some possible embodiments, the similarity association unit is used to:

[0022] The cosine similarity between any two of the plurality of photovoltaic assembly region of interest image semantic feature vectors is calculated to obtain the photovoltaic assembly local consistency topology matrix.

[0023] In some possible embodiments, the deep learning network model is a consistent topology feature extractor based on a convolutional neural network model;

[0024] Wherein, the consistency feature extraction unit is used to:

[0025] The local consistency topology matrix of the photovoltaic assembly is passed through the consistency topology feature extractor based on the convolutional neural network model to obtain the local consistency topology feature matrix of the photovoltaic assembly.

[0026] In some possible embodiments, the consistent topological feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

[0027] In some possible embodiments, the feature fusion module is used to:

[0028] The semantic feature vectors of the multiple photovoltaic component regions of interest images and the local consistent topological feature matrix of the photovoltaic components are passed through a graph neural network model to obtain the consistent topological global photovoltaic component image feature matrix.

[0029] In some possible embodiments, the surface defect determination module of the monitored photovoltaic module is used to:

[0030] The consistent topology global photovoltaic component image feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored photovoltaic component has a surface defect.

[0031] In some possible embodiments, it further includes a training module for training the regional feature extractor based on the RCNN model, the consistent topological feature extractor based on the convolutional neural network model, the graph neural network model and the classifier;

[0032] Wherein, the training module includes:

[0033] A training data acquisition unit, used to acquire training data, wherein the training data includes a training photovoltaic component image of a monitored photovoltaic component in a centralized photovoltaic power station collected by a camera of a drone, and a true value of whether the monitored photovoltaic component has a surface defect;

[0034] A training preprocessing unit, used for preprocessing the training photovoltaic assembly image to obtain a training preprocessed photovoltaic assembly image;

[0035] A training region feature extraction unit, used for passing the training preprocessed photovoltaic module image through the regional feature extractor based on the RCNN model to obtain a plurality of semantic feature vectors of the images of the regions of interest of the training photovoltaic modules;

[0036] A training region of interest distribution correction unit is used to perform feature distribution correction on the plurality of training photovoltaic assembly region of interest image semantic feature vectors to obtain a plurality of corrected photovoltaic assembly region of interest image semantic feature vectors;

[0037] A training cosine similarity calculation unit is used to calculate the cosine similarity between any two of the plurality of modified photovoltaic assembly region of interest image semantic feature vectors to obtain a training photovoltaic assembly local consistency topology matrix;

[0038] A training consistency topology feature extraction unit, used for passing the training photovoltaic assembly local consistency topology matrix through the consistency topology feature extractor based on the convolutional neural network model to obtain the training photovoltaic assembly local consistency topology feature matrix;

[0039] A training graph neural network unit is used to pass the plurality of training photovoltaic component interest region image semantic feature vectors and the training photovoltaic component local consistency topology feature matrix through the graph neural network model to obtain a training consistency topology global photovoltaic component image feature matrix;

[0040] A training feature distribution correction unit is used to perform feature distribution correction on the training consistent topology global photovoltaic assembly image feature matrix to obtain a corrected consistent topology global photovoltaic assembly image feature matrix;

[0041] A training classification unit is used to pass the modified consistent topology global photovoltaic component image feature matrix through a classifier to obtain a classification loss function value; and a training unit is used to train the regional feature extractor based on the RCNN model, the consistent topology feature extractor based on the convolutional neural network model, the graph neural network model and the classifier with the classification loss function value.

[0042] Compared with the prior art, the embodiments of the present invention utilize drones and automated image processing technology to quickly acquire and analyze images of a large number of photovoltaic modules, greatly improving the efficiency of inspections. By extracting image semantic features and consistent topological features, combined with feature fusion and defect detection algorithms, the surface defects of photovoltaic modules can be more accurately judged, reducing missed detections and false detections. Compared with traditional manual inspection methods, the use of automated image processing technology can reduce the need for professionals and reduce labor costs. Timely discovery and repair of surface defects of photovoltaic modules can ensure the normal operation of photovoltaic power stations and improve power generation efficiency and system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A block diagram of an intelligent inspection system for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention;

[0045] Figure 2 It is a flow chart of an intelligent inspection method of a centralized photovoltaic power station based on a drone according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of an intelligent inspection method architecture of a centralized photovoltaic power station based on a drone according to an embodiment of the present invention;

[0047] Figure 4 The figure is an application scenario diagram of an intelligent inspection system for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0049] Unless otherwise specified, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. "Including" or "comprising" used in the embodiments of the present invention neither limit the shapes, numbers, steps, actions, operations, components, originals and / or their groups mentioned, nor exclude the appearance or addition of one or more other different shapes, numbers, steps, actions, operations, components, originals and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number and order of the indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0050] Unless otherwise specifically stated, the relative arrangement of the components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship, and the techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices shown should be considered as part of the authorized specification. In all examples shown and discussed here, any specific other examples may have different values. It should be noted that similar symbols and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0051] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in the embodiments of the present invention and the features of the different embodiments or examples, without contradiction.

[0052] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0053] Photovoltaic modules may be affected by a variety of factors during operation, which may cause various defects on the surface of the modules, such as cracks, stains and hot spots. These defects will have an adverse effect on the power generation efficiency of the photovoltaic modules and may cause safety hazards.

[0054] Cracks on the surface of PV modules may be caused by mechanical stress, temperature changes, natural disasters or improper operation during installation. Cracks will reduce the structural strength of PV modules and may cause leakage, circuit interruption or module rupture. Stains on the surface of PV modules may be caused by the accumulation of impurities such as dust, sand, leaves, bird droppings in the atmosphere. These stains will reduce the light absorption capacity of PV modules and reduce the efficiency of converting light energy into electrical energy. Hot spots are hot spots formed by local temperature rise on the surface of PV modules. They are usually caused by local faults inside the module, such as short circuit of the cell, poor contact, etc. Hot spots will cause power loss of PV modules and may cause safety risks such as fire.

[0055] To ensure the normal operation and power generation efficiency of photovoltaic power stations, regular inspections and maintenance are crucial measures. Regular inspections can detect faults and defects of photovoltaic modules early, such as cracks, hot spots, stains, etc. Timely repair of these problems can avoid further damage and performance degradation, and ensure the normal operation of photovoltaic modules. By regularly cleaning the stains and dust on the surface of photovoltaic modules, the absorption efficiency of light can be improved, thereby improving the power generation efficiency. Cleaning can be carried out regularly, especially in dusty areas or seasons. Regular inspections and maintenance can detect and solve potential problems in time to prevent them from further developing into serious failures, which helps to extend the life of photovoltaic modules and improve the return on investment.

[0056] Traditional PV panel inspections are mainly conducted manually. Manual inspections require professionals to check each PV panel one by one, which requires a lot of time and human resources. Especially for large-scale PV power plants, the inspection process can be very cumbersome and time-consuming. Manual inspections require the employment of professionals and may require the use of special testing instruments and equipment, which can increase the operating costs of PV power plants. Manual inspections are easily affected by subjective factors such as fatigue, vision, and experience level, which can lead to certain deviations in the accuracy and consistency of the test results.

[0057] In order to overcome the limitations of traditional manual inspection methods, modern technology is being applied to the inspection and maintenance of photovoltaic modules to improve efficiency and accuracy. Using infrared thermal imagers to scan photovoltaic modules can detect problems such as abnormal surface temperature and hot spots on the modules, thereby detecting faults and defects early. Using drones equipped with high-resolution cameras or thermal imagers can quickly and comprehensively inspect photovoltaic modules. Drones can cover large areas of photovoltaic power stations, reducing the need for human resources while improving inspection efficiency and accuracy. By real-time monitoring and analysis of photovoltaic module data, combined with artificial intelligence technology, fault warning and automated inspection can be achieved. This method can improve inspection efficiency, reduce human errors, and detect and solve problems early.

[0058] In one embodiment of the present invention, Figure 1 FIG. 1 is a block diagram of an intelligent inspection system for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention. Figure 1 As shown, according to an embodiment of the present invention, an intelligent inspection system 100 for a centralized photovoltaic power station based on a drone includes: a photovoltaic module image acquisition module 110, which is used to acquire a photovoltaic module image of a monitored photovoltaic module in a centralized photovoltaic power station collected by a camera of a drone; an image semantic feature extraction module 120, which is used to extract image semantic features of an area of ​​interest of the photovoltaic module image to obtain multiple image semantic feature vectors of an area of ​​interest of the photovoltaic module; a consistent topological feature extraction module 130, which is used to extract consistent topological features from the multiple image semantic feature vectors of the area of ​​interest of the photovoltaic module to obtain a local consistent topological feature matrix of the photovoltaic module; a feature fusion module 140, which is used to fuse the multiple image semantic feature vectors of the area of ​​interest of the photovoltaic module and the local consistent topological feature matrix of the photovoltaic module to obtain a consistent topological global photovoltaic module image feature matrix; and a surface defect determination module 150 of the monitored photovoltaic module, which is used to determine whether the monitored photovoltaic module has surface defects based on the consistent topological global photovoltaic module image feature matrix.

[0059] In the photovoltaic component image acquisition module 110, the image of the monitored photovoltaic components in the centralized photovoltaic power station is acquired through the camera of the drone. It is ensured that the camera of the drone can acquire clear and high-quality photovoltaic component images for subsequent processing and analysis, and the route and shooting strategy of the drone are determined considering the layout of the photovoltaic power station and the distribution of photovoltaic components to ensure that all monitored photovoltaic components can be covered.

[0060] In the image semantic feature extraction module 120, image semantic features of the region of interest are extracted from the photovoltaic module image. The definition of the region of interest is determined, that is, the part of the photovoltaic module from which features need to be extracted, which can be defined according to the shape, position, etc. of the photovoltaic module. An appropriate image feature extraction method, such as a convolutional neural network (CNN), etc., is selected to extract a distinguishing feature vector.

[0061] In the consistent topological feature extraction module 130, consistent topological features are extracted from the semantic feature vectors of the images of the regions of interest of the multiple photovoltaic components. The definition of consistent topological features is determined, that is, the way to establish consistent relationships between the multiple photovoltaic components, which can be defined according to the layout and connection relationship of the photovoltaic components. An appropriate method is selected to extract consistent topological features, such as image segmentation, graph theory, etc., to obtain a local consistent topological feature matrix of the photovoltaic components.

[0062] In the feature fusion module 140, the semantic feature vector of the photovoltaic module region of interest image and the local consistent topological feature matrix of the photovoltaic module are fused to obtain a consistent topological global photovoltaic module image feature matrix. A suitable feature fusion method is determined, such as feature splicing, feature weighting, etc., to ensure that the contributions of different features are reasonably fused. Consider the dimension and scale of the feature to ensure that the fused feature matrix has a suitable dimension and scale for subsequent defect detection and analysis.

[0063] In the surface defect determination module 150 of the monitored photovoltaic module, it is determined whether the monitored photovoltaic module has surface defects based on the consistent topology global photovoltaic module image feature matrix. The definition and detection method of surface defects, such as detection algorithms for defects such as cracks and hot spots, are determined, and appropriate defect judgment criteria are established to determine whether the photovoltaic module has defects based on the analysis results of the feature matrix. The module is fully trained and verified to ensure accuracy and reliability.

[0064] It should be understood that the use of drones and automated image processing technology can quickly acquire and analyze a large number of images of photovoltaic modules, greatly improving the efficiency of inspections. By extracting image semantic features and consistent topological features, combined with feature fusion and defect detection algorithms, the surface defects of photovoltaic modules can be judged more accurately, reducing missed detections and false detections. Compared with traditional manual inspection methods, the use of automated image processing technology can reduce the demand for professionals and reduce labor costs. Timely detection and repair of surface defects of photovoltaic modules can ensure the normal operation of photovoltaic power stations and improve power generation efficiency and system reliability.

[0065] In response to the above technical problems, the technical concept of the present invention is to use a drone equipped with a camera as a status acquisition device for the monitored photovoltaic modules in a centralized photovoltaic power station, and combine it with a deep learning algorithm to analyze and process the collected image data to identify surface defects of the photovoltaic modules. Here, considering that drones have the characteristics of high flexibility and wide coverage, they can adapt to various terrains and environmental conditions, and can realize all-round inspections of centralized photovoltaic power stations.

[0066] Based on this, in the technical solution of the present invention, firstly, a photovoltaic module image of a monitored photovoltaic module in a centralized photovoltaic power station collected by a camera of a drone is obtained. Then, the image semantic features of the region of interest of the photovoltaic module image are extracted to obtain multiple semantic feature vectors of the region of interest of the photovoltaic module image. That is, the surface state feature information of the photovoltaic module contained in the photovoltaic module image is captured.

[0067] In a specific example of the present invention, the image semantic feature extraction module includes: a preprocessing unit, used to preprocess the photovoltaic component image to obtain a preprocessed photovoltaic component image; and a regional feature extraction unit, used to pass the preprocessed photovoltaic component image through a regional feature extractor based on an RCNN model to obtain semantic feature vectors of the images of the plurality of photovoltaic component regions of interest.

[0068] Among them, the regional feature extraction unit is used to: use a selective search algorithm to perform image segmentation and merging on the preprocessed photovoltaic component image to obtain multiple regions of interest; and use a CNN model to extract features from the multiple candidate regions to obtain semantic feature vectors of the images of the multiple photovoltaic component regions of interest.

[0069] Here, in the actual process of identifying surface defects of photovoltaic modules, the preprocessing means may include: using filtering algorithms (such as mean filtering, median filtering, Gaussian filtering) to reduce noise in the image to improve the clarity and detail visibility of the image; contrast enhancement: enhancing the contrast in the image through histogram equalization, adaptive histogram equalization or contrast stretching, etc., to make the defects more obvious; color correction: correcting the color deviation in the image and eliminating the color distortion in the photovoltaic module image to ensure the accuracy of subsequent processing. It should be understood that the preprocessing means may also include other means such as image distortion correction, which are not limited to the present invention.

[0070] More specifically, in an embodiment of the present invention, the encoding process of obtaining semantic feature vectors of multiple photovoltaic component region of interest images by passing the preprocessed photovoltaic component image through a regional feature extractor based on an RCNN model includes: firstly performing image segmentation and merging on the preprocessed photovoltaic component image using a selective search algorithm to obtain multiple regions of interest; then, performing feature extraction on the multiple candidate regions using a CNN model to obtain semantic feature vectors of the multiple photovoltaic component region of interest images. Here, the regional feature extractor based on the RCNN model can segment the preprocessed photovoltaic component image into several regions of interest, and perform feature extraction on each region of interest to extract a feature vector with semantic information.

[0071] Next, the consistent topological features are extracted from the semantic feature vectors of the images of the regions of interest of the multiple photovoltaic modules to obtain the local consistent topological feature matrix of the photovoltaic modules. It should be understood that if there are no surface defects in the photovoltaic module, there should be a high degree of similarity between its local image blocks. By extracting the consistent feature information, it can be used to measure the difference and similarity between the semantic features of each local image block, providing an important basis for judging whether there are surface defects in the photovoltaic module.

[0072] In a specific example of the present invention, the consistency topology feature extraction module includes: a similarity association unit, which is used to generate a local consistency topology matrix of photovoltaic components based on the similarity association relationship between any two semantic feature vectors of the images of the regions of interest of the photovoltaic components among the multiple semantic feature vectors of the images of the regions of interest of the photovoltaic components; and a consistency feature extraction unit, which is used to perform feature extraction on the local consistency topology matrix of the photovoltaic components using a deep learning network model to obtain the local consistency topology feature matrix of the photovoltaic components.

[0073] The similarity association unit is used to calculate the cosine similarity between any two semantic feature vectors of the plurality of semantic feature vectors of the photovoltaic component region of interest images to obtain the photovoltaic component local consistency topology matrix.

[0074] Furthermore, in a specific embodiment of the present invention, the deep learning network model is a consistent topological feature extractor based on a convolutional neural network model; wherein the consistent feature extraction unit is used to: pass the local consistent topological matrix of the photovoltaic component through the consistent topological feature extractor based on the convolutional neural network model to obtain the local consistent topological feature matrix of the photovoltaic component.

[0075] Specifically, the consistent topological feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

[0076] Furthermore, the semantic feature vectors of the images of the regions of interest of the multiple photovoltaic components and the local consistency topological feature matrix of the photovoltaic components are passed through a graph neural network model to obtain a consistent topological global photovoltaic component image feature matrix. Here, the graph neural network model can regard the photovoltaic component image as a graph structure data composed of nodes and edges, that is, the semantic feature vectors of the images of the regions of interest of the multiple photovoltaic components are regarded as node information, and the local consistency topological feature matrix of the photovoltaic components is regarded as the edge information between nodes, and the connection relationship between nodes is used to transmit and aggregate information, so as to learn a higher level and more abstract feature representation. That is, the complex structure and rich semantic information in the photovoltaic component image are learned, and the model is guided to pay attention to the relationship between the semantic features of the local image blocks of the photovoltaic component image.

[0077] In a specific embodiment of the present invention, the feature fusion module is used to: pass the semantic feature vectors of the multiple photovoltaic component area of ​​interest images and the local consistent topology feature matrix of the photovoltaic components through a graph neural network model to obtain the consistent topology global photovoltaic component image feature matrix.

[0078] Then, the consistent topology global photovoltaic component image feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored photovoltaic component has surface defects.

[0079] In a specific embodiment of the present invention, the surface defect determination module of the monitored photovoltaic component is used to: pass the consistent topology global photovoltaic component image feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored photovoltaic component has surface defects.

[0080] By using a classifier to classify the image features of photovoltaic modules, the surface defects of the monitored photovoltaic modules can be automatically detected. Compared with the traditional manual inspection method, this automated defect detection method can greatly reduce the impact of manual operation and subjective judgment, and improve the accuracy and consistency of detection. Using a classifier for defect detection can quickly process a large number of photovoltaic module image feature matrices and give corresponding classification results, which can greatly improve the efficiency of detection and save time and labor costs. By timely detecting the surface defects of photovoltaic modules, problems can be discovered and solved before they deteriorate further, which helps prevent the expansion of defects, reduce the damage of photovoltaic modules and the downtime of power stations, and improve the reliability and operation efficiency of photovoltaic power stations.

[0081] In one embodiment of the present invention, the intelligent inspection system for centralized photovoltaic power stations based on drones further includes a training module for training the regional feature extractor based on the RCNN model, the consistent topology feature extractor based on the convolutional neural network model, the graph neural network model and the classifier; wherein the training module includes: a training data acquisition unit for acquiring training data, wherein the training data includes a training photovoltaic component image of a monitored photovoltaic component in a centralized photovoltaic power station collected by a camera of the drone, and a true value of whether the monitored photovoltaic component has surface defects; a training preprocessing unit for preprocessing the training photovoltaic component image to obtain a training preprocessed photovoltaic component image; a training regional feature extraction unit for passing the training preprocessed photovoltaic component image through the regional feature extractor based on the RCNN model to obtain a plurality of training photovoltaic component region of interest image semantic feature vectors; a training region of interest distribution correction unit for performing feature distribution correction on the plurality of training photovoltaic component region of interest image semantic feature vectors to obtain a plurality of corrected photovoltaic component region of interest image semantic feature vectors; a training cosine similarity calculation unit for calculating the plurality of corrected photovoltaic component region of interest images. The cosine similarity between any two corrected photovoltaic component region of interest image semantic feature vectors in the image semantic feature vector is used to obtain the training photovoltaic component local consistency topology matrix; the training consistency topology feature extraction unit is used to pass the training photovoltaic component local consistency topology matrix through the consistency topology feature extractor based on the convolutional neural network model to obtain the training photovoltaic component local consistency topology feature matrix; the training graph neural network unit is used to pass the multiple training photovoltaic component region of interest image semantic feature vectors and the training photovoltaic component local consistency topology feature matrix through the graph neural network model to obtain the training consistency topology global photovoltaic component image feature matrix; the training feature distribution correction unit is used to perform feature distribution correction on the training consistency topology global photovoltaic component image feature matrix to obtain the corrected consistency topology global photovoltaic component image feature matrix; the training classification unit is used to pass the corrected consistency topology global photovoltaic component image feature matrix through a classifier to obtain a classification loss function value; and the training unit is used to train the regional feature extractor based on the RCNN model, the consistency topology feature extractor based on the convolutional neural network model, the graph neural network model and the classifier with the classification loss function value.

[0082] In the technical solution of the present invention, each of the multiple training photovoltaic component region of interest image semantic feature vectors expresses the image semantic features of the region of interest of the corresponding preprocessed training photovoltaic component image in the local image space domain. In this way, after the training image block consistency topological feature matrix and the sequence of the training photovoltaic component region of interest image semantic feature vectors are passed through the graph neural network model, the topological association representation of the image semantic features in the local image space domain under the image semantic feature similarity topology in the global image space domain can be further obtained. However, considering the expression independence of each training global photovoltaic component region of interest image semantic feature vector corresponding to the training photovoltaic component region of interest image semantic feature vector of the training consistency topology global photovoltaic component image feature matrix, such as the row training feature vector, the training consistency topology global photovoltaic component image feature matrix as a whole may still have an imbalance in the expression of image semantic features in each local image space domain.

[0083] Here, the inventors of the present invention have discovered that this imbalance is largely related to the feature expression scale, that is, the image semantic feature expression scale of the local image space domain of the training feature vector, and the image semantic feature association scale of the local image space domain distribution of the feature matrix between each training feature vector in the global image space domain. For example, it can be understood that relative to the scale of image space domain division, the more unbalanced the scale distribution within the local image space domain and between the local image space domains is relative to the image semantic feature distribution, the more unbalanced the overall expression of the training consistent topology global photovoltaic component image feature matrix is.

[0084] Therefore, preferably, for each of the plurality of training photovoltaic assembly region of interest image semantic feature vectors, for example, denoted as V i And the training consistent topology global photovoltaic component image feature matrix, for example, denoted as M, is optimized based on feature scale, which is expressed as: each training photovoltaic component region of interest image semantic feature vector in the multiple training photovoltaic component region of interest image semantic feature vectors and the training consistent topology global photovoltaic component image feature matrix are optimized based on feature scale to obtain a modified consistent topology global photovoltaic component image feature matrix; wherein the optimization formula is:

[0085]

[0086] Among them, V iis each semantic feature vector of the image region of interest of the plurality of training photovoltaic assembly image semantic feature vectors, and L is the semantic feature vector V of the image region of interest of the training photovoltaic assembly. i The length, v ij is the semantic feature vector V of the image of the region of interest of the training photovoltaic module i The jth eigenvalue of Represents the semantic feature vector V of the image of the region of interest of the training photovoltaic module i The square of the second norm of , S is the scale of the training consistency topology global photovoltaic module image feature matrix, that is, the width multiplied by the height, and represents the square of the Frobenius norm of the training consistency topology global photovoltaic module image feature matrix, m i,j are the i-th and j-th eigenvalues ​​of the training consistency topology global photovoltaic module image feature matrix, w 1i represents a weight coefficient for weighting each of the plurality of training PV module ROI image semantic feature vectors, w 2 represents the weight coefficient for weighting the modified consistent topology global photovoltaic component image feature matrix obtained by the sequence of the modified training photovoltaic component region of interest image semantic feature vectors; and, with the weight w 1i Each of the plurality of training photovoltaic assembly region of interest image semantic feature vectors is weighted, and the weight w is used to calculate the semantic feature vector of the region of interest of the photovoltaic assembly. 2 The modified consistent topology global photovoltaic module image feature matrix obtained by the sequence of modified training photovoltaic module region of interest image semantic feature vectors is weighted.

[0087] Here, the optimization based on feature scale can be performed through the tail distribution reinforcement mechanism of the standard Cauchy distribution to constrain the correlation of the multi-level distribution structure of the feature probability density distribution in the high-dimensional feature space based on the feature scale, so that the probability density distribution of high-dimensional features with different scales can be evenly spread in the overall probability density space, thereby compensating for the heterogeneity of probability density convergence caused by feature scale deviation. In this way, at each iteration in the training process, the weight w 1i Each of the plurality of training photovoltaic assembly region of interest image semantic feature vectors is weighted, and the weight w is used to represent the semantic feature vector of the region of interest of the photovoltaic assembly. 2By weighting the modified consistent topology global photovoltaic component image feature matrix obtained from the sequence of semantic feature vectors of the modified training photovoltaic component area of ​​interest image, the expression convergence of the modified consistent topology global photovoltaic component image feature matrix in the probability density domain can be improved, thereby improving the accuracy of the classification results obtained by the classifier.

[0088] In summary, an intelligent inspection system 100 for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention is explained, which utilizes a drone equipped with a camera as a status acquisition device for monitored photovoltaic components in a centralized photovoltaic power station, and combines a deep learning algorithm to analyze and process the collected image data, so as to identify surface defects of photovoltaic components.

[0089] As described above, the intelligent inspection system 100 for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention can be implemented in various terminal devices, such as a server for intelligent inspection of a centralized photovoltaic power station based on a drone, etc. In one example, the intelligent inspection system 100 for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention can be integrated into a terminal device as a software module and / or a hardware module. For example, the intelligent inspection system 100 for a centralized photovoltaic power station based on a drone can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intelligent inspection system 100 for a centralized photovoltaic power station based on a drone can also be one of the many hardware modules of the terminal device.

[0090] Alternatively, in another example, the UAV-based intelligent inspection system 100 for a centralized photovoltaic power station and the terminal device may be separate devices, and the UAV-based intelligent inspection system 100 for a centralized photovoltaic power station may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0091] In one embodiment of the present invention, Figure 2 The present invention is a flowchart of an intelligent inspection method for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention. Figure 3 FIG. 1 is a schematic diagram of an intelligent inspection method architecture of a centralized photovoltaic power station based on a drone according to an embodiment of the present invention. Figure 2 and Figure 3As shown, the intelligent inspection method of the centralized photovoltaic power station based on the drone includes: 210, acquiring the photovoltaic component image of the monitored photovoltaic component in the centralized photovoltaic power station collected by the camera of the drone; 220, extracting the image semantic features of the region of interest of the photovoltaic component image to obtain multiple photovoltaic component region of interest image semantic feature vectors; 230, extracting the consistent topological features from the multiple photovoltaic component region of interest image semantic feature vectors to obtain a photovoltaic component local consistent topological feature matrix; 240, fusing the multiple photovoltaic component region of interest image semantic feature vectors and the photovoltaic component local consistent topological feature matrix to obtain a consistent topological global photovoltaic component image feature matrix; and, 250, based on the consistent topological global photovoltaic component image feature matrix, determining whether the monitored photovoltaic component has surface defects.

[0092] Those skilled in the art will appreciate that the specific operations of each step in the above-mentioned intelligent inspection method for centralized photovoltaic power stations based on drones have been described in detail above. Figure 1 The invention has been introduced in detail in the description of the intelligent inspection system of centralized photovoltaic power station based on drones, and therefore, its repeated description will be omitted.

[0093] Figure 4 FIG. 1 is an application scenario diagram of an intelligent inspection system for a centralized photovoltaic power station based on a drone according to an embodiment of the present invention. Figure 4 As shown, in this application scenario, first, a photovoltaic component image of a monitored photovoltaic component in a centralized photovoltaic power station collected by a camera of a drone is obtained (for example, Figure 4 Then, the acquired photovoltaic component image is input to a server (for example, Figure 4 In S) shown in , the server is capable of processing the photovoltaic component image based on the intelligent inspection algorithm of the centralized photovoltaic power station of the drone to determine whether the monitored photovoltaic component has surface defects.

[0094] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent inspection system for centralized photovoltaic power stations based on drones, characterized in that: include: A photovoltaic module image acquisition module, used to acquire a photovoltaic module image of a monitored photovoltaic module in a centralized photovoltaic power station collected by a camera of a drone; An image semantic feature extraction module is used to extract image semantic features of a region of interest of the photovoltaic module image to obtain a plurality of image semantic feature vectors of the region of interest of the photovoltaic module; A consistent topological feature extraction module is used to extract consistent topological features from the semantic feature vectors of the images of the plurality of photovoltaic component interest regions to obtain a local consistent topological feature matrix of the photovoltaic component; A feature fusion module, used for fusing the plurality of photovoltaic component region of interest image semantic feature vectors and the photovoltaic component local consistent topological feature matrix to obtain a consistent topological global photovoltaic component image feature matrix; and The surface defect determination module of the monitored photovoltaic assembly is used to determine whether the monitored photovoltaic assembly has surface defects based on the consistent topology global photovoltaic assembly image feature matrix.

2. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 1 is characterized in that: The image semantic feature extraction module comprises: a preprocessing unit, configured to preprocess the photovoltaic assembly image to obtain a preprocessed photovoltaic assembly image; and The regional feature extraction unit is used to pass the preprocessed photovoltaic component image through a regional feature extractor based on the RCNN model to obtain semantic feature vectors of the plurality of photovoltaic component regions of interest.

3. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 2 is characterized in that: The regional feature extraction unit is used to: Using a selective search algorithm to segment and merge the preprocessed photovoltaic module image to obtain multiple regions of interest; and The CNN model is used to extract features from the multiple candidate regions to obtain image semantic feature vectors of the multiple photovoltaic component regions of interest.

4. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 3 is characterized in that: The consistent topological feature extraction module comprises: a similarity association unit, configured to generate a photovoltaic assembly local consistency topology matrix based on a similarity association relationship between any two photovoltaic assembly region of interest image semantic feature vectors among the plurality of photovoltaic assembly region of interest image semantic feature vectors; and The consistency feature extraction unit is used to extract features from the local consistency topology matrix of the photovoltaic assembly by using a deep learning network model to obtain the local consistency topology feature matrix of the photovoltaic assembly.

5. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 4 is characterized in that: The similarity association unit is used to: The cosine similarity between any two of the plurality of photovoltaic assembly region of interest image semantic feature vectors is calculated to obtain the photovoltaic assembly local consistency topology matrix.

6. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 5 is characterized in that: The deep learning network model is a consistent topological feature extractor based on a convolutional neural network model; Wherein, the consistency feature extraction unit is used to: The local consistency topology matrix of the photovoltaic assembly is passed through the consistency topology feature extractor based on the convolutional neural network model to obtain the local consistency topology feature matrix of the photovoltaic assembly.

7. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 6 is characterized in that: The consistent topological feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

8. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 7 is characterized in that: The feature fusion module is used to: The semantic feature vectors of the multiple photovoltaic component regions of interest images and the local consistent topological feature matrix of the photovoltaic components are passed through a graph neural network model to obtain the consistent topological global photovoltaic component image feature matrix.

9. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 8 is characterized in that: The surface defect determination module of the monitored photovoltaic module is used for: The consistent topology global photovoltaic component image feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored photovoltaic component has a surface defect.

10. The intelligent inspection system for centralized photovoltaic power stations based on drones according to claim 9 is characterized in that: It also includes a training module for training the regional feature extractor based on the RCNN model, the consistent topological feature extractor based on the convolutional neural network model, the graph neural network model and the classifier; Wherein, the training module includes: A training data acquisition unit, used to acquire training data, wherein the training data includes a training photovoltaic component image of a monitored photovoltaic component in a centralized photovoltaic power station collected by a camera of a drone, and a true value of whether the monitored photovoltaic component has a surface defect; A training preprocessing unit, used for preprocessing the training photovoltaic assembly image to obtain a training preprocessed photovoltaic assembly image; A training region feature extraction unit, used for passing the training preprocessed photovoltaic module image through the regional feature extractor based on the RCNN model to obtain a plurality of semantic feature vectors of the images of the regions of interest of the training photovoltaic modules; A training region of interest distribution correction unit is used to perform feature distribution correction on the plurality of training photovoltaic assembly region of interest image semantic feature vectors to obtain a plurality of corrected photovoltaic assembly region of interest image semantic feature vectors; A training cosine similarity calculation unit is used to calculate the cosine similarity between any two of the plurality of modified photovoltaic assembly region of interest image semantic feature vectors to obtain a training photovoltaic assembly local consistency topology matrix; A training consistency topology feature extraction unit, used for passing the training photovoltaic assembly local consistency topology matrix through the consistency topology feature extractor based on the convolutional neural network model to obtain the training photovoltaic assembly local consistency topology feature matrix; A training graph neural network unit is used to pass the plurality of training photovoltaic component interest region image semantic feature vectors and the training photovoltaic component local consistency topology feature matrix through the graph neural network model to obtain a training consistency topology global photovoltaic component image feature matrix; A training feature distribution correction unit is used to perform feature distribution correction on the training consistent topology global photovoltaic assembly image feature matrix to obtain a corrected consistent topology global photovoltaic assembly image feature matrix; A training classification unit is used to pass the modified consistent topology global photovoltaic assembly image feature matrix through a classifier to obtain a classification loss function value; and A training unit is used to train the regional feature extractor based on the RCNN model, the consistent topology feature extractor based on the convolutional neural network model, the graph neural network model and the classifier with the classification loss function value.

Images