Photovoltaic array operation state monitoring system and method based on unmanned aerial vehicle

Abstract

The embodiment of the invention relates to the technical field of intelligent monitoring, and particularly discloses a photovoltaic array operation state monitoring system and method based on an unmanned aerial vehicle, and the method comprises the steps: carrying out the feature analysis of surface images of all photovoltaic panels in a photovoltaic array through employing a machine vision technology based on deep learning, extracting the surface state features of each photovoltaic panel, and carrying out the recognition of the surface state features of each photovoltaic panel; and judging whether each photovoltaic panel is an abnormal photovoltaic panel or not according to the semantic association degree between the surface state feature of each photovoltaic panel and the overall average feature. Therefore, the operation state of the photovoltaic array can be accurately monitored, so that the abnormal photovoltaic panel can be timely found and processed, and the operation efficiency and the stability of the whole photovoltaic power generation system are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent monitoring technology, and more specifically, to a photovoltaic array operation status monitoring system and method based on an unmanned aerial vehicle. Background Art

[0002] With the rapid development of renewable energy, photovoltaic (PV) energy, as a clean, pollution-free energy source, has gained widespread global application. As a core component of a PV energy system, the operating status of the PV array directly impacts the overall system's power generation efficiency and service life. Therefore, real-time monitoring of the PV array and the timely detection and resolution of abnormal PV panels are crucial for ensuring stable PV system operation and improving power generation efficiency.

[0003] Traditional PV array monitoring methods typically rely on manual inspections and periodic performance testing. This approach is not only inefficient and costly, but also makes it difficult to detect and address PV panel anomalies in a timely and accurate manner. Furthermore, as PV arrays continue to expand in size and the number of panels increases, traditional monitoring methods are no longer able to meet the needs for comprehensive and accurate PV array monitoring.

[0004] In recent years, with the rapid development of drone technology, a new solution has been provided for the operation monitoring of photovoltaic arrays. Therefore, a system and method for monitoring the operation status of photovoltaic arrays based on drones are expected. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art and provides a photovoltaic array operation status monitoring system and method based on a drone.

[0006] In a first aspect, an embodiment of the present invention provides a photovoltaic array operation status monitoring system based on a drone, comprising:

[0007] A photovoltaic panel surface image acquisition module is used to acquire surface state images of all photovoltaic panels in the photovoltaic array through a camera carried by a drone to obtain a set of photovoltaic panel surface state images;

[0008] a photovoltaic panel surface state feature extraction module, configured to extract image features from each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps;

[0009] A feature enhancement module, configured to perform spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors;

[0010] a semantic association measurement module, configured to respectively calculate the semantic association of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall features of the set to obtain a set of semantic associations;

[0011] The abnormal photovoltaic panel identification module is used to set the photovoltaic panels corresponding to the semantic relevance values less than or equal to a preset threshold value in the set of semantic relevance values as abnormal photovoltaic panels.

[0012] In some possible embodiments, the photovoltaic panel surface state feature extraction module is used to: pass each photovoltaic panel surface state image in the set of photovoltaic panel surface state images through a surface state feature extractor based on a void convolutional neural network model to obtain a set of photovoltaic panel surface state feature maps.

[0013] In some possible embodiments, the feature enhancement module includes: a spatial mask unit, which is used to pass each photovoltaic panel surface state feature map in the set of photovoltaic panel surface state feature maps through a content discrimination enhancement module based on a spatial mask to obtain a set of enhanced photovoltaic panel surface state feature maps; a multi-scale feature focusing unit, which is used to pass each enhanced photovoltaic panel surface state feature map in the set of enhanced photovoltaic panel surface state feature maps through a multi-scale feature focusing module to obtain a set of enhanced photovoltaic panel surface state focusing feature vectors.

[0014] In some possible embodiments, the spatial mask unit includes: a spatial significance scoring subunit, used to calculate the photovoltaic panel surface state feature significance score map of the photovoltaic panel surface state feature map; a masking subunit, used to mask the photovoltaic panel surface state feature significance score map based on a predetermined threshold to obtain a spatial significance attention mask map; a pixel-by-pixel weighting subunit, used to use the spatial significance attention mask map as a weight, and calculate the position point multiplication of the photovoltaic panel surface state feature map and the spatial significance attention mask map to obtain the enhanced photovoltaic panel surface state feature map.

[0015] In some possible embodiments, the spatial significance scoring subunit is used to: calculate the inverse of the exponential function value with a natural constant as the base of each eigenvalue of the photovoltaic panel surface state characteristic map to obtain an indexed photovoltaic panel surface state characteristic map; calculate the cumulative sum of each eigenvalue of the indexed photovoltaic panel surface state characteristic map to obtain the global feature intensity of the photovoltaic panel surface state characteristic map; calculate the addition result of the global feature intensity and one, and divide the eigenvalue of each position of the indexed photovoltaic panel surface state characteristic map by the addition result to obtain the photovoltaic panel surface state feature significance score map.

[0016] In some possible embodiments, the multi-scale feature focusing unit includes: a multi-scale semantic feature extraction sub-unit, used to respectively extract the global semantic features and local semantic features of the enhanced photovoltaic panel surface state feature map to obtain the enhanced photovoltaic panel surface state global semantic feature vector and the enhanced photovoltaic panel surface state local semantic feature vector; a multi-scale semantic feature fusion sub-unit, used to fuse the enhanced photovoltaic panel surface state global semantic feature vector and the enhanced photovoltaic panel surface state local semantic feature vector to obtain the enhanced photovoltaic panel surface state focusing feature vector.

[0017] In some possible embodiments, the multi-scale semantic feature extraction subunit includes: an upsampling secondary subunit, used to upsample the surface state feature map of the enhanced photovoltaic panel to obtain an upsampled enhanced photovoltaic panel surface state feature map; a global semantic feature extraction secondary subunit, used to pass the upsampled enhanced photovoltaic panel surface state feature map through a global semantic feature extraction module to obtain a global semantic feature vector of the surface state of the enhanced photovoltaic panel, wherein the global semantic feature extraction module includes a global average pooling layer, a first point convolution layer, a first batch normalization layer and a first activation layer; a local semantic feature extraction secondary subunit, used to pass the upsampled enhanced photovoltaic panel surface state feature map through a local semantic feature extraction module to obtain a local semantic feature vector of the surface state of the enhanced photovoltaic panel, wherein the local semantic feature extraction module includes a second point convolution layer, a second batch normalization layer and a second activation layer.

[0018] In some possible embodiments, the semantic association measurement module includes: a feature distribution sequence cluster center calculation unit, used to calculate the global mean vector of the set of enhanced photovoltaic panel surface state focused feature vectors as the feature distribution sequence cluster center; a semantic association calculation unit, used to respectively calculate the semantic association between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center to obtain the set of semantic associations.

[0019] In some possible embodiments, the semantic association calculation unit is used to: respectively calculate the matrix product between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and each predetermined weight matrix to obtain a set of enhanced photovoltaic panel surface state semantic coding feature vectors; add a bias vector to each enhanced photovoltaic panel surface state semantic coding feature vector in the set of enhanced photovoltaic panel surface state semantic coding feature vectors and then pass the resultant data through a tanh activation function to obtain a set of activated enhanced photovoltaic panel surface state semantic coding feature vectors; respectively calculate the vector product between each activated enhanced photovoltaic panel surface state semantic coding feature vector in the set of activated enhanced photovoltaic panel surface state semantic coding feature vectors and the transposed vector of the feature distribution sequence clustering center to obtain the set of semantic associations.

[0020] In a second aspect, an embodiment of the present invention provides a method for real-time monitoring of the operating status of a photovoltaic array based on a drone, comprising:

[0021] The surface state images of all photovoltaic panels in the photovoltaic array are collected by a camera carried by the drone to obtain a set of photovoltaic panel surface state images;

[0022] performing image feature extraction on each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps;

[0023] Performing spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors;

[0024] respectively calculating the semantic relevance of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall features of the set to obtain a set of semantic relevance;

[0025] The photovoltaic panels corresponding to the semantic relevance values less than or equal to a preset threshold value in the set of semantic relevance values are set as abnormal photovoltaic panels.

[0026] Compared to existing technologies, the drone-based photovoltaic array operating status monitoring system and method provided by the present invention employs deep learning-based machine vision technology to perform feature analysis on surface images of all photovoltaic panels in the array, extracting the surface state characteristics of each panel. Based on the semantic correlation between each panel's surface state characteristics and the overall average characteristics, the system determines whether each panel is abnormal. This allows for accurate monitoring of the photovoltaic array's operating status, enabling the timely detection and resolution of abnormal panels and improving the overall efficiency and stability of the photovoltaic power generation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 4 is a block diagram of a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention.

[0029] Figure 2 Schematic diagram of the architecture of a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention.

[0030] Figure 3 4 is a block diagram of a feature enhancement module in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention.

[0031] Figure 4 4 is a block diagram of a spatial mask unit in a photovoltaic array operating status monitoring system based on a drone according to an embodiment of the present invention.

[0032] Figure 5 4 is a block diagram of a multi-scale feature focusing unit in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention.

[0033] Figure 6 4 is a block diagram of a semantic association measurement module in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention.

[0034] Figure 7 The figure is a flow chart of a method for real-time monitoring of photovoltaic array operating status based on a drone according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It is apparent that the described embodiments are only a portion of the embodiments of the present invention, rather than all of them. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without requiring creative effort are within the scope of protection of the present invention.

[0036] Unless otherwise specified, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The terms "including" or "comprising" used in the embodiments of the present invention neither limit the shapes, numbers, steps, actions, operations, components, originals and / or their groups mentioned, nor exclude the appearance or addition of one or more other different shapes, numbers, steps, actions, operations, components, originals and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number and order of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0037] Unless otherwise specifically stated, the relative arrangements of the components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn in accordance with actual proportional relationships, and that the techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices shown should be considered part of the authorized specification. In all examples shown and discussed herein, any specific other examples may have different values. It should be noted that similar symbols and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0038] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in the embodiments of the present invention and the features of different embodiments or examples, unless they are mutually inconsistent.

[0039] Flowcharts are used in this disclosure to illustrate the operations performed by systems according to embodiments of the present invention. It should be understood that the preceding or following operations do not necessarily need to be performed in exact order. Instead, various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0040] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0041] As mentioned in the background technology above, traditional photovoltaic array monitoring methods mainly rely on manual inspections and regular performance evaluations. This method is both inefficient and costly, and has limitations in timely detecting and handling abnormal conditions of photovoltaic panels. In addition, with the continuous expansion of the scale of photovoltaic arrays and the increase in the number of photovoltaic panels, traditional monitoring methods can no longer meet the needs of comprehensive and accurate monitoring of photovoltaic arrays. In response to the above technical problems, the technical concept of the present invention is to use machine vision technology based on deep learning to perform feature analysis on the surface images of all photovoltaic panels in the photovoltaic array, extract the surface state features of each photovoltaic panel, and judge whether each photovoltaic panel is an abnormal photovoltaic panel based on the semantic correlation between the surface state features of each photovoltaic panel and the overall average features. In this way, accurate monitoring of the operating status of the photovoltaic array can be achieved, so as to timely detect and handle abnormal photovoltaic panels and improve the operating efficiency and stability of the entire photovoltaic power generation system.

[0042] Figure 1 4 is a block diagram of a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention. Figure 2 FIG. 1 is a schematic diagram of the architecture of a photovoltaic array operating status monitoring system based on a drone according to an embodiment of the present invention. Figure 1 and Figure 2As shown, a photovoltaic array operation status monitoring system 100 based on a drone according to an embodiment of the present invention includes: a photovoltaic panel surface image acquisition module 110, which is used to acquire surface status images of all photovoltaic panels in a photovoltaic array by using a camera mounted on a drone to obtain a set of photovoltaic panel surface status images; a photovoltaic panel surface status feature extraction module 120, which is used to perform image feature extraction on each photovoltaic panel surface status image in the set of photovoltaic panel surface status images to obtain a set of photovoltaic panel surface status feature maps; a feature enhancement module 130, which is used to perform spatial dimension feature enhancement processing on the set of photovoltaic panel surface status feature maps to obtain a set of enhanced photovoltaic panel surface status focused feature vectors; a semantic association measurement module 140, which is used to calculate the semantic association of each enhanced photovoltaic panel surface status focused feature vector in the set of enhanced photovoltaic panel surface status focused feature vectors relative to the overall feature of the set to obtain a set of semantic associations; and an abnormal photovoltaic panel identification module 150, which is used to set photovoltaic panels corresponding to semantic associations less than or equal to a preset threshold in the set of semantic associations as abnormal photovoltaic panels.

[0043] In the above-mentioned drone-based photovoltaic array operation status monitoring system 100, the photovoltaic panel surface image acquisition module 110 is used to collect surface status images of all photovoltaic panels in the photovoltaic array through the camera carried by the drone to obtain a collection of photovoltaic panel surface status images. It should be understood that drones have the advantages of high-altitude viewing angle, high flexibility, and wide coverage. They can fly according to pre-set routes and programs to cover the entire photovoltaic array and capture surface status images of photovoltaic panels according to preset angles and heights to ensure comprehensive and rapid inspection of the entire photovoltaic array. Compared with manual inspections or fixed camera collection, drones can complete the data collection process more quickly, achieve comprehensive monitoring coverage and efficient data collection, thereby reducing labor costs and improving monitoring efficiency. At the same time, the surface status images of photovoltaic panels collected by the high-resolution camera carried by the drone can capture subtle defects on the surface of the photovoltaic panels, providing a high-quality data foundation for subsequent photovoltaic panel anomaly detection.

[0044] In the above-mentioned drone-based photovoltaic array operation status monitoring system 100, the photovoltaic panel surface state feature extraction module 120 is used to extract image features from each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps. It should be understood that due to the complexity and diversity of photovoltaic panel surface conditions, which may include various abnormal conditions such as stains, cracks, and damage, traditional rule-based or threshold-based image processing methods may find it difficult to effectively capture the complex conditions of the photovoltaic panel surface conditions. Therefore, in the technical solution of the present invention, deep learning-based image processing technology is used to extract features from the photovoltaic panel surface state images. In a specific example of the present invention, the encoding method for extracting image features from each photovoltaic panel surface state image in the set of photovoltaic panel surface state images is to pass each photovoltaic panel surface state image in the set of photovoltaic panel surface state images through a surface state feature extractor based on a void convolutional neural network model to obtain the set of photovoltaic panel surface state feature maps. Specifically, the surface state images of each photovoltaic panel are respectively input into a surface state feature extractor based on a dilated convolutional neural network model. By introducing a dilated convolution operation, the surface state feature extractor can expand the receptive field while maintaining the resolution of the feature map and extract richer contextual information. In this way, the surface state features of the photovoltaic panel, such as texture and color, can be effectively extracted from the surface state image of the photovoltaic panel, and the abnormal conditions of the photovoltaic panel surface can be accurately captured, providing effective data support for subsequent photovoltaic panel abnormality detection.

[0045] In the above-mentioned photovoltaic array operation status monitoring system 100 based on drones, the feature enhancement module 130 is used to perform spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors. Figure 3 FIG is a block diagram of a feature enhancement module in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention. Figure 3 As shown, the feature enhancement module 130 includes: a spatial mask unit 131, which is used to pass each photovoltaic panel surface state feature map in the set of photovoltaic panel surface state feature maps through a content discrimination enhancement module based on a spatial mask to obtain a set of enhanced photovoltaic panel surface state feature maps; a multi-scale feature focusing unit 132, which is used to pass each enhanced photovoltaic panel surface state feature map in the set of enhanced photovoltaic panel surface state feature maps through a multi-scale feature focusing module to obtain a set of enhanced photovoltaic panel surface state focusing feature vectors.

[0046] Specifically, the spatial mask unit 131 is used to pass each photovoltaic panel surface state feature map in the set of photovoltaic panel surface state feature maps through a content discrimination enhancement module based on a spatial mask to obtain a set of enhanced photovoltaic panel surface state feature maps. It should be understood that, considering that some photovoltaic panels in the photovoltaic array may be affected by factors such as installation position, orientation, and lighting conditions, resulting in their key features in the photovoltaic panel surface state image not being obvious or being interfered with, there may be a certain error rate in the abnormality detection directly based on the photovoltaic panel surface state feature map. Therefore, in order to further improve the accuracy of photovoltaic panel abnormality detection, in the technical solution of the present invention, a content discrimination enhancement module based on a spatial mask is introduced to perform feature enhancement processing on each photovoltaic panel surface state feature map, so as to highlight the key feature areas in the photovoltaic panel surface state feature map and suppress the interference of background noise and irrelevant information. Specifically, the content discrimination enhancement module can generate a corresponding spatial mask based on the feature distribution information of the photovoltaic panel surface state feature map, and perform corresponding element-wise multiplication operations on it with the photovoltaic panel surface state feature map, thereby selectively enhancing the feature performance of specific areas in the photovoltaic panel surface state feature map to improve the discrimination between features of different regions, so that the model can focus more on the identification and analysis of key feature areas.

[0047] Figure 4 FIG is a block diagram of a spatial mask unit in a photovoltaic array operating status monitoring system based on a drone according to an embodiment of the present invention. Figure 4 As shown, the spatial mask unit 131 includes: a spatial significance scoring subunit 1311, which is used to calculate the photovoltaic panel surface state feature significance score map of the photovoltaic panel surface state feature map; a masking subunit 1312, which is used to mask the photovoltaic panel surface state feature significance score map based on a predetermined threshold to obtain a spatial significance attention mask map; a pixel-by-pixel weighting subunit 1313, which is used to use the spatial significance attention mask map as a weight to calculate the position point multiplication of the photovoltaic panel surface state feature map and the spatial significance attention mask map to obtain the enhanced photovoltaic panel surface state feature map.

[0048] Specifically, the spatial significance scoring subunit 1311 is used to: calculate the inverse of the exponential function value with a natural constant as the base of each eigenvalue of the photovoltaic panel surface state characteristic map to obtain an indexed photovoltaic panel surface state characteristic map; calculate the cumulative sum of each eigenvalue of the indexed photovoltaic panel surface state characteristic map to obtain the global feature intensity of the photovoltaic panel surface state characteristic map; calculate the addition result of the global feature intensity and one, and divide the eigenvalue of each position of the indexed photovoltaic panel surface state characteristic map by the addition result to obtain the photovoltaic panel surface state feature significance score map.

[0049] More specifically, the spatial mask unit 131 is configured to process the photovoltaic panel surface state characteristic map using the following spatial mask formula to obtain the enhanced photovoltaic panel surface state characteristic map; wherein the spatial mask formula is:

[0050]

[0051] F i,j,k =M(i,j,k)⊙A(i,j,k)

[0052] Where A(i,j,k) is the eigenvalue of the position (i,j,k) in the photovoltaic panel surface state feature map, mask(·) represents masking, ε represents the predetermined threshold, M(i,j,k) is the eigenvalue of the position (i,j,k) in the spatial saliency attention mask map, ⊙ represents the point multiplication by position, and F i,j,k is the characteristic value of the position (i, j, k) in the surface state characteristic diagram of the enhanced photovoltaic panel.

[0053] Specifically, the multi-scale feature focusing unit 132 is used to pass each of the enhanced photovoltaic panel surface state feature maps in the set of enhanced photovoltaic panel surface state feature maps through a multi-scale feature focusing module to obtain a set of enhanced photovoltaic panel surface state focused feature vectors. It should be understood that, considering the diversity of photovoltaic panel surface state features, feature information at different scales has important reference value for photovoltaic panel anomaly detection. Therefore, in the technical solution of the present invention, a multi-scale feature focusing module is introduced to perform multi-scale feature extraction on each of the enhanced photovoltaic panel surface state feature maps, so as to make full use of the photovoltaic panel surface state feature information at different scales and improve the accuracy of anomaly detection. Specifically, the multi-scale feature focusing module extracts and fuses global semantic features and local semantic features of the enhanced photovoltaic panel surface state feature map to capture the overall state information of the photovoltaic panel, such as color distribution and texture consistency, as well as local detail information such as local damage, stains, and foreign matter, thereby improving the ability to understand the surface state of the photovoltaic panel and providing a more comprehensive and accurate feature representation for photovoltaic panel anomaly detection.

[0054] Figure 5 FIG. 1 is a block diagram of a multi-scale feature focusing unit in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention. Figure 5As shown, the multi-scale feature focusing unit 132 includes: a multi-scale semantic feature extraction sub-unit 1321, which is used to respectively extract the global semantic features and local semantic features of the enhanced photovoltaic panel surface state feature map to obtain the enhanced photovoltaic panel surface state global semantic feature vector and the enhanced photovoltaic panel surface state local semantic feature vector; a multi-scale semantic feature fusion sub-unit 1322, which is used to fuse the enhanced photovoltaic panel surface state global semantic feature vector and the enhanced photovoltaic panel surface state local semantic feature vector to obtain the enhanced photovoltaic panel surface state focusing feature vector.

[0055] Specifically, the multi-scale semantic feature extraction subunit 1321 includes: an upsampling secondary subunit, used to upsample the surface state feature map of the enhanced photovoltaic panel to obtain an upsampled enhanced photovoltaic panel surface state feature map; a global semantic feature extraction secondary subunit, used to pass the upsampled enhanced photovoltaic panel surface state feature map through a global semantic feature extraction module to obtain a global semantic feature vector of the surface state of the enhanced photovoltaic panel, wherein the global semantic feature extraction module includes a global average pooling layer, a first point convolution layer, a first batch normalization layer and a first activation layer; a local semantic feature extraction secondary subunit, used to pass the upsampled enhanced photovoltaic panel surface state feature map through a local semantic feature extraction module to obtain a local semantic feature vector of the surface state of the enhanced photovoltaic panel, wherein the local semantic feature extraction module includes a second point convolution layer, a second batch normalization layer and a second activation layer.

[0056] In the aforementioned drone-based photovoltaic array operating status monitoring system 100, the semantic association measurement module 140 is configured to calculate the semantic association of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall features of the set, thereby obtaining a set of semantic associations. It should be understood that, given the large number of photovoltaic panels in a photovoltaic array and the potential differences in the operating status of each photovoltaic panel, in order to quantitatively assess whether the operating status of each photovoltaic panel is abnormal, the technical solution of the present invention desirably uses the overall operating status features of the photovoltaic array as a benchmark, and determines whether an abnormality exists by calculating the semantic association between the operating status features of each photovoltaic panel and the overall features.

[0057] Figure 6 FIG is a block diagram of a semantic association measurement module in a photovoltaic array operation status monitoring system based on a drone according to an embodiment of the present invention. Figure 6As shown, the semantic association measurement module 140 includes: a feature distribution sequence cluster center calculation unit 141, which is used to calculate the global mean vector of the set of enhanced photovoltaic panel surface state focused feature vectors as the feature distribution sequence cluster center; a semantic association degree calculation unit 142, which is used to respectively calculate the semantic association degree between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center to obtain the set of semantic association degrees.

[0058] Specifically, the feature distribution sequence cluster center calculation unit 141 is configured to calculate the global mean vector of the set of focused feature vectors of the enhanced photovoltaic panel surface state as the feature distribution sequence cluster center. In other words, by calculating the global mean vector of the set of focused feature vectors of the enhanced photovoltaic panel surface state, noise and redundant information in the feature vector set are reduced, and more representative overall operating state characteristics of the photovoltaic array, namely, the feature distribution sequence cluster center, are extracted. This provides a reference benchmark for subsequent abnormal state assessment of each photovoltaic panel, thereby enabling abnormality detection of the photovoltaic panel.

[0059] Specifically, the semantic relevance calculation unit 142 is configured to calculate the semantic relevance between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center to obtain the set of semantic relevance. That is, by calculating the semantic relevance, the similarity between each photovoltaic panel and the operating state of the entire photovoltaic array is quantitatively evaluated, revealing the degree of deviation from the overall average characteristics of the photovoltaic array, thereby providing a basis for determining whether an anomaly exists. Specifically, if the semantic relevance is high, it indicates that the operating state of the photovoltaic panel is relatively close to the average state of the entire photovoltaic array and is in a normal operating state; conversely, if the semantic relevance is low, it may mean that the photovoltaic panel is in some abnormal state, requiring further fault location and analysis.

[0060] More specifically, the semantic association calculation unit 142 is used to: respectively calculate the matrix product between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and each predetermined weight matrix to obtain a set of enhanced photovoltaic panel surface state semantic coding feature vectors; add a bias vector to each enhanced photovoltaic panel surface state semantic coding feature vector in the set of enhanced photovoltaic panel surface state semantic coding feature vectors and then pass the resultant vector through a tanh activation function to obtain a set of activated enhanced photovoltaic panel surface state semantic coding feature vectors; respectively calculate the vector product between each activated enhanced photovoltaic panel surface state semantic coding feature vector in the set of activated enhanced photovoltaic panel surface state semantic coding feature vectors and the transposed vector of the feature distribution sequence clustering center to obtain the set of semantic associations.

[0061] That is, the semantic association between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center is calculated using the following semantic association calculation formula to obtain the set of semantic associations, wherein the semantic association calculation formula is:

[0062]

[0063] Among them, v i is the i-th enhanced photovoltaic panel surface state focusing feature vector in the set of enhanced photovoltaic panel surface state focusing feature vectors, v b is the cluster center of the feature distribution sequence, W i is the predetermined weight matrix, B i is the bias vector, tanh is the activation function, r i is the i-th semantic relevance.

[0064] In particular, in the technical solution of the present invention, the feature distribution sequence cluster center is the global mean vector of the set of enhanced photovoltaic panel surface state focused feature vectors, that is, the feature distribution sequence cluster center is used to represent the cluster center of the photovoltaic panel surface state features of the entire sample domain of the surface state images of all photovoltaic panels. And each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors respectively represents the image semantic features of the surface state images of each photovoltaic panel. Here, in the process of calculating the global mean vector of the set of enhanced photovoltaic panel surface state focused feature vectors to obtain the feature distribution sequence cluster center, local overflow features will be caused due to significant differences between samples, thereby affecting the mapping consistency of the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center to the common semantic association representation space across sample domains, and affecting the calculation accuracy of the semantic association between the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center.

[0065] Based on this, in a preferred example of the present invention, respectively calculating the semantic association between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center to obtain a set of semantic associations includes the following steps: cascading the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center into a feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector; calculating the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint mean representation matrix and the feature distribution sequence cluster center- Strengthening the photovoltaic panel surface state joint variance representation matrix, wherein the value of each position of the characteristic distribution sequence cluster center-strengthened photovoltaic panel surface state joint mean representation matrix is the mean of a pair of eigenvalues of the two positions of the characteristic distribution sequence cluster center-strengthened photovoltaic panel surface state joint representation vector corresponding to the position coordinates respectively, and the value of each position of the characteristic distribution sequence cluster center-strengthened photovoltaic panel surface state joint variance representation matrix is the variance of a pair of eigenvalues of the two positions of the characteristic distribution sequence cluster center-strengthened photovoltaic panel surface state joint representation vector corresponding to the position coordinates respectively; the characteristic distribution sequence cluster center-strengthened photovoltaic panel surface state joint representation matrix is strengthened. The transposed vector of the vector is matrix-multiplied with the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint mean representation matrix to obtain a first characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector, and the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint variance representation matrix is matrix-multiplied with the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector to obtain a second characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector, wherein the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector is a column vector; calculate the first characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector. The third characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector is obtained by adding the point sum of the joint intermediate vector of the photovoltaic panel surface state and the transposed vector of the second characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector; the matrix product of the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint mean representation matrix and the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation matrix is calculated, and the transposed vector of the characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector is matrix-multiplied by the matrix product to obtain a fourth characteristic distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector;Calculating the point sum of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector and the fourth feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint intermediate vector to obtain a corrected feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector; splitting the corrected feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector into a corrected enhanced photovoltaic panel surface state focused feature vector and a corrected feature distribution sequence cluster center based on the cascade pattern of the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center; and calculating the semantic association between the corrected enhanced photovoltaic panel surface state focused feature vector and the corrected feature distribution sequence cluster center.

[0066] Based on this, the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector is used for the joint semantic representation of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector, and the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint mean representation matrix and the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint variance representation matrix are used as the retrieval and response distribution enhancement of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector, and the feature distribution open based on group aggregation of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector is constructed. A reference-free distribution retrieval response framework under the domain is proposed to avoid the distribution response redundancy caused by the local overflow characteristics of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector through response superposition, so as to realize the faithfulness constraint of the self-aggregation statistical correlation of the response of the feature distribution sequence cluster center-enhanced photovoltaic panel surface state joint representation vector to the target common semantic semantic association representation space, improve the mapping consistency of the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center to the common semantic semantic association representation space under cross-sample domains, and improve the calculation accuracy of the semantic association between the enhanced photovoltaic panel surface state focused feature vector and the feature distribution sequence cluster center.

[0067] In the aforementioned drone-based photovoltaic array operation status monitoring system 100, the abnormal photovoltaic panel identification module 150 is configured to identify photovoltaic panels within the set of semantic relevances that have a semantic relevance less than or equal to a preset threshold as abnormal photovoltaic panels. In other words, the determination of abnormal photovoltaic panel status is achieved by setting a threshold. When the semantic relevance of a photovoltaic panel is less than or equal to the preset threshold, the panel is deemed to be abnormal. In this way, problematic photovoltaic panels can be quickly identified, providing effective guidance for subsequent troubleshooting and repair work.

[0068] In summary, a drone-based photovoltaic array operating status monitoring system according to an embodiment of the present invention has been described. It uses deep learning-based machine vision technology to perform feature analysis on the surface images of all photovoltaic panels in the photovoltaic array, extracting the surface state characteristics of each photovoltaic panel. Based on the semantic correlation between the surface state characteristics of each photovoltaic panel and the overall average characteristics, it determines whether each photovoltaic panel is abnormal. This allows for accurate monitoring of the photovoltaic array's operating status, enabling the timely detection and treatment of abnormal photovoltaic panels, thereby improving the operating efficiency and stability of the entire photovoltaic power generation system.

[0069] Figure 7 FIG is a flow chart of a method for real-time monitoring of photovoltaic array operating status based on a drone according to an embodiment of the present invention. Figure 7 As shown, the real-time monitoring method of the photovoltaic array operation status based on the drone according to the embodiment of the present invention includes the following steps: S1, collecting the surface state images of all photovoltaic panels in the photovoltaic array by using a camera carried by the drone to obtain a set of photovoltaic panel surface state images; S2, performing image feature extraction on each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps; S3, performing spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors; S4, calculating the semantic relevance of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall feature of the set to obtain a set of semantic relevance; S5, setting the photovoltaic panels corresponding to the semantic relevance less than or equal to a preset threshold in the set of semantic relevance as abnormal photovoltaic panels.

[0070] Here, those skilled in the art will understand that the specific operations of each step in the above-mentioned method for real-time monitoring of photovoltaic array operation status based on drones have been described in the above reference. Figures 1 to 6 The description of the UAV-based photovoltaic array operation status monitoring system has been introduced in detail, and therefore, its repeated description will be omitted.

[0071] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0072] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical subunits, that is, they may be located in one place, or they may be distributed on multiple network subunits. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0073] It will be understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present invention, and the present invention is not limited thereto. Those skilled in the art will appreciate that various modifications and improvements can be made without departing from the spirit and substance of the present invention, and such modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A photovoltaic array operation status monitoring system based on drones, characterized in that: include: A photovoltaic panel surface image acquisition module is used to acquire surface state images of all photovoltaic panels in the photovoltaic array through a camera carried by the drone to obtain a set of photovoltaic panel surface state images; a photovoltaic panel surface state feature extraction module, configured to extract image features from each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps; A feature enhancement module, configured to perform spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors; a semantic association measurement module, configured to respectively calculate the semantic association of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall features of the set to obtain a set of semantic associations; The abnormal photovoltaic panel identification module is used to set the photovoltaic panels corresponding to the semantic relevance values less than or equal to a preset threshold value in the set of semantic relevance values as abnormal photovoltaic panels.

2. The photovoltaic array operation status monitoring system based on drone according to claim 1 is characterized in that: The photovoltaic panel surface state feature extraction module is used to: Each photovoltaic panel surface state image in the set of photovoltaic panel surface state images is respectively passed through a surface state feature extractor based on a hole convolutional neural network model to obtain a set of photovoltaic panel surface state feature maps.

3. The photovoltaic array operation status monitoring system based on drone according to claim 2 is characterized in that: The feature enhancement module includes: a spatial mask unit, configured to pass each photovoltaic panel surface state feature map in the set of photovoltaic panel surface state feature maps through a content discrimination enhancement module based on spatial masking to obtain a set of enhanced photovoltaic panel surface state feature maps; The multi-scale feature focusing unit is used to pass each enhanced photovoltaic panel surface state feature map in the set of enhanced photovoltaic panel surface state feature maps through a multi-scale feature focusing module to obtain a set of enhanced photovoltaic panel surface state focusing feature vectors.

4. The photovoltaic array operation status monitoring system based on drone according to claim 3 is characterized in that: The spatial mask unit includes: a spatial significance scoring subunit, configured to calculate a photovoltaic panel surface state feature significance score map of the photovoltaic panel surface state feature map; a masking subunit, configured to perform masking processing on the photovoltaic panel surface state feature saliency score map based on a predetermined threshold to obtain a spatial saliency attention mask map; A pixel-by-pixel weighted subunit is used to calculate the photovoltaic panel surface state feature map and the spatial saliency attention mask map by multiplying the position points by the spatial saliency attention mask map to obtain the enhanced photovoltaic panel surface state feature map using the spatial saliency attention mask map as a weight.

5. The photovoltaic array operation status monitoring system based on drone according to claim 4 is characterized in that: The spatial significance scoring subunit is used to: Calculating the reciprocal of the exponential function value with the natural constant as the base of each characteristic value of the photovoltaic panel surface state characteristic map to obtain an indexed photovoltaic panel surface state characteristic map; Calculating the cumulative sum of each characteristic value of the indexed photovoltaic panel surface state characteristic map to obtain a global characteristic intensity of the photovoltaic panel surface state characteristic map; The sum of the global feature intensity and one is calculated, and the feature values of each position of the indexed photovoltaic panel surface state feature map are divided by the sum to obtain the photovoltaic panel surface state feature significance score map.

6. The photovoltaic array operation status monitoring system based on drone according to claim 5 is characterized in that: The multi-scale feature focusing unit includes: a multi-scale semantic feature extraction subunit, configured to respectively extract global semantic features and local semantic features of the enhanced photovoltaic panel surface state feature map to obtain an enhanced photovoltaic panel surface state global semantic feature vector and an enhanced photovoltaic panel surface state local semantic feature vector; The multi-scale semantic feature fusion subunit is used to fuse the global semantic feature vector of the surface state of the enhanced photovoltaic panel and the local semantic feature vector of the surface state of the enhanced photovoltaic panel to obtain the focused feature vector of the surface state of the enhanced photovoltaic panel.

7. The photovoltaic array operation status monitoring system based on a drone according to claim 6 is characterized in that: The multi-scale semantic feature extraction subunit includes: an upsampling secondary subunit, configured to upsample the enhanced photovoltaic panel surface state characteristic map to obtain an upsampled enhanced photovoltaic panel surface state characteristic map; A global semantic feature extraction secondary subunit is used to pass the upsampled enhanced photovoltaic panel surface state feature map through a global semantic feature extraction module to obtain a global semantic feature vector of the enhanced photovoltaic panel surface state, wherein the global semantic feature extraction module includes a global average pooling layer, a first point convolution layer, a first batch normalization layer and a first activation layer; The local semantic feature extraction secondary sub-unit is used to pass the upsampled enhanced photovoltaic panel surface state feature map through a local semantic feature extraction module to obtain the enhanced photovoltaic panel surface state local semantic feature vector, wherein the local semantic feature extraction module includes a second point convolution layer, a second batch normalization layer and a second activation layer.

8. The photovoltaic array operation status monitoring system based on a drone according to claim 7 is characterized in that: The semantic association measurement module includes: a characteristic distribution sequence cluster center calculation unit, configured to calculate a global mean vector of a set of focused characteristic vectors of the surface state of the enhanced photovoltaic panel as a characteristic distribution sequence cluster center; The semantic association calculation unit is used to respectively calculate the semantic association between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and the feature distribution sequence cluster center to obtain the set of semantic associations.

9. The photovoltaic array operation status monitoring system based on a drone according to claim 8, characterized in that: The semantic relevance calculation unit is used to: Calculating matrix products between each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors and each predetermined weight matrix to obtain a set of enhanced photovoltaic panel surface state semantic coding feature vectors; Adding a bias vector to each enhanced photovoltaic panel surface state semantic coding feature vector in the set of enhanced photovoltaic panel surface state semantic coding feature vectors and performing a tanh activation function to obtain a set of activated enhanced photovoltaic panel surface state semantic coding feature vectors; The vector products of each activated enhanced photovoltaic panel surface state semantic coding feature vector in the set of activated enhanced photovoltaic panel surface state semantic coding feature vectors and the transposed vector of the feature distribution sequence cluster center are calculated respectively to obtain the set of semantic associations.

10. A real-time monitoring method for photovoltaic array operating status based on drones, characterized in that: include: The surface state images of all photovoltaic panels in the photovoltaic array are collected by a camera carried by the drone to obtain a set of photovoltaic panel surface state images; performing image feature extraction on each photovoltaic panel surface state image in the set of photovoltaic panel surface state images to obtain a set of photovoltaic panel surface state feature maps; Performing spatial dimension feature enhancement processing on the set of photovoltaic panel surface state feature maps to obtain a set of enhanced photovoltaic panel surface state focused feature vectors; respectively calculating the semantic relevance of each enhanced photovoltaic panel surface state focused feature vector in the set of enhanced photovoltaic panel surface state focused feature vectors relative to the overall features of the set to obtain a set of semantic relevance; The photovoltaic panels corresponding to the semantic relevances in the set of semantic relevances that are less than or equal to a preset threshold are set as abnormal photovoltaic panels.

Images