Unmanned aerial vehicle inspection method and system applied to large photovoltaic power station

Abstract

The application provides a kind of unmanned aerial vehicle inspection method and system applied to large photovoltaic power station, by obtaining sample unmanned aerial vehicle inspection data carrying state annotation data, and using machine learning network to carry out cyclic network parameter optimization, generate optimized target photovoltaic power station anomaly diagnosis network, can effectively improve the accuracy and efficiency of unmanned aerial vehicle inspection. Specifically, by image semantic representation and optimization to sample unmanned aerial vehicle inspection image, generate more accurate target image semantic features, and then use image semantic restoration unit to generate power station abnormal state prediction result. By continuously iterating the network learning weight information, the finally generated target photovoltaic power station anomaly diagnosis network has higher diagnosis precision. In practical application, it can quickly and accurately identify the abnormal state in large photovoltaic power station, so as to facilitate the operation and maintenance management of large photovoltaic power station, greatly improve the operation efficiency and safety of photovoltaic power station.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic power station inspection, and in particular to a drone inspection method and system applied to large photovoltaic power stations. Background Art

[0002] With the transformation of energy structures and the rapid development of renewable energy, large-scale photovoltaic power plants, as a key component of clean energy, are expanding in scale, posing unprecedented challenges to their operation and maintenance. Traditional inspections of photovoltaic power plants rely primarily on manual labor. However, due to the vast footprint and complex equipment layout of photovoltaic power plants, manual inspections are not only time-consuming and labor-intensive, but also difficult to ensure comprehensiveness and timeliness. Therefore, how to efficiently and accurately complete the inspection of photovoltaic power plants has become a pressing issue.

[0003] Among related technologies, drone technology has been widely used in photovoltaic power plant inspections due to its efficiency and flexibility. Drones can quickly cover various areas of a photovoltaic power plant and, equipped with high-resolution cameras and other sensors, capture real-time images of the plant's operating status. However, faced with the massive amount of drone inspection image data, how to quickly and accurately identify abnormal conditions remains a key challenge in drone inspection technology. Summary of the Invention

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a drone inspection method for a large photovoltaic power station, the method comprising:

[0005] Acquire multiple sample drone inspection data carrying status annotation data of a large photovoltaic power station, each of the sample drone inspection data including a sample drone inspection image, and the status annotation data of each sample drone inspection data is a sample power station abnormal status label corresponding to the sample drone inspection image in the sample drone inspection data;

[0006] Based on multiple sample drone inspection data, the machine learning network is optimized through recurrent network parameter optimization to generate an optimized target photovoltaic power station anomaly diagnosis network.

[0007] Obtain drone inspection images of a large photovoltaic power station to be diagnosed, and use the optimized target photovoltaic power station anomaly diagnosis network to generate power station abnormality status results corresponding to the drone inspection images to be diagnosed;

[0008] The step of optimizing the cyclic network parameters includes:

[0009] For each sample unmanned aerial vehicle inspection image, the image semantic extraction unit of the initialized first image classification network is used to perform image semantic representation on the sample unmanned aerial vehicle inspection image, to generate first image semantic features of each inspection unit block in the sample unmanned aerial vehicle inspection image, based on first network learning weight information, to determine each first inspection unit block to be optimized in each inspection unit block of the sample unmanned aerial vehicle inspection image, and based on second network learning weight information, to optimize the first image semantic features of each first inspection unit block, to generate second image semantic features of each first inspection unit block after optimization.

[0010] For each sample unmanned aerial vehicle inspection image, the image semantic extraction unit of the initialized first image classification network is used to perform image semantic representation on the sample unmanned aerial vehicle inspection image, to generate first image semantic features of each inspection unit block in the sample unmanned aerial vehicle inspection image, based on first network learning weight information, to determine each first inspection unit block to be optimized in each inspection unit block of the sample unmanned aerial vehicle inspection image, and based on second network learning weight information, to optimize the first image semantic features of each first inspection unit block, to generate second image semantic features of each first inspection unit block after optimization.

[0011] Based on the power station abnormal state prediction results corresponding to each sample unmanned aerial vehicle inspection image and the state label data, first network learning errors are determined, and based on the first network learning errors, the first network learning weight information and the second network learning weight information are optimized, to generate first network learning weight information and second network learning weight information corresponding to next round of network parameter optimization.

[0012] In still another aspect, the embodiments of the present application also provide an unmanned aerial vehicle inspection system applied to a large photovoltaic power station, which comprises a processor and a machine readable storage medium, the machine readable storage medium is connected with the processor, the machine readable storage medium is used for storing programs, instructions or codes, and the processor is used for executing the programs, instructions or codes in the machine readable storage medium, to realize the above-mentioned method.

[0013] Based on the above aspects, the embodiment of the present application obtains sample drone inspection data carrying status annotation data, and uses a machine learning network to perform cyclic network parameter optimization to generate an optimized target photovoltaic power station abnormality diagnosis network, which can effectively improve the accuracy and efficiency of drone inspections. Specifically, by performing image semantic representation and optimization on the sample drone inspection images, more accurate target image semantic features are generated, and then the image semantic restoration unit is used to generate the abnormal state prediction results of the power station. By continuously iteratively optimizing the network learning weight information, the target photovoltaic power station abnormality diagnosis network finally generated has higher diagnostic accuracy. In practical applications, it can quickly and accurately identify abnormal states in large photovoltaic power stations, so as to facilitate the operation and maintenance management of large photovoltaic power stations, greatly improving the operating efficiency and safety of photovoltaic power stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The figure is a schematic diagram of the execution flow of the drone inspection method applied to large photovoltaic power stations provided by an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the hardware architecture of a drone inspection system for large photovoltaic power plants provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This is a flow chart of a drone inspection method for a large photovoltaic power station provided by an embodiment of the present invention. The drone inspection method for a large photovoltaic power station is introduced in detail below.

[0017] Step S110: Acquire multiple sample drone inspection data carrying status annotation data of a large photovoltaic power station, each of the sample drone inspection data includes a sample drone inspection image, and the status annotation data of each sample drone inspection data is the sample power station abnormal status label corresponding to the sample drone inspection image in the sample drone inspection data.

[0018] In this embodiment, the server establishes a connection with the database or data storage system of a large photovoltaic power station. In large photovoltaic power stations, routine inspections are performed using drones. These drones are equipped with high-definition cameras and other equipment. During the inspection process, they capture images of various areas of the photovoltaic power station according to predetermined routes and rules, generating a large number of drone inspection images. Furthermore, during the power station's operation and maintenance, professional technicians analyze these drone inspection images to determine whether the corresponding photovoltaic power station area in the image has any abnormal conditions, such as whether the photovoltaic panels are damaged, whether the surface is obscured by stains, and whether the wiring connections are normal. Each drone inspection image is annotated with a corresponding abnormal condition label, which serves as the status annotation data.

[0019] Suppose a large photovoltaic power plant consists of multiple areas, such as Area A, Area B, and Area C, each containing numerous photovoltaic arrays. The server retrieves sample drone inspection data from the data store. For example, a sample drone inspection image from Area A shows a portion of a row of photovoltaic panels. After analysis and annotation by technicians, the corresponding status annotation for this sample drone inspection image is "a small amount of dirt obstructing the photovoltaic panel surface." Another sample drone inspection image from Area B shows the connection wiring of a photovoltaic array, annotated with "the wiring is normal." The server continuously retrieves multiple such sample drone inspection data sets, each containing a sample drone inspection image and a corresponding sample power plant abnormality label. This data serves as the foundation for subsequent machine learning network training and optimization.

[0020] Step S120 , based on a plurality of sample drone inspection data, a cyclic network parameter optimization is performed on the machine learning network to generate an optimized target photovoltaic power station abnormality diagnosis network.

[0021] In this embodiment, a machine learning framework is running on the server, and a first image classification network has been initialized within this machine learning framework for image classification. This first image classification network includes components such as an image semantic extraction unit and an image semantic restoration unit. The server then begins to perform recurrent network parameter optimization on this machine learning network using multiple sample drone inspection data samples.

[0022] Step S130 , obtaining a drone inspection image to be diagnosed of a large photovoltaic power station, and using the optimized target photovoltaic power station abnormality diagnosis network to generate a power station abnormality status result corresponding to the drone inspection image to be diagnosed.

[0023] During the ongoing inspection of the PV power plant, a drone captures new drone inspection images for diagnosis. The server retrieves these images. Assume that these images show a new area in the PV power plant, or a new situation in a previous area.

[0024] The server loads the undiagnosed drone inspection image into the optimized target PV plant anomaly diagnosis network. This target PV plant anomaly diagnosis network processes the undiagnosed drone inspection image based on parameters previously optimized using multiple sample drone inspection data.

[0025] First, the drone inspection image to be diagnosed passes through the image semantic extraction unit. Following the same processing flow as the previous sample drone inspection image, image semantic features are generated for each inspection unit block. The network then uses the learned weights to determine the inspection unit block to be optimized and perform other optimization operations. Next, the target image semantic features are generated based on the processed image semantic features. The image semantic restoration unit then performs image semantic restoration, ultimately obtaining the power plant abnormality result corresponding to the drone inspection image to be diagnosed, such as a loose line connection. This result can provide timely and accurate diagnostic information to the PV plant's operation and maintenance personnel, allowing them to carry out appropriate maintenance and repair work.

[0026] The step of optimizing the cyclic network parameters includes:

[0027] Step S121: For each sample drone inspection image, use the image semantic extraction unit of the initialized first image classification network to perform image semantic representation on the sample drone inspection image, generate the first image semantic features of each inspection unit block in the sample drone inspection image, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample drone inspection image based on the first network learning weight information, and optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information to generate the optimized second image semantic features of each of the first inspection unit blocks.

[0028] For example, for each sample drone inspection image acquired, the server loads it into the image semantic extraction unit of the first image classification network. Assume that this image semantic extraction unit consists of multiple sequentially connected image semantic encoding nodes. For example, a sample drone inspection image showing a photovoltaic panel array in a certain area of ​​a photovoltaic power plant is segmented into multiple inspection unit blocks. For example, based on the arrangement of the photovoltaic panels, every few photovoltaic panels are divided into one inspection unit block.

[0029] The server processes each inspection unit block through the first image semantic encoding node in the image semantic extraction unit to generate the first image semantic features of each inspection unit block. These first image semantic features contain information related to the color, texture, shape, and other image semantics of the inspection unit block.

[0030] The first network learning weight information includes sub-weight information for each inspection unit block in the sample drone inspection image. This sub-weight information is used to determine whether to optimize the first image semantic features of the inspection unit block. For example, for an inspection unit block located at the edge of the image, since it may be significantly affected by uneven lighting and image acquisition angle, the corresponding sub-weight information may be higher, indicating that this inspection unit block is more likely to be determined as the first inspection unit block to be optimized. Based on this sub-weight information, the server determines which inspection unit blocks in the image are the first inspection unit blocks to be optimized.

[0031] After determining the first inspection unit blocks, the server optimizes the first image semantic features of these first inspection unit blocks based on the second network learning weight information. Assuming the second network learning weight information is a set of adjustment coefficients, the server generates optimized second image semantic features by performing specific mathematical operations (such as multiplication and addition) on the first image semantic features of the first inspection unit blocks and these adjustment coefficients. For example, if the first image semantic features of a first inspection unit block are represented as a vector [1, 2, 3] and the corresponding second network learning weight information is [0.8, 1.2, 0.9], then the optimized second image semantic features may be [1 * 0.8, 2 * 1.2, 3 * 0.9] = [0.8, 2.4, 2.7].

[0032] Step S122: For each sample drone inspection image, image semantic restoration is performed using the image semantic restoration unit of the first image classification network based on the target image semantic features of the sample drone inspection image to generate a power plant abnormal state prediction result corresponding to the sample drone inspection image, wherein the target image semantic features of the sample drone inspection image are generated based on the second image semantic features of each of the first inspection unit blocks in the sample drone inspection image and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks.

[0033] For a previously processed sample drone inspection image, the server generates a target image semantic feature for the sample drone inspection image based on the obtained second image semantic features of each first inspection unit block and the first image semantic features of the other inspection unit blocks. For example, if the image has 10 inspection unit blocks, 3 of which are first inspection unit blocks, the server will combine the second image semantic features of these 3 first inspection unit blocks with the first image semantic features of the other 7 inspection unit blocks in a certain order to form the target image semantic feature.

[0034] The server then loads the semantic features of the target image into the image semantic restoration unit of the first image classification network. This image semantic restoration unit performs image semantic restoration operations on the semantic features of the target image based on the pre-trained model structure and parameters. For example, if the semantic features of the target image are a high-dimensional vector, the image semantic restoration unit may convert this vector into a prediction result representing the abnormal state of the power station through a series of linear and nonlinear transformations. Assuming that this prediction result is a classification result, it may be one of the following: "no abnormality", "broken photovoltaic panel", "line fault", etc. This is the prediction result of the abnormal state of the power station corresponding to the sample drone inspection image.

[0035] Step S123: Determine the first network learning error based on the abnormal state prediction results of the power station and the state annotation data corresponding to each of the sample drone inspection images; optimize the first network learning weight information and the second network learning weight information based on the first network learning error; and generate the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization.

[0036] For each sample drone inspection image, the server generates a prediction of a power plant abnormality and also stores the corresponding state annotation data. For example, for one sample drone inspection image, the prediction result is "no abnormality," while the state annotation data is "a small amount of dirt obstructing the photovoltaic panel surface," indicating a discrepancy between the prediction and the actual annotation.

[0037] The server uses a specific error calculation method (such as cross-entropy error) to determine the first network learning error based on the prediction results and state annotation data of all sample drone inspection images. This error reflects the current prediction accuracy of the first image classification network.

[0038] Then, based on the first network learning error, the server optimizes the first network learning weight information and the second network learning weight information. For example, if the sub-weight information of a certain inspection unit block causes a large error in the current network, the server may adjust the value of this sub-weight information so that it can more accurately determine the first inspection unit block to be optimized in subsequent training. At the same time, the second network learning weight information will also be adjusted according to the error situation to make the optimization of the first image semantic features of the first inspection unit block more reasonable. Through such an optimization process, the server generates the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization, so as to carry out the next round of network optimization.

[0039] Based on the above steps, the embodiment of the present application obtains sample drone inspection data carrying status annotation data, and uses a machine learning network to perform cyclic network parameter optimization to generate an optimized target photovoltaic power station abnormality diagnosis network, which can effectively improve the accuracy and efficiency of drone inspections. Specifically, by performing image semantic representation and optimization on the sample drone inspection images, more accurate target image semantic features are generated, and then the image semantic restoration unit is used to generate the abnormal state prediction results of the power station. By continuously iteratively optimizing the network learning weight information, the target photovoltaic power station abnormality diagnosis network finally generated has higher diagnostic accuracy. In practical applications, it can quickly and accurately identify abnormal states in large photovoltaic power stations, so as to facilitate the operation and maintenance management of large photovoltaic power stations, greatly improving the operating efficiency and safety of photovoltaic power stations.

[0040] In a possible implementation, the image semantics extraction unit includes a plurality of sequentially connected image semantics encoding nodes, and the first network learning weight information includes node parameter information corresponding to one or more image semantics encoding nodes.

[0041] For each sample drone inspection image, the steps for generating the power plant abnormal state prediction result corresponding to the sample drone inspection image include:

[0042] Step S1221: load the sample drone inspection image into the image semantic extraction unit, use the image semantic extraction unit to perform image semantic representation on the sample drone inspection image, and generate the first image semantic feature of each inspection unit block in the sample drone inspection image corresponding to each image semantic coding node.

[0043] Step S1222: For each of the one or more image semantic coding nodes, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample UAV inspection image based on the node parameter information corresponding to the image semantic coding node, optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information, generate the second image semantic features of each of the first inspection unit blocks, use the second image semantic features of each of the first inspection unit blocks and the first image semantic features of other inspection unit blocks except the first inspection unit blocks as the target image semantic features of the image semantic coding node, and use the target image semantic features of the image semantic coding node as the loading data of the next image semantic coding node.

[0044] Step S1223, for each image semantic coding node except the one or more image semantic coding nodes, the image semantic coding node will be used to obtain the first image semantic feature of each inspection unit block in the sample UAV inspection image as the target image semantic feature of the image semantic coding node, and the target image semantic feature of the image semantic coding node will be used as the loading data of the next image semantic coding node.

[0045] Step S1224: Based on the target image semantic features of the last image semantic coding node of the image semantic extraction unit, the image semantic restoration unit is used to perform image semantic restoration to generate a power plant abnormal state prediction result corresponding to the sample drone inspection image.

[0046] In this embodiment, the image semantic extraction unit includes a plurality of sequentially connected image semantic coding nodes, and the first network learning weight information covers node parameter information corresponding to one or more image semantic coding nodes.

[0047] First, the server loads the sample drone inspection image into the image semantic extraction unit. For example, a sample drone inspection image is captured from a specific area of ​​a large photovoltaic power plant, which contains multiple photovoltaic panel arrays. After loading into the image semantic extraction unit, each image semantic encoding node in the image semantic extraction unit processes each inspection unit block in the sample drone inspection image (these inspection unit blocks are divided according to specific rules, such as the layout of the photovoltaic panels or geographical location). This generates first image semantic features for each inspection unit block in the sample drone inspection image corresponding to each image semantic encoding node. These first image semantic features are a semantic representation of the inspection unit block at that image semantic encoding node, containing information about the inspection unit block's color, texture, shape, and relationships between them.

[0048] Next, operations are performed on each of the one or more image semantic coding nodes. For one of the image semantic coding nodes, the server determines the first inspection unit blocks to be optimized among the inspection unit blocks in the sample drone inspection image based on the node parameter information corresponding to the image semantic coding node. This node parameter information may be related to factors such as the inspection unit block's position in the image, surrounding environmental characteristics, or its relationship to other inspection unit blocks. For example, at a certain image semantic coding node, based on its node parameter information, if a certain inspection unit block is located in a shadowed area of ​​the image, this inspection unit block may be determined as the first inspection unit block to be optimized, as shadows may significantly affect the image semantic features. After determining the first inspection unit block, the server optimizes the first image semantic features of each first inspection unit block based on the second network learned weight information. Assuming that the second network learned weight information is a set of numerical values ​​related to feature adjustment, the server optimizes the first image semantic features using a specific computational method (such as matrix multiplication) to generate the second image semantic features of each first inspection unit block. Then, the second image semantic features of each first inspection unit block are combined with the first image semantic features of all inspection unit blocks except for the first inspection unit blocks. This combined feature is used as the target image semantic feature of the image semantic coding node, and this target image semantic feature is used as the loading data for the next image semantic coding node. In this way, through the processing of this image semantic coding node, not only are some inspection unit blocks optimized, but also appropriate data is prepared for the next image semantic coding node.

[0049] For each image semantic coding node other than the one or more image semantic coding nodes, the operation is relatively straightforward. The server directly uses the first image semantic feature of each inspection unit block in the sample drone inspection image obtained using the image semantic coding node as the target image semantic feature of the image semantic coding node, and then uses this target image semantic feature as the loading data for the next image semantic coding node. For example, for an intermediate image semantic coding node, it does not perform special optimization operations on the inspection unit block, but simply passes the first image semantic feature generated by itself to the next node intact.

[0050] Finally, after the target image semantic features are obtained by the last image semantic encoding node in the image semantic extraction unit, the server uses the image semantic restoration unit to perform image semantic restoration based on these target image semantic features, thereby generating a prediction result for the power plant abnormality corresponding to the sample drone inspection image. This image semantic restoration unit is constructed based on previously trained data and converts the target image semantic features of the last image semantic encoding node into a prediction result related to the power plant abnormality. For example, if the target image semantic feature is a high-dimensional vector, the image semantic restoration unit may include a series of matrix transformations and nonlinear activation functions. Through these operations, the high-dimensional vector is converted into a classification result representing the power plant abnormality, such as "no abnormality", "stains on the photovoltaic panel surface", or "faulty line connection". This result is the power plant abnormality prediction result corresponding to the sample drone inspection image, providing an important reference for the operation and maintenance management of the photovoltaic power plant.

[0051] In a possible implementation, the first network learning weight information includes sub-weight information of each inspection unit block in the sample drone inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block.

[0052] Step S123 includes:

[0053] Step S1231: Determine a second network learning error based on the fusion weight information of the sub-weight information of each inspection unit block in each of the sample drone inspection images.

[0054] Step S1232: Optimize the first network learning weight information and the second network learning weight information according to the first network learning error and the second network learning error.

[0055] In this embodiment, in the entire process of abnormal diagnosis of photovoltaic power stations, the first network learning weight information includes the sub-weight information of each inspection unit block in the sample drone inspection image, and these sub-weight information plays a key role in the optimization process. When the server processes the sample drone inspection image, the sub-weight information of each inspection unit block determines whether to optimize the first image semantic features of the inspection unit block. For example, for a certain sample drone inspection image, it is taken from a specific area of ​​a large photovoltaic power station, which contains multiple inspection unit blocks divided according to certain rules. Each inspection unit block has corresponding sub-weight information, which is determined based on multiple factors, such as the position of the inspection unit block in the sample drone inspection image, the characteristics of the photovoltaic panels it contains, and its performance in historical data. If a patrol unit block is located in the edge area of ​​the sample drone inspection image, its influence on the semantic features of the entire image may be somewhat special due to edge effects (such as uneven lighting, angle problems during image acquisition, etc.). In this case, the sub-weight information of this patrol unit block may be set to a higher value, indicating that the first image semantic features of this patrol unit block are more likely to need to be optimized; and if a patrol unit block is in the central area of ​​the sample drone inspection image and the corresponding photovoltaic panel state is relatively stable, the sub-weight information of the patrol unit block may be relatively low, indicating that the necessity of optimizing its first image semantic features is relatively small.

[0056] When optimizing the first and second network learning weights based on the first network learning error, the server first determines the second network learning error based on the fused weight information of the sub-weights of each inspection unit block in each sample drone inspection image. For each sample drone inspection image, the server collects the sub-weight information for each inspection unit block. For example, given a batch of sample drone inspection images, the server analyzes the sub-weight information for each inspection unit block in each sample drone inspection image. Suppose a large photovoltaic power plant is divided into multiple regions, and the sample drone inspection images for each region contain several inspection unit blocks. The server then fuses the sub-weight information for the inspection unit blocks in these different regions and sample drone inspection images. This fusion operation may be based on a weighted average algorithm, taking into account factors such as the importance of each inspection unit block within the overall system and its relationship to other inspection unit blocks. Using this fused weight information, the server determines the second network learning error. This second network learning error reflects the comprehensive impact of the sub-weight information of each inspection unit block on the overall network learning effect under the current network parameters.

[0057] Next, the server optimizes the first and second network learning weights based on the first and second network learning errors. The first network learning error is calculated by comparing the power plant abnormality predictions for the sample drone inspection images with the actual state annotation data. For example, for a sample drone inspection image, the server predicts the power plant abnormality as "no abnormality on the photovoltaic panel surface," but the actual state annotation data is "minor stains on the photovoltaic panel surface." This difference between the predicted and actual state is quantified as part of the first network learning error. The server considers the first and second network learning errors to adjust the first and second network learning weights. The server adjusts the sub-weights for each inspection unit block in the first network learning weights based on the error. If the sub-weights for a particular inspection unit block cause significant error in the current network—for example, if the sub-weights for a particular inspection unit block in multiple sample drone inspection images consistently deviate from the actual processing of the semantic features of the first image for that inspection unit block—the server adjusts this sub-weight. The adjustment may be performed by increasing or decreasing the values ​​of the sub-weight information based on the magnitude and direction of the error, according to certain rules (such as a gradient descent algorithm). Similarly, the server adjusts the second network's learning weight information based on the combined learning errors of the first and second networks. For example, if the second network's learning weight information does not optimize the semantic features of the first image of the first inspection unit block in the current network, resulting in a significant deviation in the final power plant abnormality prediction results, the server will adjust the relevant parameters in the second network's learning weight information based on the error to improve the accuracy of the entire network when processing sample drone inspection images. Through this optimization process, the server can continuously adjust the first and second network learning weight information, enabling the entire machine learning network to more accurately generate power plant abnormality prediction results when processing sample drone inspection images, thereby improving the accuracy of abnormality diagnosis for large-scale photovoltaic power plants.

[0058] In a possible implementation, the first network learning weight information includes sub-weight information of each inspection unit block in the sample drone inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block.

[0059] Step S1222 includes:

[0060] Determine the random interference features corresponding to each inspection unit block in the sample drone inspection image, and optimize the sub-weight information of each inspection unit block based on the random interference features corresponding to each inspection unit block to generate the target optimization confidence of each inspection unit block.

[0061] Based on the target optimization confidence of each of the inspection unit blocks, each first inspection unit block to be optimized is determined.

[0062] In this embodiment, in the process of processing the sample drone inspection image, the first network learning weight information covers the sub-weight information of each inspection unit block in the sample drone inspection image. This sub-weight information plays a key role in determining whether to optimize the first image semantic features of the inspection unit block. In order to determine which of the inspection unit blocks in the sample drone inspection image are the first inspection unit blocks to be optimized, first, the server must determine the random interference features corresponding to each inspection unit block in the sample drone inspection image. For a sample drone inspection image, it is obtained by a drone inspecting a large photovoltaic power station and contains multiple inspection unit blocks. Each inspection unit block will have different random interference features due to its own environment, position, and various factors during image acquisition. For example, from the perspective of environmental factors, the unevenness of light intensity will have different characterization features in each inspection unit block. Assume that a certain inspection unit block is located on the side of a photovoltaic power station close to a building. The shadow of the building may cause uneven light intensity in the inspection unit block, with some areas brighter and some areas darker. This uneven light intensity is a random interference feature of the inspection unit block.

[0063] Considering the distribution characteristics of Gaussian noise in each inspection unit block, during the image acquisition process, Gaussian noise is inevitably introduced due to the physical characteristics of sensors and other equipment. The distribution of Gaussian noise may vary for different inspection unit blocks. For example, in the sample drone inspection image, a certain inspection unit block near the edge of the image may have relatively high Gaussian noise intensity due to problems such as signal transmission. This different Gaussian noise distribution characteristic is one of the random interference characteristics of this inspection unit block.

[0064] In addition, the color deviation features of each inspection unit block are also part of the random interference features. Due to the differences in aging degree, surface contamination, or light reflection of photovoltaic panels in different areas of the photovoltaic power station, the inspection unit blocks in the sample unmanned aerial vehicle inspection image may exhibit different color deviations. For example, if the surface of the photovoltaic panel in a certain inspection unit block has a slight stain, it may cause the color of the inspection unit block in the image to be different from that of other normal inspection unit blocks. This color deviation feature is a random interference feature of the inspection unit block.

[0065] After determining the random interference features of each inspection unit block, the server optimizes the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, thereby generating the target optimization confidence of each inspection unit block. Thus, based on the target optimization confidence of each inspection unit block, the server determines each first inspection unit block to be optimized. The server sets a threshold, for example, 0.5. If the target optimization confidence of a certain inspection unit block is greater than the threshold, the inspection unit block is determined as a first inspection unit block to be optimized. In this way, the server can reasonably determine which inspection unit blocks need to be optimized according to the actual situation of each inspection unit block in the sample unmanned aerial vehicle inspection image, thereby improving the processing effect of the entire network on the sample unmanned aerial vehicle inspection image and laying a foundation for accurately predicting the abnormal state of the power station.

[0066] In one possible implementation, the step of determining the random interference features corresponding to each inspection unit block in the sample unmanned aerial vehicle inspection image, and optimizing the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, to generate the target optimization confidence of each inspection unit block, includes:

[0067] Extracting the environmental features related to each inspection unit block from the overall environmental data of the sample unmanned aerial vehicle inspection image to generate a preliminary environmental feature set of each inspection unit block. The environmental features include the representation of the unevenness of the light intensity in each inspection unit block, the distribution of the Gaussian noise in each inspection unit block, and the color deviation features of each inspection unit block.

[0068] Performing correlation analysis on each environmental feature in the environmental feature set of each inspection unit block, specifically analyzing the mutual relationship between the light distribution and the noise level in each inspection unit block, and analyzing the relationship between the color balance and the light intensity distribution, to construct an environmental influence matrix of each inspection unit block. The environmental influence matrix describes the interaction relationship between different environmental factors in the inspection unit block.

[0069] According to the environmental impact matrix, random interference features are generated for each inspection unit block, specifically: the elements in the environmental impact matrix are weighted combined according to a predefined weighting algorithm, and the result of the weighted combination is the random interference feature corresponding to each inspection unit block.

[0070] For each inspection unit block, a mapping relationship between the random interference characteristics of the inspection unit block and the sub-weight information is established. The mapping relationship is based on a pre-constructed rule base, which is obtained by analyzing the prior photovoltaic power station inspection data and training the machine learning algorithm. In the rule base, there are corresponding adjustment rules for the random interference characteristics and the sub-weight information of the inspection unit block, thereby adjusting the sub-weight information of the inspection unit block according to the adjustment rule.

[0071] According to the positional relationship and logical correlation of each inspection unit block in the sample drone inspection image, the adjusted sub-weight information of all inspection unit blocks is fused to generate fused weight information, and the fused weight information is converted into the target optimization confidence of each inspection unit block by setting a conversion function. The conversion function is used to map the fused weight information to an interval between 0 and 1. The value in this interval represents the target optimization confidence of each inspection unit block. The higher the confidence, the more the inspection unit block needs to optimize the image semantic features.

[0072] In this embodiment, the server first extracts environmental features associated with each inspection unit block from the overall environmental data of the sample drone inspection image, thereby generating a preliminary set of environmental features for each inspection unit block. The sample drone inspection image was captured by a drone during an inspection of a large photovoltaic power plant and contains visual information of various parts of the plant. For each inspection unit block, the server needs to consider various environmental factors. For example, the characteristic of light intensity nonuniformity in each inspection unit block can vary due to the layout of photovoltaic panels, obstruction by surrounding buildings, or terrain. For example, some inspection unit blocks may be partially shaded while others are exposed to full sunlight, resulting in nonuniform light intensity. Specific manifestations of this nonuniformity, such as the ratio of shaded to illuminated areas and the gradient of light intensity, are extracted as characteristic features of light intensity nonuniformity in each inspection unit block.

[0073] At the same time, the distribution characteristics of Gaussian noise in each inspection unit block are also an important component of the environmental characteristics. In the process of the image acquisition device acquiring the sample drone inspection image, Gaussian noise will inevitably be introduced due to the physical characteristics of the sensor and the interference of the surrounding electromagnetic environment. The distribution of Gaussian noise is different for different inspection unit blocks. For example, the inspection unit block near the edge of the image may have higher Gaussian noise intensity and more uneven distribution due to signal transmission loss or sensitivity changes at the edge of the sensor. The server determines the parameters such as the mean, variance, and distribution shape of the Gaussian noise in each inspection unit block by analyzing the image data, thereby characterizing the distribution characteristics of Gaussian noise in each inspection unit block.

[0074] The color deviation characteristics of each inspection unit block should also not be ignored. During the long-term operation of photovoltaic panels in photovoltaic power stations, they may show color deviation in the sample drone inspection images due to surface contamination, aging, or local failures. For example, if there is dust accumulation on the surface of the photovoltaic panel in a certain inspection unit block, the color of that part in the image may become dim; and if there is a local short circuit or heat generation, the color of the photovoltaic panel may change subtly. The server analyzes the image color information to determine the color deviation value of each inspection unit block relative to the normal state, such as the color offset of the red, green, and blue channels, and collects this information as the color deviation characteristics of each inspection unit block to form a preliminary environmental feature set for each inspection unit block.

[0075] Next, the server performs a correlation analysis on the various environmental features in the environmental feature set of each inspection unit block. In this process, the relationship between the light distribution and the noise level within each inspection unit block is an important analysis content. In some cases, areas with uneven light intensity are often accompanied by higher noise levels. For example, in shadowed areas with weak light, the image sensor may automatically adjust the gain in order to obtain sufficient brightness information, which easily introduces more Gaussian noise. The server establishes a mathematical relationship model between light intensity and noise level by analyzing a large number of sample drone inspection images. Specifically, it will count the changes in noise levels under different light intensities and determine the correlation coefficient between light intensity and noise level in different inspection unit blocks.

[0076] The relationship between color balance and light intensity distribution is also analyzed. Differences in light intensity can significantly affect color balance. For example, in areas with stronger light intensity, colors may appear more vivid, while in areas with weaker light, colors may appear duller. The server determines the functional relationship between color balance and light intensity distribution by analyzing the changes in color channel values ​​under different light intensities. For example, for a particular inspection unit block, as the light intensity changes from strong to weak, the value of the red channel may decrease according to a certain ratio, while the values ​​of the green and blue channels will also change accordingly. Through this analysis, the server constructs an environmental impact matrix for each inspection unit block. This environmental impact matrix details the interactions between different environmental factors within the inspection unit block. For example, one element in the matrix may represent the degree of influence of light intensity nonuniformity on color deviation characteristics, while another element may represent the dependence of Gaussian noise on light intensity nonuniformity, and so on.

[0077] Then, based on the constructed environmental impact matrix, the server generates a random interference signature for each inspection unit block. Specifically, the elements in the environmental impact matrix are weighted and combined according to a predefined weighting algorithm. This weighting algorithm is determined based on a deep understanding of the PV power plant environment and image characteristics. For example, in some cases, the impact of uneven light intensity on overall image quality may be considered more significant. In this case, the matrix elements related to uneven light intensity will be given a higher weight in the weighting algorithm. Through this weighted combination, the result is the random interference signature corresponding to each inspection unit block. This random interference signature comprehensively considers the impact of various environmental factors such as light, noise, and color on the inspection unit block.

[0078] For each inspection unit block, the server establishes a mapping relationship between the random interference features of the inspection unit block and the sub-weight information. This mapping relationship is based on a pre-built rule base, which is obtained through in-depth analysis of prior photovoltaic power station inspection data and training of machine learning algorithms. In this rule base, a large number of corresponding adjustment rules between different random interference features and sub-weight information are stored. For example, if the random interference features of an inspection unit block indicate high light intensity unevenness, large Gaussian noise, and obvious color deviation, the sub-weight information of the inspection unit block may be significantly improved according to the adjustment rules in the rule base. This is because in this case, the image semantic features of the inspection unit block are subject to greater interference and need to be optimized. The server adjusts the sub-weight information of each inspection unit block according to these adjustment rules.

[0079] Finally, based on the positional relationships and logical connections between each inspection unit block in the sample drone inspection image, the server fuses the adjusted sub-weight information of all inspection unit blocks to generate fused weight information. The positional relationships between the inspection unit blocks in the image may affect their contribution to the overall image semantics. For example, inspection unit blocks located in the center of the image may be more important to the overall judgment, while inspection unit blocks located in the edge areas may be relatively less important. Furthermore, logical connections exist between inspection unit blocks. For example, adjacent inspection unit blocks may physically belong to the same photovoltaic array, and their states may be related in some way. During the fusion operation, the server takes these positional relationships and logical connections into account and uses an appropriate fusion algorithm, such as weighted averaging or a graph-theory-based algorithm, to fuse the adjusted sub-weight information of all inspection unit blocks into a single, fused weight information.

[0080] The server then converts the fused weight information into the target optimization confidence level for each inspection unit block using a predefined conversion function. This predefined conversion function is specifically designed to map the fused weight information to a range between 0 and 1. For example, a linear mapping function or a nonlinear sigmoid function might be used. The value within this range represents the target optimization confidence level for each inspection unit block. If the target optimization confidence level for a particular inspection unit block is high, close to 1, it indicates that the inspection unit block is subject to significant random interference and requires more image semantic feature optimization. If the target optimization confidence level is low, close to 0, it indicates that the inspection unit block is relatively stable and less likely to require image semantic feature optimization. Through this series of operations, the server can reasonably determine the target optimization confidence level for each inspection unit block based on the environmental characteristics of the sample drone inspection image and the characteristics of each inspection unit block, thereby providing a basis for subsequent image semantic feature optimization.

[0081] In a possible implementation, before step S121, the method further includes:

[0082] Determine a first statistic of the inspection unit block in the sample UAV inspection image.

[0083] If the first statistic is equal to the set statistic, the sample drone inspection image is used as a candidate sample drone inspection image.

[0084] If the first statistic is smaller than the set statistic, the sample drone inspection image is enhanced according to the set image element, and the enhanced sample drone inspection image is used as a candidate sample drone inspection image, wherein the statistic of the inspection unit block in the enhanced sample drone inspection image is the set statistic.

[0085] In this embodiment, the sample drone inspection image is captured by a drone during an inspection of a large photovoltaic power station and contains visual information of a specific area within the photovoltaic power station. In this sample drone inspection image, to facilitate subsequent analysis and processing, the image is divided into multiple inspection unit blocks according to certain rules. These inspection unit blocks can be divided based on factors such as the layout, geographic location, or function of the photovoltaic panels. For example, in a sample drone inspection image containing multiple photovoltaic panel arrays, each photovoltaic panel array or every few adjacent photovoltaic panels can be divided into an inspection unit block.

[0086] The server must first determine the first statistic of the inspection unit blocks in the sample drone inspection image. This first statistic may be related to the number, area, or other feature quantities related to the image structure of the inspection unit blocks. Assuming that this first statistic is the number of inspection unit blocks, the server calculates the number of inspection unit blocks contained in the sample drone inspection image through a pre-analysis algorithm of the image. This pre-analysis algorithm may be based on technologies such as image resolution, segmentation rules, and feature recognition in the image. For example, the server identifies features such as the boundaries of photovoltaic panels and connecting lines in the image, and then determines the boundaries of the inspection unit blocks according to pre-set segmentation rules, thereby accurately calculating the number of inspection unit blocks.

[0087] Next, the server will compare this first statistic with the set statistic. The set statistic is a standard quantity determined in advance based on the requirements of the network model or empirical values. If the first statistic is equal to the set statistic, the server will directly use the sample drone inspection image as a candidate sample drone inspection image. For example, if the set statistic is 100 inspection unit blocks, if the server calculates that the number of inspection unit blocks in a sample drone inspection image is exactly 100, then this sample drone inspection image can be directly used as a candidate sample drone inspection image, and the subsequent step of using the image semantic extraction unit to perform image semantic representation can be directly entered.

[0088] However, if the first statistic is less than the set statistic, the server will enhance the sample drone inspection image based on the set image elements and use the enhanced sample drone inspection image as a candidate sample drone inspection image, where the statistic of the inspection unit blocks in the enhanced sample drone inspection image is the set statistic. For example, suppose the set statistic is 100 inspection unit blocks, and the number of inspection unit blocks calculated in a sample drone inspection image is only 80. In this case, the server will enhance the sample drone inspection image based on the set image elements.

[0089] The setting image element can include some additional image information or processing rules. During the enhancement processing, the server can use various technical means. One possible way is to interpolate the image. Since the number of inspection unit blocks is small, it can be due to low image resolution or incomplete image coverage, etc. The server can increase the pixel information in the image through interpolation algorithm, so that the image can be divided into more inspection unit blocks. For example, the server uses a bilinear interpolation algorithm to calculate the value of a new pixel point according to the value of a known pixel point, thereby increasing the detail information of the image without changing the overall characteristics of the image, and thereby increasing the number of inspection unit blocks.

[0090] In addition, the server can also add some virtual inspection unit block elements according to the layout knowledge or prior information of the photovoltaic power station. For example, if it is known that the layout of the photovoltaic panels in a certain area is regular, but it is not completely displayed in the sample unmanned aerial vehicle inspection image due to the shooting angle or other reasons, the server can add some virtual elements representing the inspection unit blocks in the image according to the existing photovoltaic panel information and layout rules, so that the number of inspection unit blocks reaches the set statistical amount.

[0091] After these enhancement processing operations, the server obtains the enhanced sample unmanned aerial vehicle inspection image, and the statistical amount of the inspection unit blocks in this image has reached the set statistical amount. Then the server takes this enhanced sample unmanned aerial vehicle inspection image as a candidate sample unmanned aerial vehicle inspection image, so as to subsequently use the image semantic extraction unit of the initialized first image classification network to perform image semantic representation. Through such a preprocessing step, it can be ensured that the sample unmanned aerial vehicle inspection image entering the image semantic representation step meets the requirements of the network model in structure or feature amount, thereby improving the accuracy and effectiveness of subsequent image semantic representation and the entire photovoltaic power station anomaly diagnosis.

[0092] This preprocessing operation of the sample unmanned aerial vehicle inspection image is an important link in the entire photovoltaic power station anomaly diagnosis process. It not only meets the structural requirements of the network model for the input image, but also accurately extracts the semantic information in the image in the complex photovoltaic power station environment, and then accurately judges the abnormal state of the power station. Because in actual photovoltaic power station inspection, the quality and structure of the sample unmanned aerial vehicle inspection image can be affected by many factors, such as the flight height of the unmanned aerial vehicle, the shooting angle, the weather condition, etc. Through such a preprocessing step, the influence of these adverse factors can be overcome to some extent, and the robustness and accuracy of the entire diagnosis system can be improved.

[0093] For example, the sample drone inspection images captured by a drone can vary significantly under different weather conditions. On cloudy days, insufficient illumination can result in low image contrast, making the boundaries and details of some photovoltaic panels unclear, thus affecting the delineation of inspection unit blocks and the calculation of statistics. However, through the aforementioned enhancement processing steps, even under these less-than-ideal shooting conditions, the sample drone inspection images can meet the requirements of subsequent processing. Similarly, the drone's flight altitude and shooting angle can also affect the image. If the flight altitude is too high, the captured image may cover a larger area but contain less detailed information, resulting in a smaller number of inspection unit blocks. A poor shooting angle may result in incomplete display of some photovoltaic panels in the image, also affecting the delineation of inspection unit blocks. Enhancement processing based on defined image elements can, to a certain extent, compensate for these deficiencies, making the sample drone inspection images more suitable for subsequent image semantic representation and anomaly diagnosis.

[0094] In a possible implementation, the step of determining the first network learning weight information includes:

[0095] Based on the inspection path characteristics of each inspection unit block in multiple sample drone inspection images, the attention weight of the inspection unit block of each inspection node is analyzed.

[0096] Based on the attention weights of the inspection unit blocks of each inspection node in each of the sample drone inspection images, each first inspection node is determined.

[0097] First network learning weight information is determined based on the sub-weight information of the inspection unit blocks of each of the first inspection nodes.

[0098] In this embodiment, the sample drone inspection images are taken by drones during the inspection process of a large photovoltaic power station. These sample drone inspection images cover the status information of different areas of the photovoltaic power station. In these sample drone inspection images, each inspection unit block has its own specific inspection path characteristics. The inspection path characteristics are characteristics related to the flight trajectory of the drone during inspection, the shooting order, and the access order of each inspection unit block. For example, in a certain sample drone inspection image, the drone inspects and shoots the photovoltaic panel array in the order from left to right and from top to bottom. Then the inspection unit block located in the upper left corner of the image may be inspected earlier, while the inspection unit block located in the lower right corner may be inspected later. This inspection order constitutes part of the inspection path characteristics of each inspection unit block.

[0099] Based on the inspection path characteristics of each inspection unit block in these multiple sample drone inspection images, the server begins to analyze the attention weight of the inspection unit block at each inspection node. An inspection node here can be understood as a specific inspection point or moment in the entire inspection process. The attention weight of the inspection unit block corresponding to each inspection node reflects the importance of the inspection unit block at that specific moment or checkpoint to the entire inspection task.

[0100] To analyze this attention weight, the server considers various factors. First, based on inspection path characteristics, if an inspection unit block is located in a critical position along the inspection path in multiple sample drone inspection images—for example, a transition zone connecting different photovoltaic array areas, or near a fault-prone area—then the attention weight of this inspection unit block at the corresponding inspection node may be higher. This is because the status of the inspection unit blocks at these locations may be more important for determining the operating status of the entire photovoltaic power plant.

[0101] The server also considers the characteristics of the inspection unit blocks themselves. For example, if a particular inspection unit block contains a large number of photovoltaic panels, or if its corresponding photovoltaic panels are located in the core power generation area of ​​a power plant, then the attention weight of this inspection unit block will be increased at the corresponding inspection node. The server analyzes a large number of sample drone inspection images, comprehensively considering these factors, and then determines the attention weight of the inspection unit block for each inspection node.

[0102] Based on the attention weights of the inspection unit blocks of each inspection node in each sample drone inspection image, the server then determines each first inspection node. A first inspection node is a node that holds special significance or importance throughout the inspection process. The process of determining the first inspection node is a comprehensive one. The server sets certain criteria or thresholds. When the attention weight of the inspection unit blocks of a particular inspection node exceeds this threshold, the node is designated as the first inspection node.

[0103] For example, during an inspection of a large photovoltaic power plant, the server analyzed multiple sample drone inspection images and discovered that some inspection nodes, corresponding to inspection unit blocks, were located near key power plant equipment. These inspection unit blocks also had high attention weights across multiple sample drone inspection images. If a threshold is set to a specific value, the server will identify these inspection nodes as the first inspection nodes when their attention weights exceed this threshold.

[0104] Finally, based on the sub-weight information of the inspection unit blocks of each first inspection node, the server determines the first network learning weight information. The sub-weight information of the inspection unit blocks of the first inspection node is the weight information associated with each inspection unit block, which reflects the relative importance of this inspection unit block in the network learning process.

[0105] The server integrates and processes the sub-weight information of the inspection unit blocks of each first inspection node to determine the first network learning weight information. This integration and processing process may involve a variety of mathematical operations and logical judgments. For example, the server may perform weighted summation or averaging operations on the sub-weight information of the inspection unit blocks corresponding to each first inspection node based on the importance of each first inspection node. If a first inspection node is considered to be very critical in the entire inspection process, then the sub-weight information of the inspection unit blocks of this first inspection node may be given a higher weight when determining the first network learning weight information.

[0106] In this way, the server starts from the inspection path characteristics of each inspection unit block in multiple sample drone inspection images, gradually analyzes and determines the attention weight of the inspection unit block of each inspection node, then determines each first inspection node, and finally determines the first network learning weight information based on the sub-weight information of the inspection unit block of the first inspection node. This first network learning weight information will play an important role in the subsequent network learning and optimization process. For example, it serves as a key decision-making basis for determining whether to optimize the first image semantic features of the inspection unit block, thereby helping to improve the accuracy and effectiveness of the entire photovoltaic power plant anomaly diagnosis network.

[0107] In actual photovoltaic power station inspection scenarios, due to the large scale and complex structure of the power station, the roles and problems that may occur in the power generation process of photovoltaic panels in different areas are also different. By determining the first network learning weight information based on factors such as inspection path characteristics and attention weight, it is possible to better adapt to such complex situations. For example, in a large photovoltaic power station with multiple sub-station areas, the photovoltaic panel area close to the substation may have a higher attention weight during the inspection process because it is directly related to power transmission and conversion equipment. The corresponding first inspection node is also more likely to determine the inspection time near this area. The sub-weight information of the inspection unit blocks in these areas will also occupy an important position in determining the first network learning weight information, so that the entire network learning process can focus more specifically on the image features of these key areas, improving the ability to diagnose abnormal conditions of photovoltaic power stations.

[0108] In one possible implementation, the method further includes:

[0109] determining a power station knowledge label of the large-scale photovoltaic power station.

[0110] If the power station knowledge label is a first knowledge label, the one or more image semantic encoding nodes include each image semantic encoding node in the plurality of sequentially connected image semantic encoding nodes.

[0111] If the power station knowledge label is not the first knowledge label, the one or more image semantic encoding nodes include part of the image semantic encoding nodes in the plurality of sequentially connected image semantic encoding nodes.

[0112] In this embodiment, in the whole process of inspection data processing for large-scale photovoltaic power stations, the server needs to determine the power station knowledge label of the large-scale photovoltaic power station. This power station knowledge label is a comprehensive identification of the overall characteristics, operating state, equipment type, and other aspects of knowledge of the large-scale photovoltaic power station. For example, a large-scale photovoltaic power station may be assigned different power station knowledge labels due to factors such as the photovoltaic panel model it uses, the inverter type it installs, the geographical environment it is in (such as whether it is in a high-altitude, strong wind and sand area, etc.), or the running age of the power station.

[0113] Suppose a large-scale photovoltaic power station A, which uses a specific model of high-efficiency photovoltaic panels, installs a new type of inverter, and is located in a coastal area with a short running age. The server will integrate this information and determine the power station knowledge label of this power station by querying a pre-established power station knowledge database or through specific algorithm analysis. This label contains all the relevant feature information of the power station, and will serve as an important basis for subsequent processing of sample unmanned aerial vehicle inspection images.

[0114] After determining the power station knowledge label, the server will use the label to determine the use of image semantic encoding nodes when processing sample unmanned aerial vehicle inspection images. If the power station knowledge label is a first knowledge label, it means that the power station has certain specific attributes or is in a certain specific state, at which time one or more image semantic encoding nodes will include each image semantic encoding node in the plurality of sequentially connected image semantic encoding nodes.

[0115] Take, for example, sample drone inspection images captured during inspections of different areas of a large photovoltaic power plant, A. When the plant knowledge label for plant A is the first knowledge label, the server uses the image semantic extraction unit of the first image classification network to process the sample drone inspection images, enabling all image semantic encoding nodes. Each image semantic encoding node has a specific function, potentially extracting features at different levels of the image or representing different aspects of semantic information. For example, the first image semantic encoding node might focus on extracting basic texture features, the second might analyze the shape of objects in the image, and the last node might combine the features extracted by the previous nodes to produce a higher-level semantic representation. Due to the unique properties of plant A (identified by the first knowledge label), the server determines that all these image semantic encoding nodes are necessary to comprehensively and meticulously process the sample drone inspection images in order to accurately capture the semantic information in the images and determine whether the plant is experiencing an abnormality.

[0116] However, if the power station knowledge label is not the first knowledge label, this indicates that the power station has other different characteristics or states. In this case, one or more image semantic coding nodes will only include some of the image semantic coding nodes in the multiple sequentially connected image semantic coding nodes. For example, for another large photovoltaic power station B, its power station knowledge label is different from that of power station A, which may be due to factors such as its photovoltaic panel model is more common, its operating years are longer, or its environment is more stable. For the sample drone inspection image of power station B, the server determines that it is not necessary to enable all image semantic coding nodes based on its power station knowledge label. It may be necessary to enable only the first few image semantic coding nodes, which are sufficient to provide an effective semantic representation of the sample drone inspection image. For example, only enabling the first three image semantic coding nodes can extract enough information to judge the common abnormal conditions of power station B, because the characteristics of power station B determine that there is no need to process the more complex or special case-oriented image semantic coding nodes.

[0117] By determining the scope of image semantic coding nodes based on plant knowledge tags, the server can more efficiently and accurately process sample drone inspection images of large-scale photovoltaic power plants with different characteristics, thereby improving the accuracy and efficiency of plant abnormality diagnosis. This approach fully considers the diversity and complexity of large-scale photovoltaic power plants, performing targeted operations based on the plant's own knowledge tags, avoiding unnecessary waste of computing resources, and better adapting to the needs of different power plants, providing strong support for the effective operation and maintenance of photovoltaic power plants.

[0118] In a possible implementation, step S121 includes:

[0119] Step S1211: Generate a corresponding guidance image element based on the sample drone inspection image, where the guidance image element represents the corresponding power plant abnormal status result generated by the machine learning network based on the sample drone inspection image.

[0120] Step S1212: Using the image semantic extraction unit of the first image classification network to perform image semantic representation on the guide image element.

[0121] In this embodiment, when the image semantic extraction unit of the initialized first image classification network is used to perform image semantic representation on a sample drone inspection image, the server generates corresponding guidance image elements based on the sample drone inspection image. The sample drone inspection image is obtained by a drone during an inspection of a large photovoltaic power plant and contains visual information about the various inspection unit blocks of the photovoltaic power plant. Based on this sample drone inspection image, the server generates a guidance image element using specific algorithms and rules. This guidance image element is a special information representation that indicates the corresponding power plant abnormality status result generated by the machine learning network based on the sample drone inspection image.

[0122] For example, in a large photovoltaic power plant, a sample drone inspection image might show the array layout, surface condition, and connection wiring of the photovoltaic panels. When generating the guidance image elements, the server comprehensively considers the potential connection between this information and abnormal power plant conditions. If a photovoltaic panel area in the sample drone inspection image has abnormal color or shadows on the surface, the server will encode it in a specific way in the guidance image element. This encoding method can reflect the connection between these visual features and possible abnormal power plant conditions (such as reduced photovoltaic panel power generation efficiency or local overheating).

[0123] The server then uses the image semantic extraction unit of the first image classification network to generate an image semantic representation for the guide image element. This image semantic extraction unit is a module specifically designed to extract semantic information from relevant image elements. When processing the guide image element, the image semantic extraction unit operates according to its internal structure and algorithm.

[0124] The image semantic extraction unit is assumed to include multiple layers of feature extraction and semantic analysis mechanisms. When a guide image element is input into the unit, it first extracts basic visual features from the guide image element at a lower level, such as edge information and color distribution. For example, if the guide image element encodes abnormal photovoltaic panel color, the unit can identify the distribution range of this color in the image and its boundary with the surrounding normal color areas.

[0125] As the process goes deeper, at a higher level, the image semantic extraction unit will combine these basic visual features and analyze the relationship between them to obtain higher-level semantic information. Continuing with the example of color anomaly of a photovoltaic panel, it will not only identify the area of color anomaly, but also determine whether this color anomaly is likely to cause abnormal state of the power station according to the relationship between the color anomaly area and other parts of the photovoltaic panel (such as connecting lines, surrounding photovoltaic panels, etc.), for example, whether it will affect current transmission or power output of the entire photovoltaic panel array.

[0126] Throughout the process, the image semantic extraction unit will also use its pre-trained model parameters and algorithm rules. These parameters and rules are trained based on a large amount of sample data (including previously processed sample UAV inspection images and their corresponding guide image elements and power station abnormal state results). With these pre-trained knowledge, the image semantic extraction unit can more accurately represent the guide image elements in image semantics and convert them into a semantic representation form that can be understood and processed by subsequent modules (such as the image semantic restoration unit) so as to finally accurately generate the power station abnormal state results corresponding to the sample UAV inspection images.

[0127] This way of generating guide image elements and representing them in image semantics helps to more effectively use machine learning networks to analyze sample UAV inspection images in complex photovoltaic power station inspection scenarios and improve the accuracy and reliability of power station abnormal state diagnosis. Because the guide image elements can establish a more direct connection between the key information in the sample UAV inspection images and the power station abnormal state results, and the processing of the guide image elements by the image semantic extraction unit further excavates the semantic information behind this connection, thereby providing a more valuable information basis for the entire power station abnormal diagnosis system.

[0128] Figure 2 The hardware structure of the UAV inspection system 100 for large-scale photovoltaic power stations provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the UAV inspection system 100 for large-scale photovoltaic power stations can include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140. Figure 2 As shown in FIG. 1, the UAV inspection system 100 for large-scale photovoltaic power stations can include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.

[0129] The machine-readable storage medium 120 can store data and / or instructions. In some embodiments, the machine-readable storage medium 120 can store data obtained from an external terminal. In some embodiments, the machine-readable storage medium 120 can store data and / or instructions used by the UAV inspection system 100 for large-scale photovoltaic power stations to perform or use to complete the exemplary methods described in the present application.

[0130] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the drone inspection method applied to large photovoltaic power stations in the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected through the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0131] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned drone inspection system 100 applied to large-scale photovoltaic power stations. The implementation principles and technical effects are similar and will not be repeated here in this embodiment.

[0132] In addition, an embodiment of the present invention further provides a readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the drone inspection method applied to large photovoltaic power stations as described above is implemented.

[0133] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, multiple features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A drone inspection method for large photovoltaic power stations, characterized in that: The method comprises: Acquire multiple sample drone inspection data of a large photovoltaic power station carrying status annotation data, each of the sample drone inspection data including a sample drone inspection image, and the status annotation data of each sample drone inspection data is a sample power station abnormal status label corresponding to the sample drone inspection image; Based on multiple sample drone inspection data, the machine learning network is optimized through recurrent network parameter optimization to generate an optimized target photovoltaic power station anomaly diagnosis network. Obtain drone inspection images of large photovoltaic power plants to be diagnosed, and use the optimized target photovoltaic power plant anomaly diagnosis network to generate corresponding power plant abnormal status results; Optimization of recurrent network parameters includes: Performing image semantic representation on each sample UAV inspection image using the image semantic extraction unit of the initialized first image classification network to generate first image semantic features of each inspection unit block in the sample UAV inspection image; determining first inspection unit blocks to be optimized among the inspection unit blocks of the sample UAV inspection image based on the first network learning weight information; and optimizing the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information to generate optimized second image semantic features; Based on the target image semantic features of each sample UAV inspection image, the image semantic restoration unit of the first image classification network is used to perform image semantic restoration to generate a corresponding power plant abnormal state prediction result, where the target image semantic features of the sample UAV inspection image are generated based on the second image semantic features of each of the first inspection unit blocks and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks; Determining a first network learning error based on the power plant abnormal state prediction results and state annotation data corresponding to each of the sample drone inspection images, optimizing the first network learning weight information and the second network learning weight information based on the first network learning error, and generating the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization; Determine that each first inspection unit block to be optimized in each inspection unit block of the sample UAV inspection image includes: The environmental feature set related to each inspection unit block is extracted from the overall environmental data of the sample drone inspection image. The correlation analysis of each environmental feature in the environmental feature set is performed to construct the environmental impact matrix of each inspection unit block. Based on the environmental impact matrix, random interference features are generated for each inspection unit block. A mapping relationship between the random interference characteristics of each inspection unit block and the sub-weight information of the first network learning weight information is established. According to the positional relationship and logical correlation of each inspection unit block in the sample UAV inspection image, the adjusted sub-weight information of all inspection unit blocks is fused, and the fused weight information is converted into the target optimization confidence of each inspection unit block by setting a conversion function to determine the first inspection unit blocks to be optimized.

2. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The image semantic extraction unit includes a plurality of sequentially connected image semantic coding nodes; the first network learning weight information includes node parameter information corresponding to one or more image semantic coding nodes; For each sample drone inspection image, the steps for generating the power plant abnormal state prediction result corresponding to the sample drone inspection image include: The sample UAV inspection image is loaded into the image semantic extraction unit, and the image semantic extraction unit is used to perform image semantic representation on the sample UAV inspection image to generate a first image semantic feature corresponding to each image semantic coding node of each inspection unit block in the sample UAV inspection image; For each of the one or more image semantic coding nodes, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample UAV inspection image based on the node parameter information corresponding to the image semantic coding node, optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information, generate the second image semantic features of each of the first inspection unit blocks, use the second image semantic features of each of the first inspection unit blocks and the first image semantic features of other inspection unit blocks except the first inspection unit blocks as the target image semantic features of the image semantic coding node, and use the target image semantic features of the image semantic coding node as the loading data of the next image semantic coding node; For each image semantic coding node other than the one or more image semantic coding nodes, the first image semantic feature of each inspection unit block in the sample UAV inspection image is obtained by using the image semantic coding node as the target image semantic feature of the image semantic coding node, and the target image semantic feature of the image semantic coding node is used as the loading data of the next image semantic coding node; According to the target image semantic features of the last image semantic coding node of the image semantic extraction unit, the image semantic restoration unit is used to perform image semantic restoration to generate a power station abnormal state prediction result corresponding to the sample drone inspection image.

3. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The first network learning weight information includes sub-weight information of each inspection unit block in the sample UAV inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block; Optimizing the first network learning weight information and the second network learning weight information based on the first network learning error includes: Determining a second network learning error based on fusion weight information of sub-weight information of each inspection unit block in each of the sample UAV inspection images; The first network learning weight information and the second network learning weight information are optimized according to the first network learning error and the second network learning error.

4. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: Before performing image semantic representation on the sample UAV inspection image using the image semantic extraction unit of the initialized first image classification network, the method further includes: Determine a first statistic of the inspection unit block in the sample UAV inspection image; If the first statistic is equal to the set statistic, the sample drone inspection image is used as a candidate sample drone inspection image; If the first statistic is smaller than the set statistic, the sample drone inspection image is enhanced according to the set image element, and the enhanced sample drone inspection image is used as a candidate sample drone inspection image, wherein the statistic of the inspection unit block in the enhanced sample drone inspection image is the set statistic.

5. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The step of determining the first network learning weight information includes: Based on the inspection path characteristics of each inspection unit block in multiple sample drone inspection images, the attention weight of the inspection unit block of each inspection node is analyzed; Determine each first inspection node based on the attention weight of the inspection unit block of each inspection node in each of the sample drone inspection images; First network learning weight information is determined based on the sub-weight information of the inspection unit blocks of each of the first inspection nodes.

6. The drone inspection method for large photovoltaic power stations according to claim 2, characterized in that: The method further comprises: Determining a power station knowledge tag of the large-scale photovoltaic power station; If the power station knowledge tag is the first knowledge tag, the one or more image semantic coding nodes include each image semantic coding node in the plurality of sequentially connected image semantic coding nodes; If the power station knowledge tag is not the first knowledge tag, the one or more image semantic coding nodes include some image semantic coding nodes among the plurality of sequentially connected image semantic coding nodes.

7. The drone inspection method for large photovoltaic power stations according to claim 1, characterized in that: The image semantic extraction unit using the initialized first image classification network performs image semantic representation on the sample UAV inspection image, including: Generating a corresponding guide image element based on the sample drone inspection image, wherein the guide image element represents a corresponding power plant abnormal state result generated by the machine learning network based on the sample drone inspection image; The image semantics extraction unit of the first image classification network is used to perform image semantic representation on the guide image elements.

8. A drone inspection system for large photovoltaic power stations, characterized in that: The drone inspection system applied to large-scale photovoltaic power stations includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the drone inspection method applied to large-scale photovoltaic power stations as described in any one of claims 1 to 7 above.

Images