Unmanned aerial vehicle inspection method and system applied to large photovoltaic power station

Abstract

The invention provides an unmanned aerial vehicle inspection method and system applied to a large photovoltaic power station, and the method comprises the steps: obtaining sample unmanned aerial vehicle inspection data carrying state annotation data, carrying out the circulation network parameter optimization through a machine learning network, generating an optimized target photovoltaic power station abnormality diagnosis network, and carrying out the fault diagnosis of the target photovoltaic power station. The accuracy and efficiency of unmanned aerial vehicle inspection can be effectively improved. Specifically, image semantic representation and optimization are carried out on a sample unmanned aerial vehicle inspection image to generate more accurate target image semantic features, and then an image semantic restoration unit is utilized to generate a power station abnormal state prediction result. And by continuously iteratively optimizing the network learning weight information, the finally generated target photovoltaic power station anomaly diagnosis network has higher diagnosis precision. In practical application, the abnormal state in the large-scale photovoltaic power station can be rapidly and accurately identified, so that operation and maintenance management of the large-scale photovoltaic power station is facilitated, and the operation efficiency and safety of the photovoltaic power station are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic power station inspection, and in particular to an unmanned aerial vehicle inspection method and system applied to large photovoltaic power stations. Background Art

[0002] With the transformation of energy structure and the rapid development of renewable energy, large-scale photovoltaic power stations, as an important part of clean energy, are expanding in scale, and operation and maintenance management are also facing unprecedented challenges. The traditional inspection method of photovoltaic power stations mainly relies on manual labor, but limited by the vast area and complex equipment layout of photovoltaic power stations, manual inspection is not only time-consuming and labor-intensive, but also difficult to ensure the comprehensiveness and timeliness of the inspection. Therefore, how to efficiently and accurately complete the inspection task of photovoltaic power stations has become a problem that needs to be solved urgently.

[0003] Among the related technologies, drone technology has been widely used in the inspection of photovoltaic power stations due to its high efficiency and flexibility. Drones can quickly cover various areas of photovoltaic power stations and obtain real-time images of the operating status of power stations by carrying high-resolution cameras and other sensor equipment. However, faced with a large amount of drone inspection image data, how to quickly and accurately identify abnormal conditions has become a key problem in drone inspection technology. Summary of the invention

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a drone inspection method applied to a large photovoltaic power station, the method comprising:

[0005] Acquire multiple sample drone inspection data carrying status annotation data of a large photovoltaic power station, each of the sample drone inspection data includes a sample drone inspection image, and the status annotation data of each sample drone inspection data is a sample power station abnormal status label corresponding to the sample drone inspection image in the sample drone inspection data;

[0006] Based on multiple sample drone inspection data, the machine learning network is optimized through cyclic network parameter optimization to generate an optimized target photovoltaic power station abnormality diagnosis network;

[0007] Obtain the unmanned aerial vehicle inspection image to be diagnosed of the large photovoltaic power station, and use the optimized target photovoltaic power station abnormality diagnosis network to generate the abnormal status result of the power station corresponding to the unmanned aerial vehicle inspection image to be diagnosed;

[0008] The step of optimizing the cyclic network parameters includes:

[0009] For each sample drone inspection image, use the image semantic extraction unit of the initialized first image classification network to perform image semantic representation on the sample drone inspection image, generate first image semantic features of each inspection unit block in the sample drone inspection image, determine each first inspection unit block to be optimized in each inspection unit block of the sample drone inspection image based on the first network learning weight information, and optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information to generate optimized second image semantic features of each of the first inspection unit blocks;

[0010] For each sample drone inspection image, based on the target image semantic features of the sample drone inspection image, the image semantic restoration unit of the first image classification network is used to perform image semantic restoration to generate a prediction result of the abnormal state of the power plant corresponding to the sample drone inspection image, wherein the target image semantic features of the sample drone inspection image are generated based on the second image semantic features of each of the first inspection unit blocks in the sample drone inspection image and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks;

[0011] Based on the power plant abnormal state prediction results and state annotation data corresponding to each of the sample drone inspection images, the first network learning error is determined, and according to the first network learning error, the first network learning weight information and the second network learning weight information are optimized to generate the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization.

[0012] On the other hand, an embodiment of the present invention also provides a drone inspection system for a large photovoltaic power station, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0013] Based on the above aspects, the embodiment of the present application can effectively improve the accuracy and efficiency of drone inspections by acquiring sample drone inspection data carrying status annotation data, and using a machine learning network to perform cyclic network parameter optimization to generate an optimized target photovoltaic power station abnormality diagnosis network. Specifically, by performing image semantic representation and optimization on sample drone inspection images, more accurate target image semantic features are generated, and then the image semantic restoration unit is used to generate power station abnormal state prediction results. By continuously iterating and optimizing network learning weight information, the target photovoltaic power station abnormality diagnosis network finally generated has higher diagnostic accuracy. In practical applications, it is possible to quickly and accurately identify abnormal states in large photovoltaic power stations, so as to facilitate the operation and maintenance management of large photovoltaic power stations, greatly improving the operating efficiency and safety of photovoltaic power stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the execution flow of the drone inspection method applied to a large photovoltaic power station provided by an embodiment of the present invention.

[0015] Figure 2 It is a schematic diagram of the hardware architecture of a drone inspection system for a large photovoltaic power station provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 The figure is a flow chart of a drone inspection method for a large photovoltaic power station provided by an embodiment of the present invention. The drone inspection method for a large photovoltaic power station is introduced in detail below.

[0017] Step S110, obtaining multiple sample drone inspection data carrying status annotation data of a large photovoltaic power station, each of the sample drone inspection data includes a sample drone inspection image, and the status annotation data of each sample drone inspection data is the sample power station abnormal status label corresponding to the sample drone inspection image in the sample drone inspection data.

[0018] In this embodiment, the server establishes a connection with the database or data storage system of a large photovoltaic power station. In large photovoltaic power stations, daily inspections are performed by drones. These drones are equipped with high-definition cameras and other equipment. During the inspection process, they shoot various areas of the photovoltaic power station according to predetermined routes and rules, and obtain a large number of drone inspection images. At the same time, in the operation and maintenance management of the power station, professional technicians will analyze these drone inspection images to determine whether there is an abnormal state in the photovoltaic power station area corresponding to the image, such as whether the photovoltaic panel is damaged, whether the surface is blocked by stains, whether the line connection is normal, etc., and mark each drone inspection image with a corresponding abnormal state label, which is the state annotation data.

[0019] Assume that this large photovoltaic power station contains multiple areas, such as Area A, Area B, Area C, etc., and each area contains a large number of photovoltaic panel arrays. The server obtains sample drone inspection data from the data storage. For example, a sample drone inspection image is obtained from Area A. The sample drone inspection image shows a part of a row of photovoltaic panels. After analysis and annotation by technicians, the status annotation data corresponding to this sample drone inspection image is "a small amount of stains on the surface of the photovoltaic panel are blocked." Another sample drone inspection image is obtained from Area B. The sample drone inspection image shows the connection line of a photovoltaic panel array, and the annotation data is "the line connection is normal." The server continues to obtain multiple such sample drone inspection data. Each sample drone inspection data contains a sample drone inspection image and the corresponding sample power station abnormal status label. These data will serve as the basic data for subsequent machine learning network training and optimization.

[0020] Step S120, based on multiple sample drone inspection data, a cyclic network parameter optimization is performed on the machine learning network to generate an optimized target photovoltaic power station abnormality diagnosis network.

[0021] In this embodiment, a machine learning framework is running on the server, and a first image classification network for image classification has been initialized in the machine learning framework. The first image classification network includes components such as an image semantic extraction unit and an image semantic restoration unit. The server begins to use the acquired multiple sample drone inspection data to perform cyclic network parameter optimization on the machine learning network.

[0022] Step S130, obtaining the unmanned aerial vehicle inspection image to be diagnosed of the large photovoltaic power station, and using the optimized target photovoltaic power station abnormality diagnosis network to generate the power station abnormal status result corresponding to the unmanned aerial vehicle inspection image to be diagnosed.

[0023] During the continuous inspection of the photovoltaic power station, the drone captures new drone inspection images to be diagnosed. The server obtains these drone inspection images to be diagnosed. Assume that the drone inspection images to be diagnosed show a new area in the photovoltaic power station or a new situation in the previous area.

[0024] The server loads the unmanned aerial vehicle inspection image to be diagnosed into the optimized target photovoltaic power plant abnormality diagnosis network. The target photovoltaic power plant abnormality diagnosis network processes the unmanned aerial vehicle inspection image to be diagnosed according to the parameters optimized by multiple sample unmanned aerial vehicle inspection data.

[0025] First, the unmanned aerial vehicle inspection image to be diagnosed will pass through the image semantic extraction unit, and generate image semantic features of each inspection unit block according to the same processing flow as the previous sample unmanned aerial vehicle inspection image, and determine the inspection unit block to be optimized based on the weight information learned by the network, and optimize it. Then, the target image semantic features are generated based on the processed image semantic features, and then the image semantic restoration unit is used to restore the image semantics, and finally the abnormal state result of the power station corresponding to the unmanned aerial vehicle inspection image to be diagnosed is obtained, such as "loose line connection" and other results. This result can provide timely and accurate diagnostic information for the operation and maintenance personnel of the photovoltaic power station so that they can perform corresponding maintenance and repair work on the power station.

[0026] The step of optimizing the cyclic network parameters includes:

[0027] Step S121: for each sample UAV inspection image, use the image semantic extraction unit of the initialized first image classification network to perform image semantic representation on the sample UAV inspection image, generate the first image semantic features of each inspection unit block in the sample UAV inspection image, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample UAV inspection image based on the first network learning weight information, and optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information to generate the optimized second image semantic features of each of the first inspection unit blocks.

[0028] For example, for each sample drone inspection image obtained, the server loads it into the image semantic extraction unit of the first image classification network. Assume that this image semantic extraction unit is composed of multiple image semantic encoding nodes connected in sequence. Take a sample drone inspection image showing a photovoltaic panel array in a certain area of ​​a photovoltaic power station as an example. This sample drone inspection image is divided into multiple inspection unit blocks. For example, according to the arrangement of photovoltaic panels, every few photovoltaic panels are divided into one inspection unit block.

[0029] The server processes each inspection unit block through the first image semantic encoding node in the image semantic extraction unit to generate the first image semantic features of each inspection unit block. These first image semantic features contain information related to image semantics such as color, texture, shape, etc. of the inspection unit block.

[0030] The first network learning weight information includes the sub-weight information of each inspection unit block in the sample drone inspection image, and these sub-weight information are used to determine whether to optimize the first image semantic features of the inspection unit block. For example, for an inspection unit block located at the edge of the image, since it may be greatly affected by uneven lighting and image acquisition angle, the corresponding sub-weight information may be higher, indicating that this inspection unit block is more likely to be determined as the first inspection unit block to be optimized. The server determines which inspection unit blocks in the image are the first inspection unit blocks to be optimized based on these sub-weight information.

[0031] After determining the first inspection unit blocks, the server optimizes the first image semantic features of these first inspection unit blocks based on the second network learning weight information. Assuming that the second network learning weight information is some adjustment coefficients, the server generates the optimized second image semantic features by performing specific mathematical operations (such as multiplication, addition, etc.) on the first image semantic features of the first inspection unit blocks and these adjustment coefficients. For example, if the first image semantic features of a first inspection unit block are represented as a vector [1, 2, 3], and the corresponding second network learning weight information is [0.8, 1.2,0.9], then the optimized second image semantic features may be [1 * 0.8, 2 * 1.2, 3 * 0.9] = [0.8, 2.4, 2.7].

[0032] Step S122: For each sample drone inspection image, based on the target image semantic features of the sample drone inspection image, use the image semantic restoration unit of the first image classification network to perform image semantic restoration to generate a power plant abnormal state prediction result corresponding to the sample drone inspection image, wherein the target image semantic features of the sample drone inspection image are generated based on the second image semantic features of each of the first inspection unit blocks in the sample drone inspection image and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks.

[0033] For the previously processed sample drone inspection image, the server generates the target image semantic features of the sample drone inspection image based on the obtained second image semantic features of each first inspection unit block and the first image semantic features of other inspection unit blocks except these first inspection unit blocks. For example, assuming that the image has 10 inspection unit blocks, 3 of which are first inspection unit blocks, the server combines the second image semantic features of these 3 first inspection unit blocks and the first image semantic features of the other 7 inspection unit blocks in a certain order to form the target image semantic features.

[0034] Then, the server loads the semantic features of the target image into the image semantic restoration unit of the first image classification network. This image semantic restoration unit performs image semantic restoration operations on the semantic features of the target image based on the pre-trained model structure and parameters. For example, if the semantic features of the target image are a high-dimensional vector, the image semantic restoration unit may convert this vector into a prediction result representing the abnormal state of the power plant through a series of linear and nonlinear transformations. Assuming that this prediction result is a classification result, it may be one of the following: "no abnormality", "broken photovoltaic panel", "line fault", etc. This is the prediction result of the abnormal state of the power plant corresponding to the sample drone inspection image.

[0035] Step S123, based on the power plant abnormal state prediction results and state annotation data corresponding to each of the sample UAV inspection images, determine the first network learning error, optimize the first network learning weight information and the second network learning weight information according to the first network learning error, and generate the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization.

[0036] The server obtains the abnormal state prediction result of the power station for each sample drone inspection image, and also has the state annotation data corresponding to each sample drone inspection image. For example, for a sample drone inspection image, its prediction result is "no abnormality", while the state annotation data is "a small amount of stains on the surface of the photovoltaic panel", which means that there is a difference between the prediction result and the actual annotation.

[0037] The server determines the first network learning error based on the prediction results and state annotation data of all sample drone inspection images through a specific error calculation method (such as cross entropy error, etc.). This error reflects the prediction accuracy of the current first image classification network.

[0038] Then, based on the first network learning error, the server optimizes the first network learning weight information and the second network learning weight information. For example, if the sub-weight information of a certain inspection unit block causes a large error in the current network, the server may adjust the value of this sub-weight information so that it can more accurately determine the first inspection unit block to be optimized in subsequent training. At the same time, the second network learning weight information will also be adjusted according to the error situation to make the optimization of the first image semantic features of the first inspection unit block more reasonable. Through such an optimization process, the server generates the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization, so as to carry out the next round of network optimization.

[0039] Based on the above steps, the embodiment of the present application obtains sample drone inspection data carrying status annotation data, and uses a machine learning network to optimize cyclic network parameters to generate an optimized target photovoltaic power station abnormality diagnosis network, which can effectively improve the accuracy and efficiency of drone inspections. Specifically, by performing image semantic representation and optimization on sample drone inspection images, more accurate target image semantic features are generated, and then the image semantic restoration unit is used to generate power station abnormal state prediction results. By continuously iterating and optimizing network learning weight information, the target photovoltaic power station abnormality diagnosis network finally generated has higher diagnostic accuracy. In practical applications, it is possible to quickly and accurately identify abnormal states in large photovoltaic power stations, so as to facilitate the operation and maintenance management of large photovoltaic power stations, greatly improving the operating efficiency and safety of photovoltaic power stations.

[0040] In a possible implementation, the image semantics extraction unit includes a plurality of sequentially connected image semantics encoding nodes. The first network learning weight information includes node parameter information corresponding to one or more image semantics encoding nodes.

[0041] For each sample drone inspection image, the steps for generating the abnormal state prediction result of the power station corresponding to the sample drone inspection image include:

[0042] Step S1221, load the sample drone inspection image into the image semantic extraction unit, use the image semantic extraction unit to perform image semantic representation on the sample drone inspection image, and generate the first image semantic feature of each inspection unit block in the sample drone inspection image corresponding to each image semantic coding node.

[0043] Step S1222: For each of the one or more image semantic coding nodes, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample UAV inspection image based on the node parameter information corresponding to the image semantic coding node, optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information, generate the second image semantic features of each of the first inspection unit blocks, use the second image semantic features of each of the first inspection unit blocks and the first image semantic features of other inspection unit blocks except the first inspection unit blocks as the target image semantic features of the image semantic coding node, and use the target image semantic features of the image semantic coding node as the loading data of the next image semantic coding node.

[0044] Step S1223, for each image semantic coding node except the one or more image semantic coding nodes, the image semantic coding node will be used to obtain the first image semantic feature of each inspection unit block in the sample UAV inspection image as the target image semantic feature of the image semantic coding node, and the target image semantic feature of the image semantic coding node will be used as the loading data of the next image semantic coding node.

[0045] Step S1224, based on the target image semantic features of the last image semantic coding node of the image semantic extraction unit, the image semantic restoration unit is used to perform image semantic restoration to generate a power plant abnormal state prediction result corresponding to the sample UAV inspection image.

[0046] In this embodiment, the image semantic extraction unit includes a plurality of sequentially connected image semantic coding nodes, and the first network learning weight information covers node parameter information corresponding to one or more image semantic coding nodes.

[0047] First, the server loads the sample drone inspection image into the image semantic extraction unit. For example, there is a sample drone inspection image that is taken from a specific area of ​​a large photovoltaic power station, which contains multiple photovoltaic panel arrays. After loading into the image semantic extraction unit, for each inspection unit block in the sample drone inspection image (these inspection unit blocks are divided according to certain rules, such as the layout of photovoltaic panels or geographical location), each image semantic encoding node in the image semantic extraction unit will process it, thereby generating the first image semantic features of each inspection unit block in the sample drone inspection image corresponding to each image semantic encoding node. These first image semantic features are a semantic representation of the inspection unit block under the image semantic encoding node, which contains information about the color, texture, shape and relationship between the inspection unit blocks.

[0048] Next, operations are performed on each of the one or more image semantic coding nodes. For one of the image semantic coding nodes, the server determines the first inspection unit blocks to be optimized in the inspection unit blocks of the sample drone inspection image based on the node parameter information corresponding to the image semantic coding node. These node parameter information may be related to factors such as the position of the inspection unit block in the image, the surrounding environment characteristics, or the relationship with other inspection unit blocks. For example, under a certain image semantic coding node, according to its node parameter information, if a certain inspection unit block is located in the shadow area of ​​the image, since the shadow may have a greater impact on the image semantic features, then this inspection unit block may be determined as the first inspection unit block to be optimized. After determining the first inspection unit block, the server optimizes the first image semantic features of each first inspection unit block based on the second network learning weight information. Assuming that the second network learning weight information is a set of numerical values ​​related to feature adjustment, the first image semantic features are optimized by a specific calculation method (such as matrix multiplication, etc.), thereby generating the second image semantic features of each first inspection unit block. Then, the second image semantic features of each first inspection unit block and the first image semantic features of other inspection unit blocks except each first inspection unit block are combined, and the combined features are used as the target image semantic features of the image semantic coding node, and the target image semantic features are used as the loading data of the next image semantic coding node. In this way, through the processing of this image semantic coding node, not only some inspection unit blocks are optimized, but also suitable data are prepared for the next image semantic coding node.

[0049] For each image semantic coding node other than the one or more image semantic coding nodes, the operation is relatively straightforward. The server directly uses the first image semantic feature of each inspection unit block in the sample drone inspection image obtained using the image semantic coding node as the target image semantic feature of the image semantic coding node, and then uses this target image semantic feature as the loading data for the next image semantic coding node. For example, for a certain intermediate image semantic coding node, it does not perform special optimization operations on the inspection unit block, but simply passes the first image semantic feature generated by itself to the next node intact.

[0050] Finally, when the last image semantic encoding node in the image semantic extraction unit obtains the target image semantic feature, the server uses the image semantic restoration unit to perform image semantic restoration based on the target image semantic feature, thereby generating the prediction result of the abnormal state of the power station corresponding to the sample drone inspection image. This image semantic restoration unit is constructed based on the previous training data, and it can convert the target image semantic feature of the last image semantic encoding node into a prediction result related to the abnormal state of the power station. For example, if the target image semantic feature is a high-dimensional vector, the image semantic restoration unit may contain a series of matrix transformations and nonlinear activation functions, through which the high-dimensional vector is converted into a classification result representing the abnormal state of the power station, such as "no abnormality", "stains on the surface of the photovoltaic panel", "line connection failure", etc. This result is the prediction result of the abnormal state of the power station corresponding to the sample drone inspection image, which provides an important reference for the operation and maintenance management of the photovoltaic power station.

[0051] In a possible implementation, the first network learning weight information includes sub-weight information of each inspection unit block in the sample UAV inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block.

[0052] Step S123 includes:

[0053] Step S1231, determining the second network learning error based on the fusion weight information of the sub-weight information of each inspection unit block in each of the sample UAV inspection images.

[0054] Step S1232: Optimize the first network learning weight information and the second network learning weight information according to the first network learning error and the second network learning error.

[0055] In this embodiment, in the entire process of abnormal diagnosis of photovoltaic power stations, the first network learning weight information includes sub-weight information of each inspection unit block in the sample drone inspection image, and these sub-weight information plays a key role in the optimization process. When the server processes the sample drone inspection image, the sub-weight information of each inspection unit block determines whether to optimize the first image semantic features of the inspection unit block. For example, for a certain sample drone inspection image, it is taken from a specific area of ​​a large photovoltaic power station, which contains multiple inspection unit blocks divided according to certain rules. Each inspection unit block has corresponding sub-weight information, which is determined based on multiple factors, such as the position of the inspection unit block in the sample drone inspection image, the characteristics of the photovoltaic panels it contains, and the performance in historical data. If a patrol unit block is located in the edge area of ​​the sample drone inspection image, its influence on the semantic features of the entire image may be somewhat special due to edge effects (such as uneven lighting, angle problems during image acquisition, etc.). In this case, the sub-weight information of this patrol unit block may be set to a higher value, indicating that the first image semantic features of this patrol unit block are more likely to need to be optimized; and if a patrol unit block is in the central area of ​​the sample drone inspection image and its corresponding photovoltaic panel is relatively stable, the sub-weight information of the patrol unit block may be relatively low, indicating that the necessity to optimize its first image semantic features is relatively small.

[0056] When it comes to optimizing the first network learning weight information and the second network learning weight information based on the first network learning error, the server must first determine the second network learning error based on the fusion weight information of the sub-weight information of each inspection unit block in each sample drone inspection image. For each sample drone inspection image, the server collects the sub-weight information of each inspection unit block. For example, there is a batch of sample drone inspection images, and the server analyzes the sub-weight information of each inspection unit block in each sample drone inspection image. Assuming that a large photovoltaic power station is divided into multiple areas, and the sample drone inspection image of each area contains several inspection unit blocks, the server will perform a fusion operation on the sub-weight information of the inspection unit blocks in these different areas and different sample drone inspection images. This fusion operation may be based on a weighted average algorithm, taking into account factors such as the importance of each inspection unit block in the whole and its relationship with other inspection unit blocks. Through this fusion weight information, the server can determine the second network learning error. This second network learning error reflects the comprehensive impact of the sub-weight information of each inspection unit block on the overall network learning effect under the current network parameters.

[0057] Next, the server optimizes the first network learning weight information and the second network learning weight information according to the first network learning error and the second network learning error. The first network learning error is calculated by comparing the prediction result of the abnormal state of the power station corresponding to the sample drone inspection image with the actual state annotation data. For example, for a sample drone inspection image, the server predicts that the abnormal state of the corresponding power station is "no abnormality on the surface of the photovoltaic panel", but the actual state annotation data is "slight stains on the surface of the photovoltaic panel". The difference between the prediction result and the actual situation will be quantified as part of the first network learning error. The server takes the first network learning error and the second network learning error into consideration to adjust the first network learning weight information and the second network learning weight information. For the sub-weight information of each inspection unit block in the first network learning weight information, the server will adjust it according to the error situation. If the sub-weight information of a certain inspection unit block causes a large error in the current network, for example, in multiple sample drone inspection images, the sub-weight information of the inspection unit block at a specific position makes the processing of the first image semantic features of the inspection unit block always deviate from the actual situation, then the server will adjust this sub-weight information. The adjustment method may be to increase or decrease the value of the sub-weight information according to the size and direction of the error and in accordance with certain rules (such as an algorithm based on gradient descent, etc.). Similarly, for the second network learning weight information, the server will also adjust it according to the comprehensive situation of the first network learning error and the second network learning error. For example, if the optimization effect of the second network learning weight information on the semantic features of the first image of the first inspection unit block in the current network is not ideal, resulting in a large deviation in the final power station abnormal state prediction result, then the server will adjust the relevant parameters in the second network learning weight information according to the error situation to improve the accuracy of the entire network in the subsequent processing of the sample drone inspection image. Through such an optimization process, the server can continuously adjust the first network learning weight information and the second network learning weight information, so that the entire machine learning network can more accurately generate the power station abnormal state prediction result when processing the sample drone inspection image, thereby improving the accuracy of abnormal diagnosis of large photovoltaic power stations.

[0058] In a possible implementation, the first network learning weight information includes sub-weight information of each inspection unit block in the sample UAV inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block.

[0059] Step S1222 includes:

[0060] Determine the random interference features corresponding to each inspection unit block in the sample UAV inspection image, and optimize the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block to generate the target optimization confidence of each inspection unit block.

[0061] Based on the target optimization confidence of each of the inspection unit blocks, each first inspection unit block to be optimized is determined.

[0062] In this embodiment, in the process of processing the sample drone inspection image, the first network learning weight information covers the sub-weight information of each inspection unit block in the sample drone inspection image, and this sub-weight information plays a key role in determining whether to optimize the first image semantic features of the inspection unit block. In order to determine which of the inspection unit blocks of the sample drone inspection image are the first inspection unit blocks to be optimized, first, the server must determine the random interference features corresponding to each inspection unit block in the sample drone inspection image. For a sample drone inspection image, it is obtained by a drone inspecting a large photovoltaic power station, which contains multiple inspection unit blocks. Each inspection unit block will have different random interference features due to its own environment, position and various factors during image acquisition. For example, from the perspective of environmental factors, the unevenness of light intensity will have different characterization features in each inspection unit block. Assuming that a certain inspection unit block is located on one side of a photovoltaic power station close to a building, the shadow of the building may cause uneven light intensity in the inspection unit block, with some areas being brighter and some areas being darker. This uneven light intensity is a random interference feature of the inspection unit block.

[0063] Considering the distribution characteristics of Gaussian noise in each inspection unit block, during the image acquisition process, Gaussian noise will inevitably be introduced due to the physical characteristics of sensors and other equipment. The distribution of Gaussian noise may be different for different inspection unit blocks. For example, in the sample drone inspection image, the intensity of Gaussian noise in a certain inspection unit block near the edge of the image may be relatively large due to problems such as signal transmission. This different Gaussian noise distribution feature is one of the random interference characteristics of the inspection unit block.

[0064] In addition, the color deviation characteristics of each inspection unit block are also part of the random interference characteristics. Since photovoltaic panels in different areas of a photovoltaic power station may differ in degree of aging, surface contamination, or light reflection, the inspection unit blocks in the sample drone inspection image will show different color deviations. For example, if there are slight stains on the surface of the photovoltaic panel in a certain inspection unit block, this may cause the color of the inspection unit block in the image to be different from other normal inspection unit blocks. This color deviation feature is the random interference feature of the inspection unit block.

[0065] After determining the random interference features of each inspection unit block, the server optimizes the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, thereby generating the target optimization confidence of each inspection unit block. Thus, based on the target optimization confidence of each inspection unit block, the server determines each first inspection unit block to be optimized. The server will set a threshold, such as 0.5. If the target optimization confidence of a certain inspection unit block is greater than this threshold, then this inspection unit block will be determined as the first inspection unit block to be optimized. In this way, the server can reasonably determine which inspection unit blocks need to optimize the first image semantic features according to the actual situation of each inspection unit block in the sample drone inspection image, thereby improving the processing effect of the entire network on the sample drone inspection image, and laying the foundation for accurately predicting the abnormal state of the power station.

[0066] In a possible implementation, the step of determining the random interference features corresponding to each inspection unit block in the sample drone inspection image, optimizing the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, and generating the target optimization confidence of each inspection unit block includes:

[0067] The environmental features related to each inspection unit block are extracted from the overall environmental data of the sample UAV inspection image, and a preliminary set of environmental features for each inspection unit block is generated. The environmental features include the characterization characteristics of the unevenness of light intensity in each inspection unit block, the distribution characteristics of Gaussian noise in each inspection unit block, and the color deviation characteristics of each inspection unit block.

[0068] A correlation analysis is performed on the various environmental features in the environmental feature set of each inspection unit block. The relationship between light distribution and noise level in each inspection unit block is specifically analyzed. At the same time, the relationship between color balance and light intensity distribution is analyzed to construct an environmental impact matrix for each inspection unit block. The environmental impact matrix describes the interaction between different environmental factors within the inspection unit block.

[0069] According to the environmental impact matrix, a random interference feature is generated for each inspection unit block, specifically: each element in the environmental impact matrix is ​​weighted combined according to a predefined weighting algorithm, and the result of the weighted combination is the random interference feature corresponding to each inspection unit block.

[0070] For each inspection unit block, a mapping relationship between the random interference characteristics of the inspection unit block and the sub-weight information is established, and the mapping relationship is based on a pre-constructed rule base, and the rule base is obtained by analyzing the prior photovoltaic power station inspection data and training the machine learning algorithm. In the rule base, there are corresponding adjustment rules between the random interference characteristics and the sub-weight information of the inspection unit block, thereby adjusting the sub-weight information of the inspection unit block according to the adjustment rule.

[0071] According to the positional relationship and logical correlation of each inspection unit block in the sample UAV inspection image, the adjusted sub-weight information of all inspection unit blocks is fused to generate fused weight information, and the fused weight information is converted into the target optimization confidence of each inspection unit block by setting a conversion function. The conversion function is used to map the fused weight information to an interval between 0 and 1. The value in this interval represents the target optimization confidence of each inspection unit block. The higher the confidence, the more the inspection unit block needs to optimize the image semantic features.

[0072] In this embodiment, first, the server extracts environmental features related to each inspection unit block from the overall environmental data of the sample drone inspection image, thereby generating a preliminary set of environmental features for each inspection unit block. The sample drone inspection image is taken by the drone during the inspection of a large photovoltaic power station, and it contains visual information of each part of the photovoltaic power station. For each inspection unit block, the server needs to consider a variety of environmental factors. Taking the characterization features of the unevenness of light intensity in each inspection unit block as an example, in a large photovoltaic power station, due to the layout of photovoltaic panels, the shielding of surrounding buildings or terrain, the lighting conditions of different inspection unit blocks will be different. For example, some inspection unit blocks may be partially in the shadow, while the other part is exposed to sufficient sunlight, which leads to uneven light intensity. The specific manifestations of this unevenness, such as the ratio of the shadow area to the illuminated area, the gradient change of light intensity, etc., are all extracted as the characterization features of the unevenness of light intensity in the inspection unit block.

[0073] At the same time, the distribution characteristics of Gaussian noise in each inspection unit block are also an important part of the environmental characteristics. In the process of the image acquisition device acquiring the sample drone inspection image, Gaussian noise will inevitably be introduced due to the physical characteristics of the sensor and the interference of the surrounding electromagnetic environment. The distribution of Gaussian noise is different for different inspection unit blocks. For example, the inspection unit block near the edge of the image may have a higher intensity of Gaussian noise and a more uneven distribution of noise due to signal transmission loss or sensitivity changes at the edge of the sensor. The server determines the mean, variance, and distribution shape of the Gaussian noise in each inspection unit block by analyzing the image data, so as to characterize the distribution characteristics of Gaussian noise in each inspection unit block.

[0074] The color deviation characteristics of each inspection unit block should not be ignored. During the long-term operation of photovoltaic panels in photovoltaic power stations, they may show color deviation in the sample drone inspection images due to surface pollution, aging or local failures. For example, if there is dust accumulation on the surface of the photovoltaic panel in a certain inspection unit block, the color of that part in the image may become dim; and if there is a local short circuit or heating phenomenon, the color of the photovoltaic panel may change subtly. The server analyzes the image color information to determine the color deviation value of each inspection unit block relative to the normal state, such as the color offset of the red, green, and blue channels, and collects this information as the color deviation characteristics of each inspection unit block to form a preliminary environmental feature set for each inspection unit block.

[0075] Next, the server performs a correlation analysis on each environmental feature in the environmental feature set of each inspection unit block. In this process, the relationship between the light distribution and the noise level in each inspection unit block is an important analysis content. In some cases, areas with uneven light intensity are often accompanied by higher noise levels. For example, in shadowed areas with weak light, the image sensor may automatically adjust the gain in order to obtain sufficient brightness information, which easily introduces more Gaussian noise. The server establishes a mathematical relationship model between light intensity and noise level by analyzing a large number of sample drone inspection images. Specifically, it will count the changes in noise levels under different light intensities and determine the correlation coefficient between light intensity and noise level in different inspection unit blocks.

[0076] At the same time, the relationship between color balance and light intensity distribution is also analyzed. Different light intensities can have a significant impact on color balance. For example, in areas with strong light intensity, colors may appear more vivid, while in areas with weak light, colors may become dim. The server determines the functional relationship between color balance and light intensity distribution by analyzing the changes in color channel values ​​under different light intensities. For example, for a certain inspection unit block, when the light intensity changes from strong to weak, the value of the red channel may decrease in a certain proportion, and the values ​​of the green and blue channels will also change accordingly. Through such analysis, the server constructs an environmental impact matrix for each inspection unit block. This environmental impact matrix describes in detail the interaction between different environmental factors within the inspection unit block. For example, an element in the matrix may represent the degree of influence of light intensity non-uniformity on color deviation characteristics, while another element may represent the dependence of Gaussian noise on light intensity non-uniformity, etc.

[0077] Then, based on the constructed environmental impact matrix, the server generates random interference features for each inspection unit block. The specific operation is to weight the elements in the environmental impact matrix according to a pre-defined weighting algorithm. This weighting algorithm is determined based on a deep understanding of the environment and image characteristics of the photovoltaic power station. For example, in some cases, the impact of light intensity non-uniformity on the overall image quality may be considered more important. In this case, in the weighting algorithm, the matrix elements related to light intensity non-uniformity will be given a higher weight. Through such a weighted combination, the result obtained is the random interference feature corresponding to each inspection unit block. This random interference feature comprehensively considers the impact of multiple environmental factors such as light, noise and color on the inspection unit block.

[0078] For each inspection unit block, the server establishes a mapping relationship between the random interference features of the inspection unit block and the sub-weight information. This mapping relationship is based on a pre-built rule base, which is obtained through in-depth analysis of prior photovoltaic power station inspection data and training of machine learning algorithms. In this rule base, a large number of corresponding adjustment rules between different random interference features and sub-weight information are stored. For example, if the random interference features of an inspection unit block indicate that the light intensity is highly non-uniform, the Gaussian noise is large, and the color deviation is obvious, according to the adjustment rules in the rule base, the sub-weight information of the inspection unit block may be significantly improved. This is because in this case, the image semantic features of the inspection unit block are more disturbed and need to be optimized. The server adjusts the sub-weight information of each inspection unit block according to such adjustment rules.

[0079] Finally, according to the positional relationship and logical association of each inspection unit block in the sample drone inspection image, the server fuses the adjusted sub-weight information of all inspection unit blocks to generate fused weight information. The positional relationship of each inspection unit block in the image may affect their contribution to the overall image semantics. For example, the inspection unit block located in the center area of ​​the image may be more important to the overall judgment, while the inspection unit block located in the edge area may be relatively less important. At the same time, there is a logical association between the inspection unit blocks. For example, adjacent inspection unit blocks may physically belong to the same photovoltaic panel array, and their states may have some association. In the fusion operation, the server will consider these positional relationships and logical associations, and use appropriate fusion algorithms, such as weighted average or graph-based algorithms, to fuse the adjusted sub-weight information of all inspection unit blocks into an overall fused weight information.

[0080] Then, the server converts the fused weight information into the target optimization confidence of each inspection unit block by setting a conversion function. This set conversion function is specially designed to map the fused weight information to an interval between 0 and 1. For example, a linear mapping function or a nonlinear sigmoid function may be used. The value in this interval represents the target optimization confidence of each inspection unit block. If the target optimization confidence of a certain inspection unit block is high, close to 1, it means that the inspection unit block is subject to greater random interference and needs to optimize the image semantic features; if the target optimization confidence is low, close to 0, it means that the inspection unit block is relatively stable and does not need to optimize the image semantic features. Through such a series of operations, the server can reasonably determine the target optimization confidence of each inspection unit block according to the environmental characteristics of the sample drone inspection image and the characteristics of each inspection unit block, thereby providing a basis for the subsequent image semantic feature optimization.

[0081] In a possible implementation manner, before step S121, the method further includes:

[0082] Determine a first statistic of the inspection unit block in the sample UAV inspection image.

[0083] If the first statistic is equal to the set statistic, the sample drone inspection image is used as a candidate sample drone inspection image.

[0084] If the first statistic is smaller than the set statistic, the sample UAV inspection image is enhanced according to the set image element, and the enhanced sample UAV inspection image is used as a candidate sample UAV inspection image, wherein the statistic of the inspection unit block in the enhanced sample UAV inspection image is the set statistic.

[0085] In this embodiment, the sample drone inspection image is obtained by taking a drone when inspecting a large photovoltaic power station, and contains visual information of a specific area in the photovoltaic power station. In this sample drone inspection image, in order to facilitate subsequent analysis and processing, the image will be divided into multiple inspection unit blocks according to certain rules. These inspection unit blocks can be divided according to factors such as the layout, geographical location or function of the photovoltaic panel. For example, in a sample drone inspection image containing multiple photovoltaic panel arrays, each photovoltaic panel array or every few adjacent photovoltaic panels can be divided into an inspection unit block.

[0086] The server must first determine the first statistic of the inspection unit block in the sample drone inspection image. This first statistic may be related to the number, area, or other feature quantities related to the image structure of the inspection unit block. Assuming that this first statistic is the number of inspection unit blocks, the server calculates the number of inspection unit blocks contained in the sample drone inspection image through a pre-analysis algorithm of the image. This pre-analysis algorithm may be based on image resolution, division rules, and feature recognition techniques in the image. For example, the server identifies features such as the boundaries of photovoltaic panels and connecting lines in the image, and then determines the boundaries of the inspection unit blocks according to pre-set division rules, thereby accurately calculating the number of inspection unit blocks.

[0087] Next, the server will compare this first statistic with the set statistic. The set statistic is a standard quantity determined in advance according to the requirements of the network model or the empirical value. If the first statistic is equal to the set statistic, the server will directly use the sample drone inspection image as a candidate sample drone inspection image. For example, the statistic is set to 100 inspection unit blocks. If the server calculates that the number of inspection unit blocks in a sample drone inspection image is exactly 100, then this sample drone inspection image can be directly used as a candidate sample drone inspection image, and the subsequent step of using the image semantic extraction unit to perform image semantic representation can be directly entered.

[0088] However, if the first statistic is less than the set statistic, the server needs to enhance the sample drone inspection image according to the set image element, and use the enhanced sample drone inspection image as a candidate sample drone inspection image, where the statistic of the inspection unit blocks in the enhanced sample drone inspection image is the set statistic. For example, suppose the set statistic is 100 inspection unit blocks, and the number of inspection unit blocks calculated in a sample drone inspection image is only 80. At this time, the server will enhance the sample drone inspection image according to the set image element.

[0089] Setting image elements may include some additional image information or processing rules. During the enhancement process, the server may use a variety of technical means. One possible way is to interpolate the image. Since the number of inspection unit blocks is small, it may be due to the low image resolution or the incomplete area covered by the image. The server can use the interpolation algorithm to increase the pixel information in the image so that the image can be divided into more inspection unit blocks. For example, the server uses a bilinear interpolation algorithm to calculate the value of new pixels based on the values ​​of known pixels, thereby increasing the detailed information of the image without changing the overall characteristics of the image, thereby increasing the number of inspection unit blocks.

[0090] In addition, the server may also add some virtual inspection unit block elements based on the layout knowledge or prior information of the photovoltaic power station. For example, if it is known that the layout of photovoltaic panels in a certain area is regular, but it is not fully displayed in the sample drone inspection image due to the shooting angle or other reasons, the server can add some virtual elements representing the inspection unit blocks in the image based on the existing photovoltaic panel information and layout rules, so that the number of inspection unit blocks reaches the set statistic.

[0091] After performing these enhancement processing operations, the server obtains the enhanced sample drone inspection image, in which the statistics of the inspection unit blocks in the image have reached the set statistics, and then the server uses the enhanced sample drone inspection image as a candidate sample drone inspection image so that it can be subsequently represented by the image semantic extraction unit of the initialized first image classification network. Through such a preprocessing step, it can be ensured that the sample drone inspection image entering the image semantic representation step meets the requirements of the network model in terms of structure or feature quantity, thereby improving the accuracy and effectiveness of subsequent image semantic representation and abnormal diagnosis of the entire photovoltaic power station.

[0092] This preprocessing operation of the sample drone inspection image is an important part of the entire photovoltaic power station abnormality diagnosis process. It is not only to meet the structural requirements of the network model for the input image, but also to accurately extract the semantic information in the image in a complex photovoltaic power station environment, and then accurately judge the abnormal state of the power station. Because in the actual photovoltaic power station inspection, the quality and structure of the sample drone inspection image may be affected by many factors, such as the flight altitude of the drone, the shooting angle, the weather conditions, etc. Through such a preprocessing step, the influence of these adverse factors can be overcome to a certain extent, and the robustness and accuracy of the entire diagnosis system can be improved.

[0093] For example, the sample drone inspection images taken by drones may vary greatly under different weather conditions. On cloudy days, insufficient light may result in low contrast in the image, making the boundaries and details of some photovoltaic panels unclear, thus affecting the division of inspection unit blocks and the calculation of statistics. Through the above-mentioned enhancement processing steps, even under such less than ideal shooting conditions, the sample drone inspection images can meet the requirements of subsequent processing. Similarly, the flight altitude and shooting angle of the drone will also have an impact on the image. If the flight altitude is too high, the captured image may cover a larger area, but the image may have less detailed information, resulting in a smaller number of inspection unit blocks; if the shooting angle is not good, some photovoltaic panels may not be fully displayed in the image, which will also affect the division of inspection unit blocks. Through the enhancement processing based on the set image elements, these defects can be compensated to a certain extent, so that the sample drone inspection images can better adapt to the subsequent image semantic representation and abnormal diagnosis operations.

[0094] In a possible implementation manner, the step of determining the first network learning weight information includes:

[0095] Based on the inspection path characteristics of each inspection unit block in multiple sample UAV inspection images, the attention weight of the inspection unit block of each inspection node is analyzed.

[0096] Based on the attention weights of the inspection unit blocks of each inspection node in each of the sample UAV inspection images, each first inspection node is determined.

[0097] The first network learning weight information is determined based on the sub-weight information of the inspection unit blocks of each of the first inspection nodes.

[0098] In this embodiment, the sample drone inspection images are taken by drones during the inspection of large photovoltaic power stations. These sample drone inspection images cover the status information of different areas of the photovoltaic power station. In these sample drone inspection images, each inspection unit block has its own specific inspection path characteristics. The inspection path characteristics are characteristics related to the flight trajectory of the drone during inspection, the shooting order, and the access order of each inspection unit block. For example, in a sample drone inspection image, the drone inspects and shoots the photovoltaic panel array in the order from left to right and from top to bottom. Then the inspection unit block located in the upper left corner of the image may be inspected earlier, while the inspection unit block located in the lower right corner may be inspected later. This inspection order constitutes part of the inspection path characteristics of each inspection unit block.

[0099] Based on the inspection path characteristics of each inspection unit block in these multiple sample drone inspection images, the server begins to analyze the attention weight of the inspection unit block of each inspection node. The inspection node here can be understood as a specific inspection point or moment in the entire inspection process, and the attention weight of the inspection unit block corresponding to each inspection node reflects the importance of the inspection unit block at this specific moment or inspection point to the entire inspection task.

[0100] To analyze this attention weight, the server considers a variety of factors. First, from the inspection path characteristics, if an inspection unit block is in a critical position on the inspection path in multiple sample drone inspection images, such as a transition area connecting different photovoltaic panel array areas, or near an area prone to failure, then the attention weight of this inspection unit block at the corresponding inspection node may be higher. This is because the status of the inspection unit blocks at these locations may be more important for judging the operating status of the entire photovoltaic power station.

[0101] In addition, the server will also consider the characteristics of the inspection unit block itself. For example, if a certain inspection unit block contains a large number of photovoltaic panels, or its corresponding photovoltaic panels are in the core power generation area of ​​the power station, then at the corresponding inspection node, the attention weight of this inspection unit block will also be increased. The server analyzes a large number of sample drone inspection images and comprehensively considers these factors to determine the attention weight of the inspection unit block of each inspection node.

[0102] Based on the attention weights of the inspection unit blocks of each inspection node in each sample drone inspection image, the server then determines each first inspection node. The first inspection node is a node that has special meaning or importance in the entire inspection process. The process of determining the first inspection node is a comprehensive consideration process. The server will set some standards or thresholds. When the attention weight of the inspection unit block of a certain inspection node exceeds this threshold, the inspection node will be determined as the first inspection node.

[0103] For example, in the inspection of a large photovoltaic power station, the server analyzed multiple sample drone inspection images and found that some inspection nodes corresponded to inspection unit blocks located near the key equipment of the power station, and these inspection unit blocks had high attention weights in multiple sample drone inspection images. If the threshold is set to a certain value, when the attention weights of these inspection nodes exceed this threshold, the server will determine these inspection nodes as the first inspection nodes.

[0104] Finally, based on the sub-weight information of the inspection unit blocks of each first inspection node, the server determines the first network learning weight information. The sub-weight information of the inspection unit blocks of the first inspection node is the weight information related to each inspection unit block, which reflects the relative importance of this inspection unit block in the network learning process.

[0105] The server integrates and processes the sub-weight information of the inspection unit blocks of each first inspection node to determine the first network learning weight information. This integration and processing process may involve a variety of mathematical operations and logical judgments. For example, the server may perform weighted summation or averaging operations on the sub-weight information of the inspection unit blocks corresponding to each first inspection node according to the importance of each first inspection node. If a first inspection node is considered to be very critical in the entire inspection process, then the sub-weight information of the inspection unit blocks of this first inspection node may be given a higher weight when determining the first network learning weight information.

[0106] In this way, the server starts from the inspection path features of each inspection unit block in multiple sample drone inspection images, gradually analyzes and determines the attention weight of the inspection unit block of each inspection node, and then determines each first inspection node, and finally determines the first network learning weight information based on the sub-weight information of the inspection unit block of the first inspection node. This first network learning weight information will play an important role in the subsequent network learning and optimization process, for example, it will play a key decision-making basis in determining whether to optimize the first image semantic features of the inspection unit block, thereby helping to improve the accuracy and effectiveness of the entire photovoltaic power station abnormality diagnosis network.

[0107] In the actual inspection scenario of photovoltaic power stations, due to the large scale and complex structure of the power stations, the roles and problems that photovoltaic panels in different areas have in the power generation process are also different. By determining the first network learning weight information based on factors such as inspection path characteristics and attention weights, it is possible to better adapt to this complex situation. For example, in a large photovoltaic power station with multiple sub-station areas, the photovoltaic panel area close to the substation may have a higher attention weight during the inspection process because it is directly related to power transmission and conversion equipment, and the corresponding first inspection node is also easier to determine the inspection time near this area. The sub-weight information of the inspection unit blocks in these areas will also occupy an important position in determining the first network learning weight information, so that the entire network learning process can focus more specifically on the image features of these key areas and improve the ability to diagnose abnormal conditions of photovoltaic power stations.

[0108] In a possible implementation, the method further includes:

[0109] A power station knowledge label of the large-scale photovoltaic power station is determined.

[0110] If the power station knowledge tag is the first knowledge tag, the one or more image semantic coding nodes include each image semantic coding node in the plurality of sequentially connected image semantic coding nodes.

[0111] If the power station knowledge tag is not the first knowledge tag, the one or more image semantic coding nodes include some image semantic coding nodes among the plurality of sequentially connected image semantic coding nodes.

[0112] In this embodiment, in the entire process of processing inspection data for large photovoltaic power stations, the server needs to determine the power station knowledge label of the large photovoltaic power station. This power station knowledge label is a comprehensive identification of various aspects of knowledge such as the overall characteristics, operating status, and equipment type of the large photovoltaic power station. For example, a large photovoltaic power station may be assigned different power station knowledge labels due to factors such as the type of photovoltaic panels it uses, the type of inverter installed, the geographical environment it is in (such as whether it is located at high altitude, strong wind and sand area, etc.), or the operating years of the power station.

[0113] Assume that a large photovoltaic power station A uses a specific model of high-efficiency photovoltaic panels, has a new inverter installed, is located in a coastal area, and has a short operating life. The server will integrate this information and determine the power station knowledge label of this power station by querying a pre-established power station knowledge database or performing a specific algorithm analysis. This label contains all the feature information related to the power station, which will serve as an important basis for subsequent processing of the sample drone inspection images.

[0114] After the power plant knowledge tag is determined, the server will decide the usage of the image semantic coding node when processing the sample drone inspection image based on this tag. If the power plant knowledge tag is the first knowledge tag, it means that the power plant has certain specific attributes or is in a certain specific state. At this time, the one or more image semantic coding nodes will include each image semantic coding node in a plurality of sequentially connected image semantic coding nodes.

[0115] Take the sample drone inspection images as an example. These images are obtained by inspecting different areas of a large photovoltaic power station A. When the power station knowledge label of power station A is the first knowledge label, the server will enable all image semantic coding nodes when processing the sample drone inspection images using the image semantic extraction unit of the first image classification network. Each image semantic coding node has its specific function, which may be to extract features at different levels of the image or to represent semantic information in different aspects. For example, the first image semantic coding node may focus on extracting basic texture features of the image, the second node may focus on analyzing the shape features of objects in the image, and the last node may synthesize the features extracted by the previous nodes to obtain a higher-level semantic representation. Due to the special properties of power station A (identified by the first knowledge label), the server believes that it is necessary to use all these image semantic coding nodes to comprehensively and meticulously process the sample drone inspection images in order to accurately obtain the semantic information in the image and then determine whether the power station is in an abnormal state.

[0116] However, if the power station knowledge label is not the first knowledge label, this indicates that the power station has other different characteristics or states. In this case, one or more image semantic coding nodes will only include some of the image semantic coding nodes in a plurality of sequentially connected image semantic coding nodes. For example, for another large photovoltaic power station B, its power station knowledge label is different from that of power station A, which may be due to factors such as its photovoltaic panel model is relatively common, its operating years are longer, or its environment is relatively stable. For the sample drone inspection image of power station B, the server determines that it is not necessary to enable all image semantic coding nodes based on its power station knowledge label. It may be necessary to enable only the first few image semantic coding nodes, which are sufficient to effectively represent the sample drone inspection image semantically. For example, only enabling the first three image semantic coding nodes can extract enough information to judge the common abnormal state of power station B, because the characteristics of power station B determine that the processing of the more complex or special-case image semantic coding nodes is not required.

[0117] By determining the usage scope of image semantic coding nodes based on the knowledge tags of the power station, the server can process sample drone inspection images of large photovoltaic power stations with different characteristics more efficiently and accurately, thereby improving the accuracy and efficiency of abnormal state diagnosis of power stations. This approach fully considers the diversity and complexity of large photovoltaic power stations, performs targeted operations based on the knowledge tags of the power station itself, avoids unnecessary waste of computing resources, and can better adapt to the needs of different power stations, providing strong support for the effective operation and maintenance of photovoltaic power stations.

[0118] In a possible implementation, step S121 includes:

[0119] Step S1211: Generate a corresponding guide image element based on the sample drone inspection image, wherein the guide image element represents that the machine learning network generates a corresponding power plant abnormal status result based on the sample drone inspection image.

[0120] Step S1212: Using the image semantic extraction unit of the first image classification network to perform image semantic representation on the guide image element.

[0121] In this embodiment, when the image semantic extraction unit of the initialized first image classification network is used to perform image semantic representation on the sample drone inspection image, the server generates a corresponding guide image element based on the sample drone inspection image. The sample drone inspection image is obtained by a drone inspection of a large photovoltaic power station, which contains visual information about the blocks of each inspection unit of the photovoltaic power station. Based on this sample drone inspection image, the server generates a guide image element through specific algorithms and rules. This guide image element is a special information representation, which represents that the machine learning network generates the corresponding power station abnormal status result based on the sample drone inspection image.

[0122] For example, in a large photovoltaic power station, the sample drone inspection image may show the array layout, surface condition, and connection lines of the photovoltaic panels. When generating the guide image element, the server will comprehensively consider the potential connection between this information and the abnormal state of the power station. If a photovoltaic panel area in the sample drone inspection image has abnormal color or shadows on the surface, the server will encode it in a specific way in the guide image element. This encoding method can reflect the association between these visual features and possible abnormal power station conditions (such as reduced photovoltaic panel power generation efficiency or local overheating).

[0123] Then, the server uses the image semantic extraction unit of the first image classification network to perform image semantic representation on the guide image element. This image semantic extraction unit is a module specially designed to extract semantic information from image-related elements. When processing the guide image element, the image semantic extraction unit operates according to its internal structure and algorithm.

[0124] Assume that the image semantic extraction unit contains multiple levels of feature extraction and semantic analysis mechanisms. When the guide image element is input into the image semantic extraction unit, it first extracts the basic visual features in the guide image element at a lower level, such as edge information, color distribution, etc. For example, for the abnormal color information of the photovoltaic panel encoded in the guide image element, the image semantic extraction unit can identify the distribution range of this color in the image and the boundary with the surrounding normal color area.

[0125] As the processing progresses, at a higher level, the image semantic extraction unit will combine these basic visual features and analyze the relationship between them to obtain more advanced semantic information. Taking the photovoltaic panel color anomaly as an example, it not only identifies the color anomaly area, but also determines whether this color anomaly may cause an abnormal state of the power station based on the relationship between the color anomaly area and other parts of the photovoltaic panel (such as connecting lines, surrounding photovoltaic panels, etc.), such as whether it will affect the current transmission or the power output of the entire photovoltaic panel array.

[0126] Throughout the process, the image semantic extraction unit will also use its pre-trained model parameters and algorithm rules. These parameters and rules are trained based on a large amount of sample data (including previously processed sample drone inspection images and their corresponding guide image elements and power plant abnormal status results). Through this pre-trained knowledge, the image semantic extraction unit can more accurately represent the guide image elements and convert them into a semantic representation that can be understood and processed by subsequent modules (such as the image semantic restoration unit), so that the power plant abnormal status results corresponding to the sample drone inspection images can be accurately generated.

[0127] This method of generating guiding image elements and representing them in image semantics helps to more effectively use machine learning networks to analyze sample drone inspection images in complex photovoltaic power station inspection scenarios, and improve the accuracy and reliability of power station abnormal status diagnosis. This is because the guiding image elements can establish a more direct connection between the key information in the sample drone inspection images and the power station abnormal status results, and the image semantic extraction unit's processing of the guiding image elements further mines the semantic information behind this connection, thereby providing a more valuable information basis for the entire power station abnormality diagnosis system.

[0128] Figure 2 The hardware structure of the unmanned aerial vehicle inspection system 100 applied to a large photovoltaic power station for implementing the unmanned aerial vehicle inspection method applied to a large photovoltaic power station provided by an embodiment of the present invention is shown. Figure 2 As shown, the drone inspection system 100 applied to a large photovoltaic power station may include a processor 110 , a machine-readable storage medium 120 , a bus 130 , and a communication unit 140 .

[0129] The machine-readable storage medium 120 may store data and / or instructions. In some embodiments, the machine-readable storage medium 120 may store data obtained from an external terminal. In some embodiments, the machine-readable storage medium 120 may store data and / or instructions used by the drone inspection system 100 for a large photovoltaic power station to execute or use to complete the exemplary method described in the present invention.

[0130] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the drone inspection method applied to a large photovoltaic power station in the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected via the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0131] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned drone inspection system 100 applied to large-scale photovoltaic power stations. The implementation principles and technical effects are similar, and this embodiment will not be repeated here.

[0132] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer executable instructions are stored. When a processor executes the computer executable instructions, the drone inspection method applied to a large photovoltaic power station as described above is implemented.

[0133] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, various features are sometimes combined into one embodiment, drawing or description thereof.

Claims

1. A drone inspection method for a large photovoltaic power station, characterized in that: The method comprises: Acquire multiple sample drone inspection data carrying status annotation data of a large photovoltaic power station, each of the sample drone inspection data includes a sample drone inspection image, and the status annotation data of each sample drone inspection data is a sample power station abnormal status label corresponding to the sample drone inspection image in the sample drone inspection data; Based on multiple sample drone inspection data, the machine learning network is optimized through cyclic network parameter optimization to generate an optimized target photovoltaic power station abnormality diagnosis network; Obtain the unmanned aerial vehicle inspection image to be diagnosed of the large photovoltaic power station, and use the optimized target photovoltaic power station abnormality diagnosis network to generate the abnormal status result of the power station corresponding to the unmanned aerial vehicle inspection image to be diagnosed; The step of optimizing the cyclic network parameters includes: For each sample drone inspection image, use the image semantic extraction unit of the initialized first image classification network to perform image semantic representation on the sample drone inspection image, generate first image semantic features of each inspection unit block in the sample drone inspection image, determine each first inspection unit block to be optimized in each inspection unit block of the sample drone inspection image based on the first network learning weight information, and optimize the first image semantic features of each of the first inspection unit blocks based on the second network learning weight information to generate optimized second image semantic features of each of the first inspection unit blocks; For each sample drone inspection image, based on the target image semantic features of the sample drone inspection image, the image semantic restoration unit of the first image classification network is used to perform image semantic restoration to generate a prediction result of the abnormal state of the power plant corresponding to the sample drone inspection image, wherein the target image semantic features of the sample drone inspection image are generated based on the second image semantic features of each of the first inspection unit blocks in the sample drone inspection image and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks; Based on the power plant abnormal state prediction results and state annotation data corresponding to each of the sample drone inspection images, the first network learning error is determined, and according to the first network learning error, the first network learning weight information and the second network learning weight information are optimized to generate the first network learning weight information and the second network learning weight information corresponding to the next round of network parameter optimization.

2. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The image semantic extraction unit includes a plurality of sequentially connected image semantic coding nodes; the first network learning weight information includes node parameter information corresponding to one or more image semantic coding nodes; For each sample drone inspection image, the steps for generating the abnormal state prediction result of the power station corresponding to the sample drone inspection image include: The sample UAV inspection image is loaded into the image semantic extraction unit, and the image semantic extraction unit is used to perform image semantic representation on the sample UAV inspection image to generate a first image semantic feature corresponding to each image semantic coding node of each inspection unit block in the sample UAV inspection image; For each of the one or more image semantic coding nodes, determine the first inspection unit blocks to be optimized in the inspection unit blocks of the sample UAV inspection image according to the node parameter information corresponding to the image semantic coding node, optimize the first image semantic features of each of the first inspection unit blocks according to the second network learning weight information, generate the second image semantic features of each of the first inspection unit blocks, use the second image semantic features of each of the first inspection unit blocks and the first image semantic features of other inspection unit blocks except each of the first inspection unit blocks as the target image semantic features of the image semantic coding node, and use the target image semantic features of the image semantic coding node as the loading data of the next image semantic coding node; For each image semantic coding node except the one or more image semantic coding nodes, the image semantic coding node is used to obtain the first image semantic feature of each inspection unit block in the sample UAV inspection image as the target image semantic feature of the image semantic coding node, and the target image semantic feature of the image semantic coding node is used as the loading data of the next image semantic coding node; According to the target image semantic features of the last image semantic coding node of the image semantic extraction unit, the image semantic restoration unit is used to perform image semantic restoration to generate a power plant abnormal state prediction result corresponding to the sample UAV inspection image.

3. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The first network learning weight information includes sub-weight information of each inspection unit block in the sample UAV inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block; The optimizing the first network learning weight information and the second network learning weight information according to the first network learning error includes: Determining a second network learning error based on fusion weight information of sub-weight information of each inspection unit block in each of the sample UAV inspection images; The first network learning weight information and the second network learning weight information are optimized according to the first network learning error and the second network learning error.

4. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The first network learning weight information includes sub-weight information of each inspection unit block in the sample UAV inspection image, and the sub-weight information of an inspection unit block is used to determine whether to optimize the first image semantic feature of the inspection unit block; The step of determining each first inspection unit block to be optimized in each inspection unit block of the sample drone inspection image includes: Determine the random interference features corresponding to each inspection unit block in the sample UAV inspection image, and optimize the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, and generate the target optimization confidence of each inspection unit block; Based on the target optimization confidence of each of the inspection unit blocks, each first inspection unit block to be optimized is determined.

5. The drone inspection method for large photovoltaic power stations according to claim 4 is characterized in that: The step of determining the random interference features corresponding to each inspection unit block in the sample drone inspection image, optimizing the sub-weight information of each inspection unit block according to the random interference features corresponding to each inspection unit block, and generating the target optimization confidence of each inspection unit block includes: Extract environmental features related to each inspection unit block from the overall environmental data of the sample UAV inspection image, and generate a preliminary set of environmental features for each inspection unit block, wherein the environmental features include the characterization features of the unevenness of light intensity in each inspection unit block, the distribution features of Gaussian noise in each inspection unit block, and the color deviation features of each inspection unit block; Conduct correlation analysis on each environmental feature set of each inspection unit block, specifically analyze the relationship between light distribution and noise level in each inspection unit block, and analyze the relationship between color balance and light intensity distribution, and construct an environmental impact matrix for each inspection unit block. The environmental impact matrix describes the interaction between different environmental factors in the inspection unit block. According to the environmental impact matrix, a random interference feature is generated for each inspection unit block, specifically: each element in the environmental impact matrix is ​​weighted and combined according to a predefined weighting algorithm, and the result of the weighted combination is the random interference feature corresponding to each inspection unit block; For each inspection unit block, a mapping relationship between the random interference feature and the sub-weight information of the inspection unit block is established, wherein the mapping relationship is based on a pre-constructed rule base, wherein the rule base is obtained by analyzing the prior photovoltaic power station inspection data and training the machine learning algorithm, wherein in the rule base, there are corresponding adjustment rules for the random interference feature and the sub-weight information of the inspection unit block, thereby adjusting the sub-weight information of the inspection unit block according to the adjustment rule; According to the positional relationship and logical correlation of each inspection unit block in the sample UAV inspection image, the adjusted sub-weight information of all inspection unit blocks is fused to generate fused weight information, and the fused weight information is converted into the target optimization confidence of each inspection unit block by setting a conversion function. The conversion function is used to map the fused weight information to an interval between 0 and 1. The value in this interval represents the target optimization confidence of each inspection unit block. The higher the confidence, the more the inspection unit block needs to optimize the image semantic features.

6. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: Before the image semantic extraction unit of the initialized first image classification network performs image semantic representation on the sample drone inspection image, the method further includes: Determine a first statistic of the inspection unit block in the sample UAV inspection image; If the first statistic is equal to the set statistic, the sample drone inspection image is used as a candidate sample drone inspection image; If the first statistic is smaller than the set statistic, the sample UAV inspection image is enhanced according to the set image element, and the enhanced sample UAV inspection image is used as a candidate sample UAV inspection image, wherein the statistic of the inspection unit block in the enhanced sample UAV inspection image is the set statistic.

7. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The step of determining the first network learning weight information includes: Based on the inspection path characteristics of each inspection unit block in multiple sample drone inspection images, the attention weight of the inspection unit block of each inspection node is analyzed; Determine each first inspection node based on the attention weight of the inspection unit block of each inspection node in each of the sample drone inspection images; The first network learning weight information is determined based on the sub-weight information of the inspection unit blocks of each of the first inspection nodes.

8. The drone inspection method for large photovoltaic power stations according to claim 2 is characterized in that: The method further comprises: Determining a power station knowledge tag of the large photovoltaic power station; If the power station knowledge tag is the first knowledge tag, the one or more image semantic coding nodes include each image semantic coding node in the plurality of sequentially connected image semantic coding nodes; If the power station knowledge tag is not the first knowledge tag, the one or more image semantic coding nodes include some image semantic coding nodes among the plurality of sequentially connected image semantic coding nodes.

9. The drone inspection method for large photovoltaic power stations according to claim 1 is characterized in that: The image semantic extraction unit using the initialized first image classification network performs image semantic representation on the sample drone inspection image, including: Generate a corresponding guide image element based on the sample drone inspection image, wherein the guide image element represents that the machine learning network generates a corresponding power station abnormal state result based on the sample drone inspection image; The image semantics extraction unit of the first image classification network is used to perform image semantic representation on the guide image element.

10. A drone inspection system for large photovoltaic power stations, characterized in that: The drone inspection system applied to large-scale photovoltaic power stations includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the drone inspection method applied to large-scale photovoltaic power stations as described in any one of claims 1 to 9 above.

Images