New energy unattended station automatic inspection method and system

Abstract

The embodiment of the invention relates to the technical field of intelligent inspection, and provides an automatic inspection method and system for a new energy unattended station, and the method comprises the steps: obtaining a real-time inspection image frame collected by an unmanned plane; performing real-time image quality evaluation on the real-time inspection image frame based on an edge calculation unit of the unmanned aerial vehicle to obtain a problem type and an image quality score; performing hierarchical intervention and image optimization on the real-time inspection image frame based on the problem type and the image quality score to obtain an optimized inspection image frame; and performing defect detection on the optimized inspection image frame to obtain a defect record. In this way, the accuracy and reliability of defect recording are improved, and a high-quality data basis and reliable technical support are provided for efficient and accurate automatic operation and maintenance of a new energy unattended station.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent inspection, in particular to a new energy unmanned station automatic inspection method and system. BACKGROUND

[0002] With the rapid development of new energy industry, a large number of unmanned or less manned stations are widely distributed, with a large number of equipment and a wide area, which poses a severe challenge to the efficiency and safety of operation and inspection. The traditional mode relying on manual inspection not only has high operation intensity and high cost, but also is easily affected by subjective factors and environmental conditions, and it is difficult to ensure the comprehensiveness of inspection and the timeliness of problem discovery. Therefore, it is an urgent need to build an automatic and intelligent inspection system to ensure the safe and stable operation of new energy stations.

[0003] At present, using unmanned aerial vehicles to carry high-definition camera equipment for automatic inspection has become the mainstream technical path. Although this method can significantly improve the data collection efficiency, the quality and usability of the inspection data still face challenges in actual application, especially in the accurate spatial mapping of defects and station assets. Specifically, the positioning coordinate system of the unmanned aerial vehicle itself is not unified with the engineering coordinate system of the station asset management, leading to difficulty in matching the data source; the slight changes in the attitude of the unmanned aerial vehicle during flight and the signal interference that may occur near large metal structures will cause cumulative errors in positioning accuracy, resulting in deviations between image positions and actual asset positions; at the same time, a single inspection image often contains a large number of assets with similar appearances, and even if a defect is identified in the center of the image, it is difficult to uniquely determine the specific equipment corresponding to it. Existing technical solutions often rely on post-processing and lack real-time verification and correction mechanisms during the inspection process, making it difficult to directly convert the inspection results into precise operation instructions, which affects the final effectiveness of automatic inspection.

[0004] Therefore, an optimized new energy unmanned station automatic inspection method is expected. SUMMARY

[0005] The present application aims to at least solve one of the problems in the prior art, and provides a new energy unmanned station automatic inspection method and system.

[0006] In one aspect of the present application, a new energy unmanned station automatic inspection method is provided, comprising:

[0007] acquiring real-time inspection image frames collected by an unmanned aerial vehicle;

[0008] performing real-time image quality assessment on the real-time inspection image frames based on an edge computing unit of the unmanned aerial vehicle to obtain a problem type and an image quality score;

[0009] based on the problem type and the image quality score, the real-time inspection image frame is intervened and optimized to obtain an optimized inspection image frame;

[0010] defect detection is performed on the optimized inspection image frame to obtain a defect record.

[0011] Optionally, the UAV-based edge computing unit performs real-time image quality evaluation on the real-time inspection image frame to obtain a problem type and an image quality score, including:

[0012] The real-time inspection image frame is preprocessed to obtain a preprocessed real-time inspection image frame;

[0013] Statistical physical feature extraction and deep semantic feature extraction are performed on the preprocessed real-time inspection image frame to obtain a statistical feature vector and a deep feature vector;

[0014] The statistical feature vector and the deep feature vector are fused and multi-task predicted to obtain the problem type and the image quality score.

[0015] Optionally, the statistical physical feature extraction and deep semantic feature extraction performed on the preprocessed real-time inspection image frame to obtain a statistical feature vector and a deep feature vector, including:

[0016] Histogram analysis is performed on the preprocessed real-time inspection image frame to obtain a histogram vector;

[0017] Based on the Laplacian operator, the preprocessed real-time inspection image frame is analyzed for sharpness to obtain a sharpness value;

[0018] Saturation analysis is performed on the preprocessed real-time inspection image frame to obtain a saturation vector;

[0019] The histogram vector, the sharpness value, and the saturation vector are concatenated to obtain the statistical feature vector.

[0020] Optionally, the statistical physical feature extraction and deep semantic feature extraction performed on the preprocessed real-time inspection image frame to obtain a statistical feature vector and a deep feature vector, including:

[0021] The preprocessed real-time inspection image frame is input into a pre-trained CNN model backbone network to obtain the deep feature vector.

[0022] Optionally, the statistical feature vector and the deep feature vector are fused and multi-task predicted to obtain the problem type and the image quality score, including:

[0023] The statistical feature vector and the deep feature vector are fused to obtain a fused feature vector;

[0024] inputting the fusion feature vector into a quality regression branch and a problem classification branch of a multi-task prediction head respectively to obtain the problem type and the image quality score.

[0025] Optionally, based on the problem type and the image quality score, performing hierarchical intervention and image optimization on the real-time inspection image frame to obtain an optimized inspection image frame, comprising:

[0026] in response to the image quality score exceeding a quality good threshold, taking the real-time inspection image frame as the optimized inspection image frame;

[0027] in response to the image quality score being between a quality medium threshold and a quality good threshold, performing image optimization based on the problem type through a retake strategy to obtain the optimized inspection image frame;

[0028] in response to the image quality score being below a quality poor threshold, performing image enhancement on the real-time inspection image frame to obtain the optimized inspection image frame.

[0029] Optionally, in response to the image quality score being between a quality medium threshold and a quality good threshold, performing image optimization based on the problem type through a retake strategy to obtain the optimized inspection image frame, comprising:

[0030] if the problem type is reflection or shadow, fine-tuning the position of the unmanned aerial vehicle to retake until the image quality score of the new image exceeds the quality good threshold to obtain the optimized inspection image frame;

[0031] if the problem type is overexposure or underexposure, triggering the camera of the unmanned aerial vehicle to perform HDR surround exposure and fuse images with different exposure compensation values to obtain the optimized inspection image frame.

[0032] Optionally, in response to the image quality score being below a quality poor threshold, performing image enhancement on the real-time inspection image frame to obtain the optimized inspection image frame, comprising:

[0033] if the problem type is fog, inputting the real-time inspection image frame into a fast de-fogging module to obtain the optimized inspection image frame.

[0034] Another aspect of the present application provides a new energy unmanned station automatic inspection system, comprising:

[0035] an image acquisition module configured to acquire a real-time inspection image frame collected by an unmanned aerial vehicle;

[0036] a real-time image quality evaluation module configured to perform real-time image quality evaluation on the real-time inspection image frame based on an edge computing unit of the unmanned aerial vehicle to obtain a problem type and an image quality score.

[0037] The hierarchical intervention and image optimization module is configured to perform hierarchical intervention and image optimization on the real-time inspection image frame based on the problem type and the image quality score to obtain an optimized inspection image frame.

[0038] The defect detection module is configured to perform defect detection on the optimized inspection image frame to obtain a defect record.

[0039] In another aspect of the present application, an electronic device is provided, comprising:

[0040] at least one processor; and,

[0041] a memory in communication with the at least one processor; wherein,

[0042] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic inspection of the new energy unattended station as described above.

[0043] Compared with the prior art, the present application intervenes from the source of data acquisition by using the edge computing unit carried by the unmanned aerial vehicle, performs real-time quality evaluation and diagnosis on each frame of image, and performs immediate intervention and optimization according to the preset strategy. In this way, the accuracy and reliability of the defect record are improved, and high-quality data basis and reliable technical support are provided for realizing efficient and accurate automatic operation and maintenance of the new energy unattended station. BRIEF DESCRIPTION OF DRAWINGS

[0044] One or more embodiments are illustrated by way of example in the drawings in which like reference numerals indicate like elements, and in which:

[0045] Figure 1 A flowchart of a new energy unattended station automatic inspection method according to an embodiment of the present application is provided.

[0046] Figure 2 A data flow diagram of a new energy unattended station automatic inspection method according to another embodiment of the present application is provided.

[0047] Figure 3 A block diagram of a new energy unattended station automatic inspection system according to another embodiment of the present application is provided. DETAILED DESCRIPTION

[0048] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, it should be understood by those skilled in the art that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the embodiments can be combined and referred to each other without contradiction.

[0049] One embodiment of the present application relates to a new energy unmanned station automatic inspection method, the flowchart is as shown in Figure 1 The steps S1 to S4 are described in detail below. Figure 2 The steps S1 to S4 are described in detail below.

[0050] S1, acquiring real-time inspection image frames collected by a UAV.

[0051] Specifically, in the operation environment of the new energy station unmanned operation, the real-time inspection image collected by the UAV can effectively solve the problem of low efficiency and limited coverage of manual inspection, and provide a high-quality data basis for automatic defect detection of station equipment.

[0052] Among them, the real-time inspection image frame refers to a single static digital image captured by a UAV deployed in an inspection task at a specific time point and spatial position for a specific inspection target in a new energy unmanned station. The UAV generally refers to an unmanned aerial vehicle equipped with a flight control system, a navigation positioning module and a high-definition image acquisition device (such as a visible light camera, an infrared thermal imager, etc.), which serves as a carrier for collecting inspection images and autonomously or remotely flies on a predetermined route to perform an inspection task.

[0053] In implementation, first, before the start of the inspection task, the flight route of the unmanned aerial vehicle is planned in advance according to the equipment layout, inspection requirements and environmental characteristics of the new energy unmanned station. The flight route is composed of a series of accurate three-dimensional space waypoints, each of which corresponds to one or a group of equipment to be inspected, and the shooting parameters at the point are preset, such as camera pitch angle, yaw angle, focal length, etc. After the start of the inspection, the unmanned aerial vehicle takes off autonomously according to the built-in flight control algorithm and navigation system (such as GPS / RTK, IMU), and strictly follows the pre-planned flight route, and monitors the position and attitude of the unmanned aerial vehicle in real time. When the unmanned aerial vehicle flies to a preset waypoint, the flight control system will accurately trigger the image acquisition device on board to execute the shooting action according to the task instruction. At this moment, the image sensor is exposed, the optical signal is converted into an electrical signal, and the internal image signal processor (ISP) converts and encodes it into a frame of digital image data. A frame of image can be formally understood as a two-dimensional or three-dimensional matrix, for example, for a gray-scale image I, the value I(x, y) at the pixel coordinate (x, y) represents the brightness of the point. For a color image, it usually contains three channels of red (R), green (G) and blue (B), represented as I(x, y) = [R(x, y), G(x, y), B(x, y)]. The image frame data is generated and immediately transmitted to the edge computing unit carried by the unmanned aerial vehicle through the internal bus or wireless link along with its metadata (such as the GPS coordinates of the shooting point, timestamp, unmanned aerial vehicle attitude angle, etc.), thereby completing a single acquisition action.

[0054] S2, the edge computing unit of the unmanned aerial vehicle performs real-time image quality evaluation on the real-time inspection image frame to obtain a problem type and an image quality score.

[0055] Specifically, the image quality collected by the unmanned aerial vehicle in the complex outdoor environment has high uncertainty. If all images (including a large number of low-quality and invalid images) are transmitted to the backend or directly used for defect analysis without screening, not only the limited storage and communication bandwidth resources of the unmanned aerial vehicle will be greatly wasted, but also the accuracy of the subsequent defect detection algorithm will be seriously affected, leading to missed detection and false detection, and ultimately the efficiency of the automated inspection will be greatly reduced. Therefore, by performing real-time image quality evaluation at the unmanned aerial vehicle end, the usability of the image can be identified and quantified immediately from the data source, providing accurate decision basis for subsequent hierarchical intervention (such as direct adoption, instruction for re-shooting or local enhancement), which is the core of improving the reliability and efficiency of the entire automated inspection system.

[0056] The edge computing unit refers to an embedded computing device carried on the unmanned aerial vehicle, having certain data processing, computing and storage capabilities, which can execute algorithm models at a location close to the data source to realize low-delay data analysis and decision-making. The problem type is a classification result, which indicates the specific reason for poor image quality, such as reflection, shadow, overexposure, underexposure, blur or fog, etc. The image quality score is a continuous numerical value that quantitatively represents the overall quality level of the image. The higher the score, the better the quality.

[0057] In specific implementation, S2 includes the following steps S21-S23.

[0058] S21, pre-processing the real-time inspection image frame to obtain a pre-processed real-time inspection image frame.

[0059] Specifically, when the real-time inspection image frame enters the edge computing unit, the edge computing unit will first perform standardization processing on it. This can include uniformly adjusting the image size to the input size required by the subsequent neural network model (for example, 224 pixels x 224 pixels), performing color space conversion (such as from YCbCr to RGB), and normalizing the pixel values (for example, scaling the pixel value range of 0-255 to the floating point number range of 0-1) to eliminate data differences under different scales and lighting conditions, laying a foundation for stable operation and accurate feature extraction of the subsequent model.

[0060] S22, statistical physical feature extraction and deep semantic feature extraction are performed on the pre-processed real-time inspection image frame to obtain a statistical feature vector and a deep feature vector.

[0061] Specifically, a single-dimensional feature cannot completely describe the quality of an image, and relying solely on statistical physical features such as sharpness or brightness may misjudge images with complex textures or specific artistic styles (such as high contrast); and relying solely on deep semantic features may ignore some underlying image flaws that affect basic usability. Therefore, this embodiment extracts features in two dimensions of statistical physics and deep semantics in parallel, building a more three-dimensional and robust image representation system that can balance the underlying physical fidelity of the image and the recognizability of the high-level content, providing comprehensive and complementary information input for subsequent feature fusion and multi-task prediction, thereby significantly improving the accuracy and reliability of image quality evaluation.

[0062] The statistical physical feature refers to an objective quantitative index describing the basic physical property of an image, which can be obtained by directly calculating the statistical distribution of image pixels, and the index is independent of the understanding of the image content, such as brightness distribution, contrast, sharpness, color saturation and the like. The deep semantic feature is a high-level abstract representation learned from an image by a deep learning model (especially a convolutional neural network), which can capture the semantic information of objects, scenes, textures and the like contained in the image, and reflects the features of the image content level.

[0063] In a specific implementation, after receiving the preprocessed real-time inspection image frame, the statistical physical feature extraction task and the deep semantic feature extraction task are synchronously started.

[0064] The statistical physical feature extraction task includes the following steps: first, histogram analysis is performed on the preprocessed real-time inspection image frame to obtain a histogram vector; in this process, the gray histogram of the image is calculated, that is, the number of pixels of each gray level (such as 0 to 255) in the image is counted, thereby forming a vector with a length of 256. The histogram vector can intuitively reflect the overall brightness distribution and contrast information of the image, which is crucial for determining whether the image has overexposure or underexposure problems. Then, sharpness analysis is performed on the preprocessed real-time inspection image frame based on a Laplacian operator to obtain a sharpness value; in this process, the image is convolved with a Laplacian kernel (such as a 3x3 convolution kernel with a center of -4 or -8 and a surrounding of 1). As a second-order differential operator, the Laplacian operator is very sensitive to edges and noise in the image. The variance or standard deviation of the pixel values of the convolved image can be used as a single numerical value to measure the sharpness of the image, that is, the sharpness value; the larger the sharpness value, the sharper the edges of the image, and the clearer the image. Further, saturation analysis is performed on the preprocessed real-time inspection image frame to obtain a saturation vector; in this process, the image is first converted from the RGB color space to the HSV (hue, saturation, lightness) or HSL (hue, saturation, brightness) color space, which is more consistent with human visual perception, and then the saturation (S) channel is extracted. By calculating the pixel value histogram or average value of the S channel, a saturation vector or value representing the color vividness of the image can be obtained. Subsequently, the histogram vector, the sharpness value and the saturation vector are concatenated to obtain a statistical feature vector. That is, by concatenating, the physical features of different dimensions are integrated into a longer and unified feature vector, which comprehensively describes the underlying physical quality of the image.

[0065] The deep semantic feature extraction task includes the following steps: inputting the preprocessed real-time inspection image frame into a pre-trained CNN (Convolutional Neural Network) model backbone network to obtain a deep feature vector. That is, a lightweight and efficient pre-trained CNN model, such as MobileNetV2, EfficientNet-Lite, or ResNet-18, is pre-deployed on the edge computing unit of the unmanned aerial vehicle. After the preprocessed real-time inspection image frame is input into the pre-trained CNN model, the pre-trained CNN model sequentially passes through multiple convolutional layers, activation functions (such as ReLU), pooling layers, and other structures in the model. In this forward propagation process, the network gradually learns high-level semantic features such as texture, components, and target objects from low-level features such as edges and colors. Finally, the activation value extracted from the end of the CNN model backbone network (for example, the position after the global average pooling layer and before the original classification head) constitutes a deep feature vector containing rich semantic information. The dimension of this vector is determined by the structure of the selected model, and the content information of the image is encoded in a highly condensed manner.

[0066] In one specific example, after the edge computing unit of the unmanned aerial vehicle receives a frame of 224 pixel x 224 pixel preprocessed photovoltaic module image, the feature extraction module immediately and in parallel performs the statistical physical feature extraction task and the deep semantic feature extraction task. In the statistical physical feature extraction process: the feature extraction module first calculates the 256-order gray histogram of the preprocessed photovoltaic module image to obtain a 256-dimensional histogram vector; at the same time, the feature extraction module applies the Laplacian operator to convolve the preprocessed photovoltaic module image and calculates the variance of the response to obtain a sharpness value of 785.4; then, the feature extraction module converts the preprocessed photovoltaic module image to HSV space and extracts the S channel to calculate the average saturation. Finally, the 256-dimensional histogram vector, the sharpness value 785.4, and the average saturation are concatenated to form the final statistical feature vector. In the deep semantic feature extraction process: the preprocessed photovoltaic module image is input into the MobileNetV2 model backbone network built-in and pre-trained on ImageNet. After a series of complex convolution and nonlinear transformation, the global average pooling layer of the MobileNetV2 model backbone network outputs a 1280-dimensional floating-point vector, which is the deep feature vector of the input preprocessed photovoltaic module image. Through the above process, the statistical feature vector and the deep feature vector of the frame of preprocessed photovoltaic module image are generated, which are two key data representations for the next stage of feature fusion and multi-task prediction.

[0067] S23, performing feature fusion on the statistical feature vector and the deep feature vector and multi-task prediction to obtain the question type and the image quality score.

[0068] Specifically, only statistical physical features or deep semantic features have their limitations, so the embodiment performs feature fusion on the statistical feature vector and the deep feature vector and multi-task prediction to effectively integrate and deeply analyze the respective independent feature information extracted from different dimensions in the previous stage, thereby generating a comprehensive and accurate judgment of the image quality. Specifically, first, by performing feature fusion on the statistical feature vector and the deep feature vector, the data describing the bottom-layer physical properties of the image can be combined with the information reflecting the high-layer semantic content to form a unified representation that is much richer and more robust than a single feature; on this basis, a multi-task prediction architecture is used to perform multi-task prediction to obtain the question type and the image quality score. This shared representation learning method can promote the model to explore the internal correlation between features and improve the prediction accuracy of the two tasks, ultimately achieving accurate, specific, and interpretable evaluation of the image quality.

[0069] Among them, multi-task prediction is a machine learning paradigm that simultaneously learns and optimizes multiple related tasks by using a shared model, and its core idea is to use the common information between tasks to improve the learning efficiency and generalization ability of each task.

[0070] In implementation, S23 first performs feature fusion on the statistical feature vector and the depth feature vector to obtain a fused feature vector. In this process, the statistical feature vector and the depth feature vector generated in the previous step are taken as inputs, and a vector concatenation is performed, i.e., the two vectors are concatenated head to tail to form a longer fused feature vector. This method completely retains all the information of the two original vectors, providing a comprehensive data basis for subsequent prediction. Then, the fused feature vector is input into the quality regression branch and the problem classification branch of the multi-task prediction head to obtain the problem type and the image quality score. The multi-task prediction head is a part of the deep neural network model located after the feature extraction backbone network, which receives the fused feature vector and distributes it to different sub-networks or branches dedicated to processing specific tasks. In this process, the multi-task prediction head internally contains two parallel processing paths, one is the quality regression branch and the other is the problem classification branch. In the quality regression branch, the fused feature vector is input into a series of fully connected layers in this branch, and after nonlinear transformation, the final output layer containing only a single neuron is used to predict the image quality score. This output layer usually uses a linear activation function or a Sigmoid activation function (if the score is normalized to 0-1) to output a continuous scalar value. During model training, the optimization goal of the quality regression branch is to minimize the regression loss between the predicted score and the true human-labeled score, such as mean square error loss. In the problem classification branch, the fused feature vector is also input into a series of fully connected layers in this branch, and the final output layer contains N neurons, where N is equal to the total number of predefined problem types (e.g., normal, glare, shadow, overexposure, underexposure, blur, etc.). A Softmax activation function is applied after the output layer to convert the output values of the N neurons into a probability distribution, where each value represents the probability that the image belongs to the corresponding problem type. During training, the optimization goal of the problem classification branch is to minimize the loss of the classification task, such as cross-entropy loss. The total loss of the multi-task prediction head is the weighted sum of the losses of the two branches, i.e., the quality regression branch and the problem classification branch, and the total loss is optimized through the backpropagation algorithm, so that the multi-task prediction head can learn both the quality scoring and problem classification tasks well.

[0071] In one specific example, after obtaining a 258-dimensional statistical feature vector and a 1280-dimensional deep feature vector of a frame of photovoltaic module image, firstly, feature fusion is performed: concatenating the two vectors, i.e., the 258-dimensional statistical feature vector and the 1280-dimensional deep feature vector, to generate a 258+1280=1538-dimensional fusion feature vector. Subsequently, the 1538-dimensional fusion feature vector is simultaneously sent to the two branches of the multi-task prediction head, i.e., the quality regression branch and the problem classification branch. In the quality regression branch, after calculation through several fully connected layers, the output layer finally outputs a floating-point number 62.7, which is the predicted image quality score. At the same time, in the problem classification branch, the 1538-dimensional fusion feature vector also passes through its exclusive fully connected layers, and the final Softmax output layer gives a probability distribution vector, for example, the probabilities on the six categories [normal, glare, shadow, overexposure, underexposure, blur] are [0.05, 0.85, 0.03, 0.02, 0.03, 0.02] respectively. Since the probability of the glare category is the highest, i.e., 0.85, it can be determined that the problem type of the frame of photovoltaic module image is glare. Finally, the evaluation result of the frame of photovoltaic module image is: the problem type is glare, and the image quality score is 62.7. This clear and specific evaluation result will directly guide the subsequent hierarchical intervention and image optimization decision.

[0072] S3, based on the problem type and the image quality score, performing hierarchical intervention and image optimization on the real-time inspection image frame to obtain an optimized inspection image frame.

[0073] Specifically, the unmanned aerial vehicle inspects the site environment which is variable, and the collected images have uneven quality. If a single processing method (e.g., all re-shooting or all enhancement) is adopted for all unqualified images, the inspection efficiency will be greatly reduced and unnecessary energy consumption and computing overhead will be increased. Therefore, according to the quality level of the image, i.e., the image quality score, and the specific reason for the quality problem, i.e., the problem type, differential and targeted intervention measures are taken. This hierarchical processing method can optimize unqualified images to meet the analysis requirements at the optimal cost (including time, computing power, unmanned aerial vehicle endurance, etc.), thereby maximizing the overall efficiency and economy of automatic inspection while ensuring the final defect detection accuracy.

[0074] Among them, hierarchical intervention and image optimization refer to dividing images into different levels according to a preset image quality score threshold, and matching different optimization processing procedures for images of each level, so as to realize reasonable allocation of resources and accurate solution of problems.

[0075] Illustratively, the hierarchical intervention and image optimization of S3 mainly include the following three cases.

[0076] The first case: in response to the image quality score exceeding the quality excellent threshold, the real-time inspection image frame is directly identified as the optimized inspection image frame. In this scenario, the edge computing unit determines that the current image quality score has reached the optimal standard and does not require any additional processing. Therefore, the original real-time inspection image frame is directly identified as the optimized inspection image frame and is immediately sent to the subsequent defect detection module for analysis, and the entire intervention and optimization process is completed with the highest efficiency.

[0077] The second case: in response to the image quality score being between the quality medium threshold and the quality excellent threshold, the image is optimized based on the problem type through a re-shooting strategy to obtain the optimized inspection image frame. In this scenario, although the image has some defects, the problem is not serious and can be fundamentally improved by adjusting the shooting conditions. At this time, a specific re-shooting strategy can be selected according to the problem type. Specifically, if the problem type is reflection or shadow, the UAV position is fine-tuned for re-shooting until the image quality score of the new image exceeds the quality excellent threshold to obtain the optimized inspection image frame. This means that the UAV can be slightly adjusted in position or attitude (e.g., slightly change the hovering position or the pitch angle of the camera) to avoid a specific angle that causes reflection or shadow by controlling the flight control or gimbal of the UAV, and then re-shoot. This process forms a closed loop, i.e., the image quality is immediately evaluated again after re-shooting, and the cycle is executed until the image quality score of the newly obtained image meets the excellent standard. If the problem type is overexposure or underexposure, the camera of the UAV is triggered to perform HDR surround exposure and fuse images with different exposure compensation values to obtain the optimized inspection image frame. In this scenario, the camera can be instructed to enable the high dynamic range (HDR) mode, quickly and continuously shoot multiple (usually three or five) images with different exposure compensation (EV) values, and then fuse these images into an optimized image that retains rich details in both bright and dark parts through in-machine algorithms or fusion algorithms on the edge computing unit.

[0078] The third case: in response to the image quality score being below the quality poor threshold, the real-time inspection image frame is image-enhanced to obtain the optimized inspection image frame. This usually means that the image is severely disturbed and it is difficult to solve the problem by re-shooting (e.g., heavy fog weather) or the cost of re-shooting is too high. At this time, a pre-set image enhancement algorithm can be called to process the current image. Specifically, if the problem type is fog, the real-time inspection image frame is input into a fast de-fogging module to obtain the optimized inspection image frame. The de-fogging module usually includes an efficient image processing algorithm (e.g., an algorithm based on the dark channel prior theory or a lightweight de-fogging neural network model) that can estimate the concentration and distribution of fog by analyzing the color and brightness distribution of the image and separate it from the original image to restore the clear scene content and generate the optimized inspection image frame.

[0079] In one specific example, assuming the preset quality excellent threshold is 90, the quality medium threshold is 60, and the quality poor threshold is 60, the evaluation result of the UAV edge computing unit for a frame of real-time inspection image is: the image quality score is 62.7, and the problem type is reflection. Therefore, first, it is judged that the quality score 62.7 is between the medium threshold 60 and the excellent threshold 90, which meets the second case of hierarchical intervention and image optimization. Then, it is checked that the problem type is reflection, so the re-shooting strategy for the reflection problem is triggered. The hierarchical intervention and image optimization module sends an instruction to the flight control system of the UAV to fine-tune the pitch angle of the camera holder to 2 degrees upward, and after adjustment, the shooting instruction is triggered, and the UAV obtains a new frame of real-time inspection image. This new frame of real-time inspection image is immediately subjected to image quality evaluation, and the result is that the image quality score of the new frame of real-time inspection image is 95.3, and the problem type is normal. At this time, the quality score 95.3 of the new frame of real-time inspection image exceeds the quality excellent threshold 90, and meets the condition of direct adoption, so the new frame of real-time inspection image is finally determined as the optimized inspection image frame.

[0080] S4, performing defect detection on the optimized inspection image frame to obtain a defect record.

[0081] Specifically, defect detection is a computer vision task that uses deep learning or traditional image processing algorithms to analyze input optimized inspection image frames at the pixel, region, or object level to automatically identify, locate, and classify various predefined device defects contained in the image. The resulting defect record is a structured data entry. A complete defect record usually includes the unique identification of the defect, the ID of the image frame it belongs to, the type of the defect (for example, for photovoltaic modules, it can be "hot spot", "hidden crack", "dirt", "obstruction", etc.), the precise location of the defect in the image (usually represented by bounding box coordinates [x_min, y_min, x_max, y_max] or mask image), the confidence score of the detection result, and associated metadata such as shooting time, UAV GPS coordinates, and asset ID being inspected, etc. By performing defect detection on the optimized inspection image frame, device faults or abnormal states can be detected in a timely manner, avoiding potential safety hazards or power generation efficiency problems caused by long manual inspection cycles.

[0082] In implementation, upon receiving an optimized inspection image frame, the defect detection algorithm is activated. The defect detection algorithm is typically a specially trained deep learning model, such as a target detection model (e.g., YOLO series, Faster R-CNN, RetinaNet, etc.) or a semantic / instance segmentation model (e.g., U-Net, Mask R-CNN, etc.). These models are pre-trained on a large image dataset containing annotations of various defects of new energy equipment, and learn to recognize the features of specific defects from pixel patterns. The specific execution process of S4 is as follows: First, the received optimized inspection image frame is pre-processed as necessary to match the input requirements of the defect detection model, which may include operations such as size scaling, normalization, etc. Subsequently, the pre-processed optimized inspection image frame is input into the defect detection model for forward propagation calculation. Taking a target detection model as an example, the pre-processed optimized inspection image frame will pass through the backbone network of the model to extract multi-level feature maps, then pass through the neck network for feature fusion, and finally pass through the head network to decode each position on the feature map, outputting a series of candidate bounding boxes, the class probability corresponding to each bounding box, and a confidence score. After the defect detection model outputs the original results, S4 also performs post-processing operations, such as applying a confidence threshold to filter out low-probability detection results, and using a non-maximum suppression (NMS) algorithm to merge overlapping bounding boxes pointing to the same target, retaining the one with the highest confidence. For each valid detection result that remains after post-processing and screening, its information can be formatted and encapsulated into a defect record together with the metadata of the image frame.

[0083] In one specific example, the hierarchical intervention is performed on a photovoltaic module image frame with a quality score of 95.3 after image optimization. The S4 starts a YOLOv5 model specially used for detecting surface defects of photovoltaic modules. The model is trained with a large number of defect samples including hot spots, hidden cracks, and stains, and has high detection accuracy. The high-quality photovoltaic module image is input into the model, and the image is inferred by the model. The model identifies a suspected hot spot in a specific area of the image. After post-processing, a detection result with a bounding box coordinate, a hot spot class, and a confidence score of 0.94 is confirmed. Subsequently, a new defect record is generated, which includes the following information: defect ID (e.g., a UUID), associated image frame ID, asset ID (e.g., "photovoltaic array A - string 3 - component 12"), defect type: "hot spot", image position, confidence: 0.94, timestamp, and accurate GPS coordinates of the UAV at the time. The complete defect record is stored in the database and can be uploaded to the backend operation and maintenance platform in real time or after the UAV returns. The operation and maintenance platform can generate a precise and pending work order to assign an operation and maintenance personnel to conduct on-site verification and maintenance. Thus, the complete automatic inspection process from image acquisition to executable operation instruction generation completes a complete closed loop.

[0084] The new energy unmanned station automatic inspection method provided by the embodiments of the present application has the advantages that, compared with the prior art, the edge computing unit carried by the UAV is used to intervene from the source of data acquisition, real-time quality evaluation and diagnosis are performed on each image frame, and immediate intervention and optimization are performed according to the preset strategy. In this way, the accuracy and reliability of the defect record are improved, and high-quality data basis and reliable technical support are provided for efficient and accurate automatic operation and maintenance of the new energy unmanned station.

[0085] Another embodiment of the present application relates to a new energy unmanned station automatic inspection system 300, as shown in Figure 3 The system includes an image acquisition module 310, a real-time image quality evaluation module 320, a hierarchical intervention and image optimization module 330, and a defect detection module 340.

[0086] The image acquisition module 310 is used to acquire real-time inspection image frames collected by the UAV.

[0087] The real-time image quality evaluation module 320 is used to perform real-time image quality evaluation on the real-time inspection image frames based on the edge computing unit of the UAV to obtain a problem type and an image quality score.

[0088] The hierarchical intervention and image optimization module 330 is used to perform hierarchical intervention and image optimization on the real-time inspection image frames based on the problem type and the image quality score to obtain an optimized inspection image frame.

[0089] The defect detection module 340 is configured to perform defect detection on the optimized inspection image frame to obtain a defect record.

[0090] The specific implementation method of the new energy unattended station automatic inspection system provided by the embodiment of the present application can be referred to the new energy unattended station automatic inspection method provided by the embodiment of the present application, which will not be described here.

[0091] The new energy unattended station automatic inspection system provided by the embodiment of the present application, compared with the prior art, by using the edge computing unit carried by the unmanned aerial vehicle, intervening from the source of data acquisition, performing real-time quality evaluation and diagnosis on each frame of image, and performing immediate intervention and optimization according to the preset strategy. In this way, the accuracy and reliability of the defect record are improved, and high-quality data basis and reliable technical support are provided for realizing efficient and accurate automatic operation and maintenance of the new energy unattended station.

[0092] The new energy unattended station automatic inspection system 300 can be implemented in various wireless terminals, such as a server with a new energy unattended station automatic inspection algorithm. In one possible implementation, the new energy unattended station automatic inspection system 300 can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the new energy unattended station automatic inspection system 300 can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the new energy unattended station automatic inspection system 300 can also be one of the many hardware modules of the wireless terminal.

[0093] The new energy unattended station automatic inspection system 300 and the wireless terminal can also be separate devices, and the new energy unattended station automatic inspection system 300 can be connected to the wireless terminal through a wired and / or wireless network, and transmit interactive information in accordance with the agreed data format.

[0094] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the new energy unattended station automatic inspection method described in the above embodiment.

[0095] The memory and the processor are connected via a bus. The bus can include any number of interconnecting buses and bridges depending on the specific application of the mobile terminal. The bus connects the various circuits of the memory and the processor together and mediates data communication among the various circuits. Since the bus is known, it will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single device or a plurality of devices, such as a plurality of receivers and transmitters, which are configured to transmit and receive data over a transmission medium. The data processed by the processor is transmitted over the wireless medium via the antenna. Further, the antenna also receives data and transmits the data to the processor.

[0096] The processor is responsible for managing the bus and general processing, which can include the execution of software stored in the memory. The processor can be implemented with one or more general purpose processors, digital signal processors, microprocessors, microcontrollers, or combinations of such devices.

[0097] Those skilled in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and various changes can be made in form and details in actual application without departing from the spirit and scope of the present application.

Claims

1. A new energy unmanned station automatic inspection method, characterized in that, The method comprises: acquiring real-time inspection image frames collected by a UAV; performing real-time image quality evaluation on the real-time inspection image frames based on an edge computing unit of the UAV to obtain a problem type and an image quality score; based on the problem type and the image quality score, performing hierarchical intervention and image optimization on the real-time inspection image frames to obtain optimized inspection image frames; performing defect detection on the optimized inspection image frames to obtain defect records.

2. The automatic inspection method of the new energy unattended station according to claim 1, characterized in that, The real-time image quality evaluation on the real-time inspection image frames based on the edge computing unit of the UAV to obtain a problem type and an image quality score comprises: performing preprocessing on the real-time inspection image frames to obtain preprocessed real-time inspection image frames; performing statistical physical feature extraction and deep semantic feature extraction on the preprocessed real-time inspection image frames to obtain a statistical feature vector and a deep feature vector; performing feature fusion and multi-task prediction on the statistical feature vector and the deep feature vector to obtain the problem type and the image quality score.

3. The new energy unattended station automatic inspection method according to claim 2, characterized in that, The statistical physical feature extraction and deep semantic feature extraction on the preprocessed real-time inspection image frames to obtain a statistical feature vector and a deep feature vector comprises: performing histogram analysis on the preprocessed real-time inspection image frames to obtain a histogram vector; performing sharpness analysis on the preprocessed real-time inspection image frames based on a Laplacian operator to obtain a sharpness value; performing saturation analysis on the preprocessed real-time inspection image frames to obtain a saturation vector; concatenating the histogram vector, the sharpness value, and the saturation vector to obtain the statistical feature vector.

4. The automatic inspection method of the new energy unattended station according to claim 2, characterized in that, The statistical physical feature extraction and deep semantic feature extraction on the preprocessed real-time inspection image frames to obtain a statistical feature vector and a deep feature vector comprises: inputting the preprocessed real-time inspection image frames into a pre-trained CNN model backbone network to obtain the deep feature vector.

5. The automatic inspection method of the new energy unattended station according to claim 2, characterized in that, The feature fusion and multi-task prediction on the statistical feature vector and the deep feature vector to obtain the problem type and the image quality score comprises: performing feature fusion on the statistical feature vector and the deep feature vector to obtain a fused feature vector; inputting the fused feature vector into a quality regression branch and a problem classification branch of a multi-task prediction head respectively to obtain the problem type and the image quality score.

6. The automatic inspection method of the new energy unattended station according to claim 1, characterized in that, The hierarchical intervention and image optimization on the real-time inspection image frames based on the problem type and the image quality score to obtain optimized inspection image frames comprises: in response to the image quality score exceeding a quality good threshold, taking the real-time inspection image frames as the optimized inspection image frames; in response to the image quality score being between a quality medium threshold and the quality good threshold, performing image optimization based on the problem type through a retake strategy to obtain the optimized inspection image frames; in response to the image quality score being below a quality poor threshold, performing image enhancement on the real-time inspection image frames to obtain the optimized inspection image frames.

7. The new energy unattended station automatic inspection method according to claim 6, characterized in that, The response to the image quality score being between the quality medium threshold and the quality excellent threshold, based on the problem type, is to perform image optimization through a retake strategy to obtain the optimized inspection image frame, including: If the problem type is reflection or shadow, fine-tune the UAV position for retake until the image quality score of the new image exceeds the quality excellent threshold to obtain the optimized inspection image frame; If the problem type is overexposure or underexposure, trigger the camera of the UAV to perform HDR surround exposure and fuse images with different exposure compensation values to obtain the optimized inspection image frame.

8. The automatic inspection method of the new energy unattended station according to claim 6, characterized in that, The response to the image quality score being below the poor quality threshold is to perform image enhancement on the real-time inspection image frame to obtain the optimized inspection image frame, including: If the problem type is fog, input the real-time inspection image frame into a fast fog removal module to obtain the optimized inspection image frame.

9. A new energy unmanned station automatic inspection system, characterized in that, The method comprises: An image acquisition module for acquiring real-time inspection image frames collected by a UAV; A real-time image quality evaluation module for evaluating the real-time inspection image frames based on the edge computing unit of the UAV to obtain a problem type and an image quality score; A hierarchical intervention and image optimization module for performing hierarchical intervention and image optimization on the real-time inspection image frames based on the problem type and the image quality score to obtain optimized inspection image frames; A defect detection module for detecting defects in the optimized inspection image frames to obtain defect records.

10. An electronic device, comprising: The method comprises: At least one processor; And A memory in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the new energy unmanned station automatic inspection method of any one of claims 1 to 8.

Images