Photovoltaic inspection intelligent decision-making method and device based on multi-modal data fusion and small sample learning

Abstract

The invention discloses a photovoltaic inspection intelligent decision-making method and device based on multi-modal data fusion and small sample learning. The method mainly comprises the following steps: carrying out unified preprocessing on multi-modal photovoltaic inspection data; fault identification is carried out based on multi-modal data fusion and a small sample learning model; quantitatively evaluating the influence degree of the fault on the photovoltaic system; and generating an operation and maintenance work order, tracking the state of the work order in real time, and optimizing the small sample learning model according to a maintenance result. According to the method and the device, a multi-modal data fusion and small sample learning technology is utilized, so that the fault identification capability of a small sample condition is effectively improved; and a complete operation and maintenance closed-loop mechanism is formed from the fault diagnosis result to the actual operation and maintenance decision and then to the optimization of the small sample model, so that the fault processing efficiency is improved, and the operation and maintenance cost and risk are reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for photovoltaic power plants, and in particular to an intelligent decision-making method and device for photovoltaic inspection based on multimodal data fusion and few-sample learning. Background Technology

[0002] Existing photovoltaic power plant inspection methods mainly include single-modal inspection, traditional data fusion inspection, and intelligent decision-making inspection based on traditional machine learning. The main shortcomings of these technologies are: 1. Data dimension is limited, resulting in incomplete fault identification. Single-mode devices can only cover 1-2 types of faults (such as infrared only measuring hot spots), and cannot achieve unified identification of multiple types of faults such as "hot spots, microcracks, junction box faults, and glass breakage". Multiple inspections are required to cover all fault types, which is inefficient.

[0003] 2. The data integration layer is shallow, and the information utilization rate is low. Traditional fusion technology remains at the data level and does not extract deep modal features; moreover, it does not dynamically allocate the weights of each modality (for example, the value of infrared data is much higher than that of visible light in hot spot faults), which dilutes key information and results in a fault omission rate of over 20%.

[0004] 3. Highly dependent on sample size; poor performance in scenarios with small sample sizes. Traditional machine learning models require a large number of labeled samples, but photovoltaic fault samples are scarce (especially rare faults), making it impossible to train the model effectively; existing small-sample models are not adapted to photovoltaic multimodal data and have insufficient generalization ability in small-sample scenarios.

[0005] 4. The decision-making logic is simplistic and lacks operational guidance. Existing technologies can only output "whether a fault exists" and "fault type", but cannot provide fault severity assessment (such as how much the hot spot temperature exceeds the threshold), maintenance priority (such as whether to deal with hot spot or minor microcrack first), and operation and maintenance plan (such as manual inspection vs. secondary confirmation by drone). They require further human decision-making and have not achieved a closed loop of "inspection-decision-operation and maintenance".

[0006] It is evident that existing technologies have not effectively addressed the issues of deep fusion of multimodal data, accurate diagnosis in small sample scenarios, and adaptive learning, which limits their application in complex faults (such as multi-factor coupled faults), rare faults, and dynamic operation and maintenance scenarios, making it difficult to meet the "real-time, accurate, and intelligent" operation and maintenance needs of large-scale photovoltaic power plants. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this disclosure provides an intelligent decision-making method for photovoltaic inspection based on multimodal data fusion and few-shot learning. It constructs a photovoltaic inspection intelligent decision-making system characterized by "deep data fusion, low sample requirements, and strong decision-making capabilities," specifically solving the following problems: 1. Achieve deep integration of multimodal inspection data from infrared, visible light, EL, and environmental factors (temperature, humidity, wind speed), breaking through the bottleneck of incomplete information from single-modal data and improving the coverage of fault identification; 2. Construct a small-sample learning model adapted to photovoltaic inspection scenarios to solve the model training problem caused by the scarcity of fault samples and achieve the small-sample adaptation capability of "training with only 5-20 labeled samples"; 3. Improve the accuracy of photovoltaic fault identification, ensuring that the accuracy of common fault identification (hot spots, microcracks) is ≥90%, and the accuracy of rare fault identification (backsheet aging) is ≥80%; 4. Construct an integrated logic of "fault identification - severity assessment - maintenance decision-making" to output directly implementable operation and maintenance solutions (including fault location, severity, maintenance priority, and handling suggestions) to reduce the cost of manual decision-making.

[0008] The intelligent decision-making method for photovoltaic inspection based on multimodal data fusion and few-sample learning disclosed herein mainly includes the following steps: S1 performs unified preprocessing on multimodal photovoltaic inspection data, including: data cleaning, outlier detection and processing, and data normalization; S2, fault identification based on multimodal data fusion and few-shot learning model; S3, quantitatively assess the impact of the fault on the photovoltaic system, including fault impact assessment and loss calculation. The fault impact assessment adopts a probability-based impact assessment method; the loss calculation uses machine learning algorithms to quantify the loss. S4 generates a maintenance plan based on the fault identification results and tracks the maintenance status. It also optimizes the small sample learning model based on the maintenance results.

[0009] Furthermore, in step S1: The multimodal photovoltaic inspection data includes: unstructured data, semi-structured data, and structured data from drones, ground robots, fixed sensors, and weather stations. The unstructured data includes visible light and / or infrared images, the semi-structured data includes laser point clouds, and the structured data includes time-series parameters. The data cleaning includes: removing data from deactivated equipment, deleting outliers, and filling in missing values; Outlier detection employs a combination of statistical outlier detection methods and machine learning-based outlier detection methods. Data normalization employs maximum-minimum normalization and statistically based Z-Score normalization.

[0010] Furthermore, step S2 specifically includes: A pre-trained model library was built, including: an image feature extraction model based on ResNet50 and a time series data modeling model based on Transformer. Pre-training was completed using publicly available photovoltaic fault datasets and a large amount of historical normal data. A fusion algorithm of "meta-learning + transfer learning" is introduced, using MAML as the framework, to perform meta-training on novel faults to quickly adapt to new fault features; and knowledge distillation is used to transfer the general features of the pre-trained model to the few-sample model to reduce sample dependence. The system employs an "attention mechanism + weighted voting" approach to integrate multimodal features for identification, assigning different weights to features of different modalities and outputting the fault type and confidence level.

[0011] Furthermore, step S3 specifically includes: A loss prediction model based on an XGBoost-LSTM hybrid architecture is constructed. Based on the input fault characteristics, component parameters, and meteorological data, the model outputs the average daily power generation loss of a single component. In the hybrid architecture, the XGBoost part is used to process the static characteristics of the component, and the LSTM part is used to process the dynamic meteorological data. Risk level assessment: A risk matrix is ​​constructed based on "loss amount × spread rate × repair cost" to classify the failure into several levels; Decision suggestion generation: Automatically generate operation and maintenance suggestions based on the fault risk level.

[0012] Furthermore, step S4 specifically includes: Fault information is automatically synchronized to the power plant OMS system via API interface, avoiding manual entry. The fault information includes: component ID, fault type, risk level, and decision recommendation. Generate maintenance work orders, including navigation coordinates obtained based on component GIS positioning, and fault handling operation guidelines; and track the work order status in real time; The actual repair results of maintenance work orders are collected as incremental training data for the small sample model. Every few feedback data points trigger a model optimization and fine-tuning to continuously improve recognition accuracy.

[0013] A photovoltaic inspection intelligent decision-making device based on multimodal data fusion and few-shot learning, applying the above method, mainly includes: The data preprocessing module is used to perform unified preprocessing on multimodal photovoltaic inspection data, including: data cleaning, outlier detection and processing, and data normalization. The small sample fault identification module is used for fault identification based on multimodal data fusion and small sample learning model; The fault impact quantification module is used to quantitatively assess the degree of impact of faults on photovoltaic systems. The operation and maintenance closed-loop module is used to generate operation and maintenance work orders and track the status of work orders in real time, and optimize the small sample learning model based on the maintenance results.

[0014] Compared with the prior art, the beneficial effects of this disclosure are: 1. Data Compatibility: Existing technologies suffer from inconsistent data formats from different sources and a lack of standardized fusion interfaces, resulting in data silos. The data preprocessing module disclosed in this paper establishes a multi-source data adaptation interface, supporting automatic access to more than 12 data formats. It also uses a format conversion engine to uniformly convert unstructured, semi-structured, and structured data into the JSON standard format. This design effectively integrates previously scattered and diverse data, resolving data incompatibility issues and improving data fusion efficiency by 80%, providing a comprehensive and unified data foundation for subsequent fault identification and decision-making.

[0015] 2. Regarding small-sample fault identification: Existing deep learning algorithms rely on massive amounts of labeled samples, resulting in low recognition rates when novel fault samples are scarce. This publication constructs a pre-trained model library, utilizing publicly available photovoltaic fault datasets and a large amount of historical normal data for pre-training. Simultaneously, it introduces a fusion algorithm of "meta-learning + transfer learning," using MAML as the framework for meta-training on novel faults to quickly adapt to new fault features. Furthermore, knowledge distillation transfers the general features of the pre-trained model to the small-sample model, reducing sample dependence. In addition, an "attention mechanism + weighted voting" fusion of multimodal features is employed for identification. These techniques work together to improve the novel fault identification rate from below 60% to ≥92%, and reduce the false positive rate from 15%-25% to ≤5%, effectively solving the problem of difficult small-sample fault identification.

[0016] 3. Quantification of Fault Impact: Existing systems can only identify faults but cannot quantify their impact on power generation and risk level, leading to confusion in maintenance priorities. The fault impact quantification module disclosed in this paper constructs an "XGBoost-LSTM" hybrid prediction model. Inputting fault characteristics, component parameters, and meteorological data, it can accurately output the average daily power generation loss of a single component with an error ≤3%. Simultaneously, a risk matrix is ​​constructed based on "loss amount × diffusion rate × repair cost," classifying faults into four levels and automatically generating maintenance recommendations based on the risk level. Through these technologies, a quantitative assessment of fault impact is achieved, improving the efficiency of maintenance priority decision-making by 60%.

[0017] 4. Operation and Maintenance Closed-Loop: Existing inspection data lacks automatic linkage with the power plant's operation and maintenance management system, resulting in delayed fault handling responses and a lack of model optimization feedback mechanisms. The disclosed operation and maintenance closed-loop module automatically synchronizes fault information to the power plant's OMS system via an API interface, avoiding manual data entry. It automatically generates operation and maintenance work orders and tracks their status in real time. Simultaneously, it collects the actual repair results from these work orders as incremental training data for a small-sample model, triggering model fine-tuning every 100 feedback data points. This design enables rapid fault response, reducing response time from 1-3 days to within 2 hours, and establishes a self-circulating mechanism of "inspection-decision-operation and maintenance-optimization" through continuous model optimization.

[0018] 5. Cost aspect: Due to the technical advantages of this disclosure in data compatibility, small sample fault identification, fault impact quantification and operation and maintenance closed loop, the annual operation and maintenance cost of GW-level power plants is reduced by 30% - 40%, and the power generation loss is reduced by 15% - 20%, achieving significant cost control and economic benefit improvement. Attached Figure Description

[0019] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments of this disclosure taken in conjunction with the accompanying drawings, in which the same reference numerals generally represent the same components.

[0020] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure. Detailed Implementation

[0021] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0022] This disclosure provides a photovoltaic (PV) inspection intelligent decision-making method and device based on multimodal data fusion and few-sample learning. It aims to solve the technical bottlenecks in existing PV inspection systems, such as "data incompatibility, low fault identification rate with few samples, inability to quantify fault impact, and broken operation and maintenance loop," thereby realizing the full value chain transformation of inspection data. The basic process is attached. Figure 1 As shown.

[0023] In one exemplary implementation, a photovoltaic inspection intelligent decision-making system based on multimodal data fusion and few-shot learning mainly includes: The system comprises four core units: a data preprocessing module, a small-sample fault identification module, a fault impact quantification module, and an operation and maintenance closed-loop module. These units are interconnected through standardized interfaces. Among them: The data preprocessing module is used to perform unified preprocessing on multi-source heterogeneous inspection data, including data cleaning, outlier detection and processing, and data normalization. Data cleaning includes removing data from deactivated equipment, deleting outliers, and filling in missing values. Outlier detection employs a combination of statistical outlier detection methods and machine learning-based anomaly detection methods. Data normalization uses maximum-minimum value normalization and statistical Z-score normalization.

[0024] The small-sample fault identification module employs a multimodal data fusion approach for fault identification, comprising three sub-modules: multimodal data integration and feature extraction, deep learning-based fault identification model training, and prediction. Multimodal data integration and feature extraction fuses the outputs of multiple models through ensemble learning methods, including algorithms such as Random Forest and XGBoost. The deep learning-based fault identification model uses a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to achieve in-depth mining of multimodal data.

[0025] The fault impact quantification module is used to assess the degree of impact of faults on photovoltaic systems. It includes two sub-modules: fault impact assessment and loss calculation. The fault impact assessment adopts a probability-based impact assessment method, combined with fault tree analysis and Bayesian networks; the loss calculation considers factors such as fault type, fault severity, and system operating status, and uses machine learning algorithms to quantify the loss.

[0026] The operation and maintenance closed-loop module organically combines fault diagnosis results with operation and maintenance decisions, comprising two sub-modules: fault diagnosis result analysis and operation and maintenance strategy optimization. Fault diagnosis result analysis extracts fault modes and evolution patterns through historical fault data mining and analysis, providing a reference for operation and maintenance decisions. Operation and maintenance strategy optimization employs a multi-objective optimization algorithm based on fuzzy mathematics, comprehensively considering factors such as fault impact, maintenance costs, and operation and maintenance efficiency to generate the optimal operation and maintenance plan.

[0027] The modules interact flexibly and share information through standardized interfaces, ensuring the system's modularity and scalability. The system also includes an online learning module to continuously optimize the algorithm models of each module, improving the overall system performance and prediction accuracy.

[0028] The implementation methods of each module are further explained below: 1. Data Preprocessing Module Data access layer: Sets up multi-source data adaptation interface, supports automatic access of more than 12 data formats such as UAV (visible light / infrared images), ground robot (laser scanning data), fixed sensor (current / voltage / temperature), weather station (irradiance / wind speed / temperature), etc., and uses a format conversion engine to uniformly convert unstructured data (images), semi-structured data (laser point clouds), and structured data (time series parameters) into JSON standard format; Data cleaning layer: Adaptive threshold filtering + illumination compensation algorithm is used to process image data (such as eliminating backlight and reflection, and correcting brightness fluctuations in cloudy weather); sensor drift data is removed by using the 3σ principle + sliding window anomaly detection; and a spatiotemporal alignment algorithm is used to match multi-source data from different acquisition times to the same component ID (time error ≤ 5 minutes). Feature extraction layer: Extracts SIFT features (texture), HOG features (shape) and infrared temperature gradient features from image data; extracts statistical features such as mean, variance and peak value from time series data; extracts surface flatness features from laser point cloud to form a multimodal feature matrix.

[0029] 2. Small Sample Fault Identification Module Pre-trained model library: Construct an image feature extraction model based on ResNet50 and a time series data modeling model based on Transformer, and complete pre-training using public photovoltaic fault datasets (such as PVDefectDataset) and 100,000+ historical normal data; Few-sample fine-tuning unit: Introduces a fusion algorithm of "meta-learning + transfer learning": Using MAML (model-independent meta-learning) as the framework, it performs 5-10 rounds of meta-training on novel faults (such as EVA delamination with only 20-50 samples) to quickly adapt to new fault features; at the same time, it transfers the general features of the pre-trained model to the few-sample model through knowledge distillation to reduce sample dependence; Multimodal fusion recognition unit: It adopts "attention mechanism + weighted voting" to fuse multimodal features: it assigns 60% weight to image features (focusing on visual faults), 30% weight to temporal features (focusing on electrical anomalies), and 10% weight to laser features (focusing on physical deformation), and outputs fault type (such as hidden cracks, hot spots, and loose connections in junction boxes) and confidence level (threshold ≥90% is judged as a valid fault).

[0030] 3. Fault Impact Quantification Module Loss prediction model: Construct an “XGBoost-LSTM” hybrid prediction model: Input fault characteristics (such as microcrack length, hot spot temperature difference), component parameters (such as power level, service life), and meteorological data (average irradiance over the past 30 days), and output the average daily power generation loss of a single component (error ≤ 3%). Risk Level Assessment Unit: Based on "loss amount × diffusion rate × repair cost", a risk matrix is ​​constructed to classify faults into four levels (Level I: emergency repair, such as inverter short circuit; Level II: 72-hour repair, such as large-area hot spots; Level III: routine maintenance, such as minor microcracks; Level IV: observation and tracking, such as initial EVA yellowing). Decision suggestion generation unit: Automatically generates operation and maintenance suggestions based on risk level, such as "prioritize the allocation of emergency repair personnel + spare component model" for Level I faults, and "combine with quarterly maintenance for unified handling" for Level III faults.

[0031] 4. Operation and Maintenance Closed-Loop Module OMS system interface unit: Automatically synchronizes fault information (component ID, fault type, risk level, decision recommendation) to the power plant OMS system via API interface, avoiding manual data entry; Work order management unit: Automatically generates maintenance work orders, including navigation coordinates (based on component GIS positioning), operation instructions (such as steps for replacing hidden crack components), and tracks the work order status in real time (not dispatched / processing / completed). Model optimization feedback unit: Collects the "actual repair results" of maintenance work orders (such as fault misjudgment / missed judgment marking, actual power generation loss data) as incremental training data for small sample models. Every 100 feedback data triggers a model fine-tuning to continuously improve recognition accuracy.

[0032] Application Examples In a real-world deployment of a 100MW photovoltaic power plant, the system is implemented through the following steps: 1. Data Preprocessing First, in the data preprocessing module, 12 data sources are integrated, including visible light images (JPG format) and infrared images (TIFF format) collected by UAVs, laser point cloud data from ground robots (LAS format), time-series current and voltage data from fixed sensors (CSV format), and irradiance data from weather stations (JSON format). The format conversion engine automatically converts unstructured image data to the JSON standard format (e.g., using the OpenCV library for image parsing), semi-structured laser point cloud data is converted into a JSON array of point cloud coordinates using the PCL library's point cloud processing algorithm, and structured time-series data is serialized using the Pandas library. The data cleaning layer applies adaptive threshold filtering (threshold range 0.1-0.3) and an illumination compensation algorithm (based on histogram equalization) to process image data under cloudy weather conditions; sensor drift data is filtered for outliers using the 3σ principle (σ value set to 2) and sliding window detection (window size 10 minutes); and a spatiotemporal alignment algorithm uses GPS timestamps and component IDs to match data (maximum time error ≤ 3 minutes).

[0033] 2. Small sample fault identification (1) Integration and feature extraction of multimodal data The feature extraction layer extracts SIFT features (≥100 key points), HOG features (64-dimensional histogram of oriented gradients), and infrared temperature gradient features from the image (gradient calculation based on the Sobel operator); extracts mean, variance (window size 5 minutes), and peak value (threshold 1.5 times the standard deviation) from the time-series data; extracts surface smoothness features (smoothness error ≤2mm) from the laser point cloud, and finally generates a multimodal feature matrix (256×256 dimensions).

[0034] (2) Training and prediction of fault identification model based on deep learning For EVA delamination faults (only 30 labeled samples), the pre-trained model library uses ResNet50 (ImageNet pre-trained weights) to process image data and Transformer model (hidden layer dimension 512) to process time series data. Pre-training is performed based on PVDefectDataset (containing 50,000 fault images) and historical normal data (100,000 records) (training cycle 100 rounds, learning rate 0.001).

[0035] The small-sample fine-tuning unit uses the MAML framework (5 inner loop steps, 10 outer loop steps) to perform meta-training on EVA delamination samples (task batch size 8), and simultaneously transfers ResNet50 general features to the new model through knowledge distillation (temperature parameter T=2).

[0036] The multimodal fusion recognition unit applies an attention mechanism (8 heads of multi-head attention). The weighted voting weights are set as follows: image features 60% (focusing on texture anomalies), temporal features 30% (focusing on current fluctuations), and laser features 10% (focusing on deformation). When the output fault type is "EVA delamination", the confidence level is ≥95%, and a fault judgment threshold of 90% is set (if it is lower, it is marked as pending verification).

[0037] In actual testing, the recognition rate was improved to 94% on a new fault dataset, while the false positive rate was reduced to 4%.

[0038] 3. Quantification of Fault Impact The loss prediction model employs a hybrid XGBoost-LSTM architecture: the XGBoost part (tree depth 10, learning rate 0.1) handles static features (such as component power rating and service life), while the LSTM part (64 hidden units, time step 30 days) handles dynamic meteorological data (average irradiance over the past 30 days). Inputs include fault features such as microcrack length (mm) and hotspot temperature difference (°C), and outputs the average daily power generation loss per component (e.g., a microcrack length of 5mm corresponds to a loss of 0.5 kWh / day). The model is trained using 10,000 historical fault data points, and cross-validation (5-fold) ensures an error ≤ 2.5%.

[0039] The risk level assessment unit constructs a risk matrix: loss amount (weight 0.5), diffusion rate (weight 0.3, based on the fault diffusion rate model), and repair cost (weight 0.2, such as component replacement cost of 500), and divides it into four risk levels (Level I: loss amount > 10 kWh / day, diffusion rate > 5% / day, repair cost > 1000; Level II: loss amount 5-10 kWh / day, diffusion rate 2-5% / day; Level III: loss amount 1-5 kWh / day; Level IV: loss amount < 1 kWh / day).

[0040] The decision recommendation generation unit automatically pushes operation and maintenance instructions based on the risk level (such as Level I fault recommendation "emergency repair within 24 hours + spare component model JKM385M").

[0041] 4. Closed-loop operation and maintenance The OMS system interface unit synchronizes fault information to the power plant OMS system through a RESTful API interface (JSON format) (e.g., using Apache Kafka to implement real-time data streaming). The fault data includes component ID (format PV-001), fault type (e.g., "hot spot"), risk level (e.g., "Level II"), and decision recommendations (e.g., "72-hour repair"). The work order management unit automatically generates work orders (based on component GIS positioning, with coordinate accuracy of ±0.5 meters), and the operation guide details the steps (e.g., hot spot component replacement: 1. Disconnect the inverter power supply; 2. Remove the fixing bolts; 3. Install the new component with a torque of 10Nm), and tracks the status in real time (status update frequency every 5 minutes, notifying maintenance personnel via SMS). The model optimization feedback unit collects actual repair results (such as misjudgment labels "false_positive" or actual power generation loss data). When the accumulated feedback data reaches 100, incremental training is triggered (fine-tuning period of 1 hour, learning rate of 0.0001). For example, after a misjudgment event, 20 new EVA delamination samples are added to retrain the model, and the recognition accuracy is improved by 1.2%.

[0042] In the year-round implementation of a GW-level power plant, the system deployment cost includes hardware (edge ​​computing nodes 500,000) and software (algorithm license 200,000). The operation and maintenance closed-loop module reduces fault response time to an average of 1.5 hours (actual data), and the model optimization feedback unit is updated quarterly (500 incremental data entries), reducing annual operation and maintenance costs by 35% (from 5 million to 3.25 million) and power generation loss by 18% (from 5% to 4.1%). Economic benefits are calculated through ROI (investment payback period ≤ 2 years), and the continuous operation of the system forms a closed loop of "inspection-decision-operation and maintenance-optimization" (cycle ≤ 24 hours).

[0043] In this embodiment, by fusing and integrating multimodal data, the limitation of traditional systems relying on only a single data source is overcome, and comprehensive monitoring and fault diagnosis of the photovoltaic system's operating status are achieved, improving the accuracy and comprehensiveness of fault diagnosis. Employing few-shot learning techniques improves the system's ability to identify unknown fault types and complex fault scenarios, enhances the system's adaptability and self-adaptability, and enables it to respond quickly and identify new types of faults. The fault impact quantification module enables accurate assessment of fault losses, providing a scientific basis for operation and maintenance decisions and effectively solving the problem of the inability to quantify fault impact in existing systems. By designing an operation and maintenance closed-loop module, the fault diagnosis results are combined with actual operation and maintenance decisions, forming a complete operation and maintenance closed-loop mechanism, which improves the efficiency of fault handling and reduces operation and maintenance costs and risks. By using a standardized interface design, the data interaction between modules becomes more flexible, solving the problem of data incompatibility in the existing system and improving the overall synergy of the system.

[0044] The above technical solutions are merely exemplary embodiments of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the specific embodiments of the present invention. Therefore, the methods described above are only preferred and not restrictive.

Claims

1. A photovoltaic inspection intelligent decision-making method based on multimodal data fusion and few-shot learning, characterized in that, Includes the following steps: S1 performs unified preprocessing on multimodal photovoltaic inspection data, including: data cleaning, outlier detection and processing, and data normalization; S2, fault identification based on multimodal data fusion and few-shot learning model; S3, quantitatively assess the impact of the fault on the photovoltaic system, including fault impact assessment and loss calculation. The fault impact assessment adopts a probability-based impact assessment method; the loss calculation uses machine learning algorithms to quantify the loss. S4 generates a maintenance plan based on the fault identification results and tracks the maintenance status. It also optimizes the small sample learning model based on the maintenance results.

2. The method according to claim 1, characterized in that, In step S1: The multimodal photovoltaic inspection data includes: unstructured data, semi-structured data, and structured data from drones, ground robots, fixed sensors, and weather stations. The unstructured data includes: visible light and / or infrared images; the semi-structured data includes: laser point clouds; and the structured data includes: time-series parameters. The data cleaning includes: removing data from deactivated equipment, deleting outliers, and filling in missing values; Outlier detection employs a combination of statistical outlier detection methods and machine learning-based outlier detection methods. Data normalization employs maximum-minimum normalization and statistically based Z-Score normalization.

3. The method according to claim 1, characterized in that, Step S2 specifically includes: A pre-trained model library was built, including: an image feature extraction model based on ResNet50 and a time series data modeling model based on Transformer. Pre-training was completed using publicly available photovoltaic fault datasets and a large amount of historical normal data. A fusion algorithm of "meta-learning + transfer learning" is introduced, using MAML as the framework, to perform meta-training on novel faults to quickly adapt to new fault features; and knowledge distillation is used to transfer the general features of the pre-trained model to the few-sample model to reduce sample dependence. The system employs an "attention mechanism + weighted voting" approach to integrate multimodal features for identification, assigning different weights to features of different modalities and outputting the fault type and confidence level.

4. The method according to claim 1, characterized in that, Step S3 specifically includes: A loss prediction model based on an XGBoost-LSTM hybrid architecture is constructed. Based on the input fault characteristics, component parameters, and meteorological data, the model outputs the average daily power generation loss of a single component. In the hybrid architecture, the XGBoost part is used to process the static characteristics of the component, and the LSTM part is used to process the dynamic meteorological data. Risk level assessment: A risk matrix is ​​constructed based on "loss amount × spread rate × repair cost" to classify the failure into several levels; Decision suggestion generation: Automatically generate operation and maintenance suggestions based on the fault risk level.

5. The method according to any one of claims 1-4, characterized in that, Step S4 specifically includes: Fault information is automatically synchronized to the power plant OMS system via API interface, avoiding manual entry. The fault information includes: component ID, fault type, risk level, and decision recommendation. Generate maintenance work orders, including navigation coordinates obtained based on component GIS positioning, and fault handling operation guidelines; and track the work order status in real time; The actual repair results of maintenance work orders are collected as incremental training data for the small sample model. Every few feedback data points trigger a model optimization and fine-tuning to continuously improve recognition accuracy.

6. A photovoltaic inspection intelligent decision-making device based on multimodal data fusion and few-shot learning, applying the method described in any one of claims 1-5, characterized in that, include: The data preprocessing module is used to perform unified preprocessing on multimodal photovoltaic inspection data, including: data cleaning, outlier detection and processing, and data normalization. The small sample fault identification module is used for fault identification based on multimodal data fusion and small sample learning model; The fault impact quantification module is used to quantitatively assess the degree of impact of faults on photovoltaic systems. The operation and maintenance closed-loop module is used to generate operation and maintenance work orders and track the status of work orders in real time, and optimize the small sample learning model based on the maintenance results.

Images