A photovoltaic panel loss prediction method based on fusion of visual images and physical parameters

Abstract

The application discloses a photovoltaic panel loss prediction method based on fusion of visual images and physical parameters, and belongs to the technical field of electric power. The prediction method comprises the following steps: acquiring photovoltaic panel data, and configuring the obtained photovoltaic panel data to a training set and a test set after preprocessing; acquiring an abnormality detection model of the photovoltaic panel based on information in the training set; acquiring loss prediction high-dimensional features of the photovoltaic panel based on the abnormality detection model; and eliminating errors of the loss prediction high-dimensional features based on semiconductor physical constraints to obtain accurate photovoltaic panel loss prediction results. The problems that defects of the photovoltaic panel cannot be found in time and the negative influence degree caused by the defects of the photovoltaic panel cannot be accurately predicted in the prior art are solved, and thus the prediction accuracy and efficiency of the photovoltaic panel loss are improved.

Description

Technical Field

[0001] This application belongs to the field of power technology, specifically, it relates to a method for predicting photovoltaic panel losses based on the fusion of visual images and physical parameters. Background Technology

[0002] With the rapid development of the photovoltaic (PV) industry, the operation and maintenance of photovoltaic power plants face enormous challenges. PV modules can experience various anomalies during operation (such as hot spots, cracks, shading, and detachment). If these anomalies are not detected in time, they can seriously affect power generation efficiency and even lead to safety accidents. Traditional inspection methods mainly rely on manual visual inspection or electrical characteristic analysis, which have fundamental drawbacks, such as low efficiency (photovoltaic power plants typically cover a vast area and have tens of thousands of modules; relying on manual inspection of each module is not only extremely time-consuming but also cannot meet the high-frequency, full-coverage inspection needs of large-scale power plants), high cost (maintaining a large professional operation and maintenance team requires huge investment in manpower, and obtaining detailed electrical data incurs expensive hardware purchase and installation costs), high subjectivity (the identification results of anomalies highly depend on the personal experience and subjective judgment of the inspectors; differences in standards among different personnel and visual fatigue caused by long hours of work can easily lead to missed or misjudged minor defects), high safety risks (many photovoltaic power plants are built in complex geographical environments such as steep slopes, water surfaces, or building rooftops; manual on-site operations not only face the risk of falling from heights but also the threat of high-temperature environments and high-voltage electric shock), and difficulties in data management (data is often complex, fragmented, or offline and lacks precise spatial coordinate positioning, making it difficult to establish digital archives to trace the long-term health evolution trend of modules). With the rapid development of computer vision and machine learning technologies, a new technological path has been provided to solve the aforementioned operational bottlenecks. Image-based detection technology can achieve non-contact, large-area, and automated inspection, and has broad application prospects in improving operational safety and efficiency, accurately locating fault points, and effectively reducing labor costs.

[0003] Most vision-based inspection methods focus on "discovering" problems (i.e., identifying the type and location of anomalies) without directly considering the actual impact of these problems on energy production. Power loss in photovoltaic (PV) modules is typically caused by internal physical defects (such as microcracks, broken grids, and hot spots) or external environmental factors (such as dust accumulation, shading, and high temperatures). These factors ultimately manifest as an increase in the module's series resistance or a decrease in its parallel resistance, leading to a reduction in output power. While vision-based methods can detect minute cracks or hot spots on PV panels, they cannot accurately assess the current power degradation or power generation loss of the module. Therefore, monitoring the health status and assessing the loss of PV modules has become a core pain point in operation and maintenance.

[0004] Therefore, developing a real-time and accurate method to promptly detect and analyze the impact of problems on photovoltaic panels is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the difficulties in quickly analyzing photovoltaic panels and determining the extent of their impact in existing technologies, this application discloses the following technical solution:

[0006] A photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters, the prediction method comprising:

[0007] Acquire photovoltaic panel data, preprocess the obtained photovoltaic panel data, and then configure it into the training set and the test set;

[0008] Based on the information in the training set, an anomaly detection model for photovoltaic panels is obtained;

[0009] Based on the anomaly detection model, high-dimensional features for predicting the loss of photovoltaic panels are obtained;

[0010] Based on semiconductor physical constraints, error elimination is performed on the high-dimensional features of the loss prediction to obtain accurate photovoltaic panel loss prediction results.

[0011] Optionally, the step of acquiring photovoltaic panel data, preprocessing the obtained photovoltaic panel data, and configuring it into the training and testing sets includes:

[0012] The photovoltaic panel data includes photovoltaic panel image information, photovoltaic panel electrical data, and photovoltaic panel environmental parameters;

[0013] Establish the correspondence between the acquired photovoltaic panel image information, the synchronously acquired photovoltaic panel electrical data, and the photovoltaic panel environmental parameters;

[0014] Visible light and infrared images from photovoltaic panel images are continuously acquired and preprocessed to obtain preprocessed photovoltaic panel images;

[0015] All photovoltaic panels are labeled with information to obtain labeled images;

[0016] All the annotated images were randomly assigned to the training and test sets according to an 8:2 ratio, with the number of images in the training and test sets set based on the anomaly category. The photovoltaic panel electrical data and photovoltaic panel environmental parameters corresponding to the annotated images were also assigned simultaneously.

[0017] Optionally, obtaining the anomaly detection model for photovoltaic panels based on information from the training set includes:

[0018] The photovoltaic panel image information in the training set is registered to achieve alignment of the photovoltaic panel images and maximum information preservation.

[0019] Obtain the visible light and infrared light weight vectors within the photovoltaic panel image, and perform channel-dimensional mutual enhancement based on MLP and Softmax to obtain the enhanced feature maps of visible light and infrared light;

[0020] The enhanced feature maps of visible light and infrared light are mixed based on MLP, and normalized cross-weight calibration is performed based on the Sigmoid function to obtain the enhanced features after secondary enhancement.

[0021] An anomaly detection model is obtained by co-training the enhanced features based on prototype image contrast regularization.

[0022] Optionally, the registration process for the photovoltaic panel image information in the training set to achieve alignment and maximum information preservation of the photovoltaic panel images includes:

[0023] The low-resolution visible light image in the photovoltaic panel image is upsampled according to the width ratio of the infrared image to obtain a width-matched visible light image. The equation for determining the width-matched visible light image is as follows:

[0024] ,

[0025] in, This represents the new visible light image after resizing; Upsample() represents the upsampling algorithm; I vis Represents a low-resolution visible light image; s represents the scaling factor; W ir Indicates the width of the infrared image; W vis Indicates the width of the infrared image;

[0026] The coordinate system of the size-matched visible light image is adjusted based on the scaling factor to obtain new coordinates. The equation for determining the new coordinates is:

[0027] ,

[0028] in, Relative to I vis The original coordinates; This represents the new coordinates after scaling.

[0029] Obtain the center of the bounding box in the infrared image and the center of the bounding box in the scaled visible light image, and determine the equation as follows:

[0030] ,

[0031] Among them, C ir Indicates the center of the infrared image annotation box; This indicates the center of the bounding box in the scaled visible light image; These represent the coordinates of the infrared image bounding box; These represent the coordinates of the bounding box in the visible light image;

[0032] Obtain the overlapping region of the scaled visible light image and infrared image, and obtain the coordinates of the maximum cropping window of the infrared image. The equation is then determined as follows:

[0033] ,

[0034] in, These represent the maximum and minimum coordinates of the maximum cropping window in the infrared image, respectively. H represents the maximum and minimum coordinates of the overlapping region, respectively; ir Indicates the height of the infrared image; W vis-crop This represents the maximum cropping window for the infrared image;

[0035] Based on the maximum cropping coordinate window of the infrared image, the maximum cropping coordinate window of the visible light image is obtained, and the equation is determined as follows:

[0036]

[0037] Among them, W vis-crop The maximum cropping coordinate window of the visible light image is represented by v; v represents the translational deviation vector between the center of the infrared image bounding box and the center of the scaled visible light image bounding box, with the equation being: .

[0038] Optionally, the step of obtaining visible light and infrared light weight vectors within the photovoltaic panel image and performing channel-dimensional mutual enhancement based on MLP and Softmax to obtain enhanced feature maps of visible light and infrared light includes:

[0039] Based on the channel attention mechanism, weight vectors for visible light and infrared light images are generated. The equation for determining the weight vectors is as follows:

[0040] ,

[0041] in, Represents the weight vector of a visible light image; Represents the weight vector of an infrared image; This represents the channel attention mechanism algorithm; F vis-align Represents a visible light image feature map; F ir-align Represents the feature map of an infrared light image;

[0042] Based on MLP, mixed features are obtained, and the equation for determining the mixed features is:

[0043] ,

[0044] Among them, F mix This indicates mixed features; MLP() represents the MLP algorithm equation; Concat() represents concatenation;

[0045] Based on the channel attention mechanism, cross-modal weights for hybrid features are generated. The equation for determining the cross-modal weights is as follows:

[0046] ,

[0047] in, Indicates cross-modal weights;

[0048] Based on Softmax, the cross-modal weights are calibrated and enhanced to obtain the enhanced feature vector, with the equation as follows:

[0049] ,

[0050] in, This represents the feature vector after visible light image enhancement; This represents the feature vector after infrared image enhancement;

[0051] Based on the enhanced feature vector, the enhanced feature map is obtained, and the equation for determining the enhanced feature map is:

[0052] ,

[0053] in, This represents the feature map after enhancement of a visible light image; This represents the feature map after infrared light image enhancement.

[0054] Optionally, the step of performing a hybrid processing on the enhanced feature maps of visible and infrared light based on MLP, and then performing normalized cross-weight calibration based on the Sigmoid function to obtain the enhanced features after secondary enhancement includes:

[0055] Based on MLP, the enhanced feature maps of visible light and infrared light are mixed, and the mixing equation is:

[0056] ,

[0057] Among them, M cross Represents the space diagram after mixing;

[0058] Normalized cross-weight calibration is performed based on the Sigmoid function to obtain the features after secondary enhancement. The equation for determining the features after secondary enhancement is as follows:

[0059] ,

[0060] in, This represents the visible light feature map after secondary enhancement; This represents the infrared light feature map after secondary enhancement; This represents the visible light space map after normalized cross-weight calibration. This represents the infrared spatial map after normalized cross-weight calibration.

[0061] The equations for determining the visible light spatial map and the infrared light spatial map after normalized cross-weight calibration are as follows:

[0062] ,

[0063] Among them, M vis Represents the visible light space diagram; M ir Represents an infrared light spatial diagram;

[0064] The equations for determining the visible light space map and the infrared light space map are:

[0065] ;

[0066] The enhanced features are then fused to achieve bidirectional enhancement of both dimensionality and spatial features. The fusion equation is as follows:

[0067] ,

[0068] Among them, F fused Indicates the result of the fusion process; P pred Represents a structured tensor; H det ( ) represents the detector head network equation.

[0069] Optionally, the step of co-training the enhanced features based on prototype image contrast regularization to obtain the anomaly detection model includes:

[0070] Obtain the final bounding box coordinates of the original infrared image and the upsampled visible light image. The equation for determining the final bounding box coordinates is:

[0071] ,

[0072] in, Represents the final bounding box coordinates of the original infrared image; Represents the final bounding box coordinates of the upsampled visible light image; This indicates the cropping window prediction box; This represents the coordinate offset of the top-left corner of the original infrared image cropping window; This represents the offset corresponding to the cropping window of the upsampled visible light image;

[0073] The original image is cropped based on the final bounding box coordinates, and the precise feature region corresponding to each anomaly in the cropped original image is obtained. Then, average pooling is performed. The processing equation is as follows:

[0074] ,

[0075] in, Represents the visible light prototype of the i-th detected anomaly; Represents the infrared light prototype of the i-th detected anomaly; AvgPool() represents the average pooling algorithm; RoIAlign() represents the RoIAlign algorithm; This represents the feature map of the upsampled visible light image; This represents the original visible light image after cropping. Represents the feature map of the original infrared light image; This refers to the original infrared image after the incident.

[0076] Based on the detected abnormal visible and infrared light prototypes, the weights of edges between nodes of the same type are obtained, and the equation is determined as follows:

[0077] ,

[0078] Among them, A ij z represents the weight of edges between nodes of the same type. i z represents the prototype of the i-th detected anomaly. j This represents the prototype of the j-th detected anomaly;

[0079] The detected anomalies are subjected to prototype graph regularization to obtain the sum of regularized losses. The sum of losses equation is as follows:

[0080] ,

[0081] in, This indicates that the visible light prototype is used to query the loss in the infrared image; This indicates that the infrared light prototype is used to query the loss in the visible light image; vis,i represents the visible light prototype extracted from the graphic information; ir,i represents the infrared light prototype separated from the graphic information; Indicates the loss ratio; N represents the total number of nodes; L gcr Indicates the total loss;

[0082] The total model loss is determined by the following equation:

[0083] ,

[0084] Among them, Ltotal L represents the total loss of the model; yolo This indicates the loss in YOLO detection; The hyperparameter representing the contribution of the balancing loss term.

[0085] Optionally, obtaining high-dimensional features for predicting photovoltaic panel losses based on the anomaly detection model includes:

[0086] Based on the input information from visible and infrared light, image texture information containing the defect location is obtained. The equation for determining the image texture information is:

[0087] ,

[0088] Among them, F rgb This represents visible light image texture information containing the location of the defect; F ir This represents the infrared image texture information containing the location of the defect; F CNN Represents the feature extraction backbone network; X rgb Indicates visible light input; Represents the visible light weighting parameter; X ir Indicates external infrared light input; P represents the infrared weighting parameter; pos Represents the sinusoidal position encoding matrix;

[0089] Based on a binary mask, background information in the image texture information containing the defect location is removed to obtain the defect feature image;

[0090] Thermodynamic parameters within the defect feature image are obtained, and when the thermodynamic parameters and the defect feature image correspond, multimodal fusion features after spatiotemporal alignment and background suppression are obtained;

[0091] Based on a multilayer perceptron, affine transformation coefficients are dynamically generated from environmental boundary conditions. These coefficients are then fused with the multimodal fusion features that have undergone spatiotemporal alignment and background suppression through a Hadamard product to obtain high-dimensional features for photovoltaic panel loss prediction.

[0092] Optionally, the step of eliminating errors in the high-dimensional features of the loss prediction based on semiconductor physical constraints to obtain accurate photovoltaic panel loss prediction results includes:

[0093] To obtain the ideal parameters of the photovoltaic panel under the current environmental conditions, the equation for determining the ideal parameters of the photovoltaic panel is:

[0094] ,

[0095] in, Indicates the ideal parameters of a photovoltaic panel; f phy( ) represents the SDM physical equation; G represents radiance; T represents temperature; datasheet represents the photovoltaic panel data sheet; I ph Represents the photogenerated current; I0 ​​represents the diode reverse saturation current; R s Represents series resistance; R sh This represents the parallel resistance; n represents the diode ideality factor;

[0096] Based on the high-dimensional characteristics of the photovoltaic panel's loss prediction, residual parameters are predicted, and the residual parameter prediction equation is as follows:

[0097] ,

[0098] in, H represents the predicted residual parameter; regression ( ) represents the regression head network; F final This represents the high-dimensional characteristics of photovoltaic panel loss prediction.

[0099] The ideal parameters of the photovoltaic panel are corrected based on the predicted residual parameters to obtain the photovoltaic panel loss prediction curve.

[0100] The beneficial effects of this application include:

[0101] 1. Achieved non-contact photovoltaic internal resistance quantification: The model predicts the loss of photovoltaic modules using only captured RGB and IR images and environmental parameters, effectively replacing expensive EL detection.

[0102] 2. Improved loss prediction accuracy in complex scenarios: Compared to predictions that rely solely on SCADA data, the introduction of visual features can effectively reduce the prediction error of power loss caused by local occlusion, bird droppings, microcracks, etc.

[0103] 3. It has flexible engineering application capabilities: The model can be deployed on edge computing devices, supporting both real-time high-frequency prediction through communication with inverters and offline image import for regular health checks. One algorithm meets two operation and maintenance needs.

[0104] 4. Significantly improved accuracy: Through multimodal attention fusion and detection, label-driven cross-modal registration and prototype-based graph contrast regularization collaborative training, a significant improvement in accuracy is achieved. Attached Figure Description

[0105] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the embodiments of this application or the prior art will be briefly introduced below. Obviously, the following description is only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:

[0106] Figure 1 A flowchart of a photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters provided in this application embodiment;

[0107] Figure 2 The image shows the application effect of a photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters provided in this application embodiment. Detailed Implementation

[0108] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, in the embodiments of this application, "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0109] This application aims to develop a photovoltaic (PV) panel loss prediction method based on the fusion of visual images and physical parameters. This method utilizes readily available visible light (RGB) and infrared (IR) images instead of expensive EL images. Through a high-precision, robust bi-branch multimodal fusion detection framework, it leverages label-driven automatic registration, mutual attention feature fusion, and regularized training using graph contrastive learning to significantly improve the average precision and recall rate of PV module anomaly detection. The anomaly detection image results are combined with electrical characteristics and environmental parameters to establish a nonlinear mapping between visual features and electrical losses. The goal is to provide reliable estimations even with only images and environmental parameters, thereby guiding the refined operation and maintenance of power plants. Figure 1 The diagram shown is a flowchart of a photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters disclosed in this application. Specifically:

[0110] S110. Obtain photovoltaic panel data, and after preprocessing the obtained photovoltaic panel data, configure it into the training set and the test set.

[0111] S120. Based on the information in the training set, obtain an anomaly detection model for photovoltaic panels.

[0112] S130. Based on the anomaly detection model, obtain the high-dimensional features for predicting the loss of the photovoltaic panel.

[0113] S140. Based on semiconductor physical constraints, error elimination is performed on the high-dimensional features of the loss prediction to obtain accurate photovoltaic panel loss prediction results.

[0114] The following will provide a detailed explanation of all the steps above:

[0115] As described in step S110, the purpose of this step is to obtain all the data in the photovoltaic panel loss prediction process, which can then be used for model training. Specifically:

[0116] The photovoltaic panel data includes photovoltaic panel image information, photovoltaic panel electrical data, and photovoltaic panel environmental parameters;

[0117] Establish the correspondence between the acquired photovoltaic panel image information, the synchronously acquired photovoltaic panel electrical data, and the photovoltaic panel environmental parameters;

[0118] Visible light and infrared images from photovoltaic panel images are continuously acquired and preprocessed to obtain preprocessed photovoltaic panel images;

[0119] All photovoltaic panels are labeled with information to obtain labeled images;

[0120] All the annotated images were randomly assigned to the training and test sets according to an 8:2 ratio, with the number of images in the training and test sets set based on the anomaly category. The photovoltaic panel electrical data and photovoltaic panel environmental parameters corresponding to the annotated images were also assigned simultaneously.

[0121] In terms of image acquisition, to overcome the inherent limitations of single-modal datasets in robust photovoltaic anomaly detection, this study constructed a new and comprehensive multimodal dataset, named PV-VISIR. This dataset contains 2981 carefully collected image pairs from seven different and typical power plant projects. To ensure diversity, the collection process covered photovoltaic panels of various shapes, sizes, and orientations, and was conducted under diverse lighting, background, and weather conditions. The FLIRT610 camera, capable of simultaneously capturing visible light (VIS) and infrared (IR) images, was used for acquisition. Following collection, a rigorous annotation process was performed to establish a baseline truth for model training. With the assistance of photovoltaic experts, the LabelImg annotation tool was used to manually label the category and bounding box location of all identifiable instances. After initial screening, 2146 pairs of VIS-IR images were fully annotated. These annotations are divided into nine different categories: offline, dirt, hotspots, leaves, bird droppings, shadows, cracks, good (representing normal conditions), and unknown (representing unidentifiable anomalies). To develop and evaluate the model, this labeled dataset was divided into a training set (80%) and a test set (20%).

[0122] In terms of electrical data acquisition, data including voltage, current, and power were obtained, and missing values ​​were filled using linear interpolation. Outliers were removed and corrected; for example, data with P < 0 (sensor drift) were set to 0, and data with P > P installed (installed capacity) were considered outliers. If the error between the recorded power and the calculated value of P = V × I exceeded a certain threshold (e.g., 5%), it indicated a sensor malfunction and the data was removed. The data was also normalized and scaled to the [0,1] interval. To develop and evaluate the model, the dataset was divided into a training set (80%) and a test set (20%).

[0123] Regarding environmental parameter acquisition, parameters such as temperature, humidity, wind speed, irradiance, and panel temperature were obtained. Missing values ​​were filled using linear interpolation. Outliers were removed and corrected; for example, data with G < 0 were set to 0; data with G > 1400 W / m² (solar constant) were generally considered outliers and removed. Not all parameters are useful for prediction; redundant features can slow down model training or even introduce noise. The correlation coefficients between each environmental parameter and power (P) were calculated. Irradiance (G) typically had the highest correlation (>0.9), followed by panel temperature (Tmod). Humidity (H) usually showed a negative correlation. The data was normalized and scaled to the [0,1] interval. To develop and evaluate the model, the dataset was divided into a training set (80%) and a test set (20%).

[0124] As described in step S120, the purpose of this step is to process the input information based on multiple core detection stages to obtain a prediction model. Specifically:

[0125] The photovoltaic panel image information in the training set is registered to achieve image alignment and maximum information preservation. Specifically:

[0126] The low-resolution visible light image in the photovoltaic panel image is upsampled according to the width ratio of the infrared image to obtain a width-matched visible light image. The equation for determining the width-matched visible light image is as follows:

[0127] ,

[0128] in, This represents the new visible light image after resizing; Upsample() represents the upsampling algorithm; I vis Represents a low-resolution visible light image; s represents the scaling factor; W ir Indicates the width of the infrared image; W vis Indicates the width of the infrared image;

[0129] The coordinate system of the size-matched visible light image is adjusted based on the scaling factor to obtain new coordinates. The equation for determining the new coordinates is:

[0130] ,

[0131] in, Relative to I vis The original coordinates; This represents the new coordinates after scaling.

[0132] Obtain the center of the bounding box in the infrared image and the center of the bounding box in the scaled visible light image, and determine the equation as follows:

[0133] ,

[0134] Among them, C ir Indicates the center of the infrared image annotation box; This indicates the center of the bounding box in the scaled visible light image; These represent the coordinates of the infrared image bounding box; These represent the coordinates of the bounding box in the visible light image;

[0135] Obtain the overlapping region of the scaled visible light image and infrared image, and obtain the coordinates of the maximum cropping window of the infrared image. The equation is then determined as follows:

[0136] ,

[0137] in, These represent the maximum and minimum coordinates of the maximum cropping window in the infrared image, respectively. H represents the maximum and minimum coordinates of the overlapping region, respectively; ir Indicates the height of the infrared image; W vis-crop This represents the maximum cropping window for the infrared image;

[0138] Based on the maximum cropping coordinate window of the infrared image, the maximum cropping coordinate window of the visible light image is obtained, and the equation is determined as follows:

[0139]

[0140] Among them, W vis-crop The maximum cropping coordinate window of the visible light image is represented by v; v represents the translational deviation vector between the center of the infrared image bounding box and the center of the scaled visible light image bounding box, with the equation being: .

[0141] The visible and infrared light weight vectors within the photovoltaic panel image are obtained, and channel-dimensional mutual enhancement is performed based on MLP and Softmax to obtain enhanced feature maps of visible and infrared light. Specifically:

[0142] Based on the channel attention mechanism, weight vectors for visible light and infrared light images are generated. The equation for determining the weight vectors is as follows:

[0143] ,

[0144] in, Represents the weight vector of a visible light image; Represents the weight vector of an infrared image; This represents the channel attention mechanism algorithm; F vis-align Represents a visible light image feature map; F ir-align Represents the feature map of an infrared light image;

[0145] Based on MLP, mixed features are obtained, and the equation for determining the mixed features is:

[0146] ,

[0147] Among them, F mix This indicates mixed features; MLP() represents the MLP algorithm equation; Concat() represents concatenation;

[0148] Based on the channel attention mechanism, cross-modal weights for hybrid features are generated. The equation for determining the cross-modal weights is as follows:

[0149] ,

[0150] in, Indicates cross-modal weights;

[0151] Based on Softmax, the cross-modal weights are calibrated and enhanced to obtain the enhanced feature vector, with the equation as follows:

[0152] ,

[0153] in, This represents the feature vector after visible light image enhancement; This represents the feature vector after infrared image enhancement;

[0154] Based on the enhanced feature vector, the enhanced feature map is obtained, and the equation for determining the enhanced feature map is:

[0155] ,

[0156] in, This represents the feature map after enhancement of a visible light image; This represents the feature map after infrared light image enhancement.

[0157] The enhanced feature maps of visible and infrared light are mixed based on MLP, and normalized cross-weight calibration is performed based on the Sigmoid function to obtain the enhanced features after secondary enhancement. Specifically:

[0158] Based on MLP, the enhanced feature maps of visible light and infrared light are mixed, and the mixing equation is:

[0159] ,

[0160] Among them, M cross Represents the space diagram after mixing;

[0161] Normalized cross-weight calibration is performed based on the Sigmoid function to obtain the features after secondary enhancement. The equation for determining the features after secondary enhancement is as follows:

[0162] ,

[0163] in, This represents the visible light feature map after secondary enhancement; This represents the infrared light feature map after secondary enhancement; This represents the visible light space map after normalized cross-weight calibration. This represents the infrared spatial map after normalized cross-weight calibration.

[0164] The equations for determining the visible light spatial map and the infrared light spatial map after normalized cross-weight calibration are as follows:

[0165] ,

[0166] Among them, M vis Represents the visible light space diagram; M ir Represents an infrared light spatial diagram;

[0167] The equations for determining the visible light space map and the infrared light space map are:

[0168] ;

[0169] The enhanced features are then fused to achieve bidirectional enhancement of both dimensionality and spatial features. The fusion equation is as follows:

[0170] ,

[0171] Among them, F fused Indicates the result of the fusion process; P pred Represents a structured tensor; H det ( ) represents the detector head network equation.

[0172] The anomaly detection model is obtained by co-training the enhanced features based on prototype image contrast regularization. Specifically:

[0173] Obtain the final bounding box coordinates of the original infrared image and the upsampled visible light image. The equation for determining the final bounding box coordinates is:

[0174] ,

[0175] in, Represents the final bounding box coordinates of the original infrared image; Represents the final bounding box coordinates of the upsampled visible light image; This indicates the cropping window prediction box; This represents the coordinate offset of the top-left corner of the original infrared image cropping window; This represents the offset corresponding to the cropping window of the upsampled visible light image;

[0176] The original image is cropped based on the final bounding box coordinates, and the precise feature region corresponding to each anomaly in the cropped original image is obtained. Then, average pooling is performed. The processing equation is as follows:

[0177] ,

[0178] in, Represents the visible light prototype of the i-th detected anomaly; Represents the infrared light prototype of the i-th detected anomaly; AvgPool() represents the average pooling algorithm; RoIAlign() represents the RoIAlign algorithm; This represents the feature map of the upsampled visible light image; This represents the original visible light image after cropping. Represents the feature map of the original infrared light image; This refers to the original infrared image after the incident.

[0179] Based on the detected abnormal visible and infrared light prototypes, the weights of edges between nodes of the same type are obtained, and the equation is determined as follows:

[0180] ,

[0181] Among them, A ij z represents the weight of edges between nodes of the same type. i z represents the prototype of the i-th detected anomaly. j This represents the prototype of the j-th detected anomaly;

[0182] The detected anomalies are subjected to prototype graph regularization to obtain the sum of regularized losses. The sum of losses equation is as follows:

[0183] ,

[0184] in, This indicates that the visible light prototype is used to query the loss in the infrared image; This indicates that the infrared light prototype is used to query the loss in the visible light image; vis,i represents the visible light prototype extracted from the graphic information; ir,i represents the infrared light prototype separated from the graphic information; Indicates the loss ratio; N represents the total number of nodes; L gcr Indicates the total loss;

[0185] The total model loss is determined by the following equation:

[0186] ,

[0187] Among them, L total L represents the total loss of the model; yolo This indicates the loss in YOLO detection; The hyperparameter representing the contribution of the balancing loss term.

[0188] The Multimodal Attention Fusion (MMAF) module aims to enable two modalities to "mutually guide" feature extraction, rather than simply superimposing them. The MMAF module is divided into two parts: Channel Attention Fusion (which generates modality-specific channel weights and calibrates these weights using cross-modal information to achieve channel-level mutual enhancement) and Spatial Attention Fusion (which, based on Channel Attention Fusion, further generates a modality-specific spatial attention map and calibrates it using cross-modal information to achieve spatial-level mutual enhancement). Channel Attention Fusion: It calculates the channel weights for VIS (visible light) and IR (infrared) separately. Simultaneously, it concatenates the features of both and generates a "hybrid channel weight" using MLP. The Softmax-normalized hybrid weights are then used to calibrate (multiply) the original weights of VIS and IR respectively. Finally, the calibrated IR features are added to the VIS features, and vice versa, achieving channel-level mutual enhancement. Spatial Attention Fusion: Following Channel Fusion, similar logic is used. Spatial descriptors are extracted using max pooling and average pooling, and a spatial attention map is generated through convolution. Similarly, a "hybrid spatial attention map" is generated to calibrate the spatial weights of each modality, achieving mutual enhancement of spatial dimensions. Final fusion: The two feature paths, after channel and spatial mutual enhancement, are element-wise added and input into the YOLOv8 detector head.

[0189] To overcome the domain gap between modalities, Prototype Graph Contrast Regularization (PGCR) is introduced as an auxiliary branch (co-training via PGCR) to further enhance the model's robustness and generalization ability. While the aforementioned detection paths form the basis of the proposed model, direct supervision alone is insufficient due to two inherent limitations. First, despite initial label-driven cross-modal registration, slight spatial errors remain due to image upsampling, coordinate recalibration, and inaccurate labeling, hindering perfect alignment. Second, significant domain differences exist between visible and infrared images; even with a shared encoder, features may occupy different subspaces because the two modalities capture different physical information. These challenges indicate that supervisory loss from the main detection branch is insufficient to force the model to learn truly robust and modality-invariant features.

[0190] To address these issues, the proposed framework adds an auxiliary branch and introduces a prototype graph contrastive regularization term. The core principle of this design is to break free from the constraints of direct pixel-level correspondences and instead enforce consistency in a higher-dimensional object-level feature space. It achieves this by explicitly encouraging the convergence of feature representations of anomalies of the same class, thus learning a unified representation regardless of the input modality.

[0191] After performing prototype image comparison regularization, and reprojecting these coordinates back onto their respective original images, object-level features can be accurately extracted. RoIAlign is used to extract the precise feature regions corresponding to each anomaly, respectively from the original infrared feature map F. ir-orig and upsampled visible feature map

[0192] F′ vis-orig Extracting from the region. Then, spatial average pooling is applied to each extracted region to fold its spatial dimensions into a feature vector, which is the prototype of each modality.

[0193] In this process, the original graph is constructed and the GCN is propagated. G is formally defined by its node features and weighted adjacency matrix, which encodes internal and multimodal relationships.

[0194]

[0195] After constructing the graph, the module uses a Graph Convolutional Network (GCN) to propagate information between nodes. The GCN layer updates the feature representation of each node by aggregating messages from its direct neighbors, allowing each prototype to refine and fuse features with those of other similar anomalies (including anomalies from another modality) in its batch. The feature propagation formula in the GCN layer is as follows:

[0196]

[0197] Where Z′ is the updated node feature matrix, A is the adjacency matrix, and I is the identity matrix. yes The diagonal matrix is ​​denoted by W. W is a learnable weight matrix implemented as a linear layer for feature transformation, and σ is the ReLU activation function. The output matrix Z′ contains the fused prototype, which is then separated into visible prototypes of graphical information. vis,i} and infrared prototype { ir,i}

[0198] Specifically, a high-dimensional visual loss feature vector (F) with anomaly detection results is generated through a dual-branch multimodal fusion framework. v ).

[0199] As described in step S130, the purpose of this step is to determine the problem area information in the image, thereby enabling a joint analysis of environmental and electrical information. Specifically:

[0200] Based on the input information from visible and infrared light, image texture information containing the defect location is obtained. The equation for determining the image texture information is:

[0201] ,

[0202] Among them, F rgb This represents visible light image texture information containing the location of the defect; F ir This represents the infrared image texture information containing the location of the defect; F CNN Represents the feature extraction backbone network; X rgb Indicates visible light input; Represents the visible light weighting parameter; X ir Indicates external infrared light input; P represents the infrared weighting parameter; pos Represents the sinusoidal position encoding matrix;

[0203] Based on a binary mask, background information in the image texture information containing the defect location is removed to obtain the defect feature image;

[0204] Thermodynamic parameters within the defect feature image are obtained, and when the thermodynamic parameters and the defect feature image correspond, multimodal fusion features after spatiotemporal alignment and background suppression are obtained;

[0205] Based on a multilayer perceptron, affine transformation coefficients are dynamically generated from environmental boundary conditions. These coefficients are then fused with the multimodal fusion features that have undergone spatiotemporal alignment and background suppression through a Hadamard product to obtain high-dimensional features for photovoltaic panel loss prediction.

[0206] This step aims to address the spatial location information loss problem caused by the translation invariance of standard convolutional neural networks (CNNs). Because photovoltaic modules employ a specific series-parallel circuit topology, the same type of defect, if located at different positions within the cell string, will have drastically different impacts on the current path triggering state and the final IV curve characteristics. Specifically, and They are visible light and infrared inputs, respectively. For feature extraction backbone network, This is a sinusoidal position encoding matrix. These are the weight parameters. Coordinate information is injected into the feature map by adding elements one by one, giving the model spatial sensitivity.

[0207] In this step, after obtaining features containing location information, a binary mask output from the upstream detection task is used as prior knowledge to construct a "hard attention" mechanism. Unlike conventional global attention, the purpose of this invention is physical isolation. To prevent background interference from participating in subsequent electrical parameter regression, mathematical methods are used to shield feature fusion from background noise, forcing the model to focus computational resources and gradient updates entirely on detected anomalous regions, thereby reducing the random uncertainty introduced by background complexity. ( )=

[0208] The bias matrix is ​​constructed using the input anomaly detection mask. In the subsequent Softmax operation, the weight of the background region will be made close to zero, thereby achieving physical isolation at the feature level.

[0209] After eliminating background interference, this step addresses establishing semantic alignment between visual morphological features (RGB) and thermodynamic features (IR) for multimodal fault detection. The aim is to establish a mapping between morphological features and thermodynamic features. Unlike simple feature concatenation, features are only highly activated when visually abnormal regions also possess specific thermodynamic responses. This mechanism effectively suppresses spurious defects that only manifest in a single modality, improving the robustness of feature representation.

[0210] Q=

[0211] K=

[0212] V=

[0213] =Softmax( + V

[0214] , , The projection matrix is ​​learnable. As an additive bias term involved in the calculation of the attention score, it ensures that feature interactions are limited only to the region defined by the anomaly mask. This refers to the multimodal fusion feature after spatiotemporal alignment and background suppression.

[0215] This module aims to address the nonlinear coupling problem of the influence of environmental boundary conditions (irradiance G, temperature T, etc.) on defects. Due to the physical nature of the photovoltaic effect, the same physical defect (such as shading area) will produce significantly different magnitudes of photocurrent loss under different irradiance levels. Directly inputting environmental parameters into the network makes it difficult to capture this multiplicative relationship. Therefore, a characteristic linear modulation mechanism is adopted to dynamically generate affine transformation coefficients using environmental parameters, and to perform channel-level scaling and translation of image features to conform to the linear or nonlinear gain relationship in physical laws.

[0216] ( )= ([G, ])

[0217] = +

[0218] It is a multilayer perceptron used to learn the mapping from environmental parameters to feature adjustment factors. and Through Hadama accumulation Directly applied to the fusion features, resulting in the final features It has environmental adaptability.

[0219] As described in step S140, the purpose of this step is to obtain specific photovoltaic panel loss prediction results. Specifically:

[0220] To obtain the ideal parameters of the photovoltaic panel under the current environmental conditions, the equation for determining the ideal parameters of the photovoltaic panel is:

[0221] ,

[0222] in, Indicates the ideal parameters of a photovoltaic panel; f phy ( ) represents the SDM physical equation; G represents radiance; T represents temperature; datasheet represents the photovoltaic panel data sheet; I ph Represents the photogenerated current; I0 ​​represents the diode reverse saturation current; R s Represents series resistance; R sh This represents the parallel resistance; n represents the diode ideality factor;

[0223] Based on the high-dimensional characteristics of the photovoltaic panel's loss prediction, residual parameters are predicted, and the residual parameter prediction equation is as follows:

[0224] ,

[0225] in, H represents the predicted residual parameter; regression ( ) represents the regression head network; F final This represents the high-dimensional characteristics of photovoltaic panel loss prediction.

[0226] The ideal parameters of the photovoltaic panel are corrected based on the predicted residual parameters to obtain the photovoltaic panel loss prediction curve.

[0227] This stage serves as a bridge connecting deep learning and the physical model. To ensure that the output strictly adheres to the physical laws of semiconductors (such as the logarithmic change of open-circuit voltage with temperature and the linear change of current with irradiance), the model does not directly regress the point set of the IV curve. Instead, it predicts the residuals of the single-diode model (SDM) parameters relative to the "fault-free theoretical state." This method utilizes the SDM as an inductive bias (ensuring that the generated IV curve is always smooth and physically meaningful), while simultaneously fitting parameter drift caused by complex defects through a neural network.

[0228] The obtained parameters are further corrected, and the curve is solved:

[0229] = +λ

[0230] I= - -1)-

[0231] It is a regression head network. The amount of parameter degradation predicted by AI (such as series resistance) Incremental, photocurrent The loss is calculated as follows: I and V represent the output current and voltage of the photovoltaic module. ε is a sign vector indicating the direction of change of control parameters (e.g., resistance can only increase). Finally, the predicted IV curve is obtained by solving the above transcendental equation using Newton's iteration method.

[0232] like Figure 2 The image shown is an application effect diagram of a photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters provided in an embodiment of this application.

[0233] The beneficial effects of this application include:

[0234] 1. Achieved non-contact photovoltaic internal resistance quantification: The model predicts the loss of photovoltaic modules using only captured RGB and IR images and environmental parameters, effectively replacing expensive EL detection.

[0235] 2. Improved loss prediction accuracy in complex scenarios: Compared to predictions that rely solely on SCADA data, the introduction of visual features can effectively reduce the prediction error of power loss caused by local occlusion, bird droppings, microcracks, etc.

[0236] 3. It has flexible engineering application capabilities: The model can be deployed on edge computing devices, supporting both real-time high-frequency prediction through communication with inverters and offline image import for regular health checks. One algorithm meets two operation and maintenance needs.

[0237] 4. Significantly improved accuracy: Through multimodal attention fusion and detection, label-driven cross-modal registration and prototype-based graph contrast regularization collaborative training, a significant improvement in accuracy is achieved.

[0238] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to computer program instructions. The aforementioned computer program can be stored in a non-volatile storage medium, and when executed, it performs the steps of the above method embodiments. Alternatively, if the integrated unit of the present invention is implemented as a software functional module and sold or used as an independent product, it can also be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention.

[0239] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters, characterized in that, The prediction method includes: Acquire photovoltaic panel data, preprocess the obtained photovoltaic panel data, and then configure it into the training set and the test set; Based on the information in the training set, an anomaly detection model for photovoltaic panels is obtained; Based on the anomaly detection model, high-dimensional features for predicting the loss of photovoltaic panels are obtained; Based on semiconductor physical constraints, error elimination is performed on the high-dimensional features of the loss prediction to obtain accurate photovoltaic panel loss prediction results.

2. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 1, characterized in that, The process of acquiring photovoltaic panel data, preprocessing the obtained photovoltaic panel data, and configuring it into the training and testing sets includes: The photovoltaic panel data includes photovoltaic panel image information, photovoltaic panel electrical data, and photovoltaic panel environmental parameters; Establish the correspondence between the acquired photovoltaic panel image information, the synchronously acquired photovoltaic panel electrical data, and the photovoltaic panel environmental parameters; Visible light and infrared images from photovoltaic panel images are continuously acquired and preprocessed to obtain preprocessed photovoltaic panel images; All photovoltaic panels are labeled with information to obtain labeled images; All the annotated images were randomly assigned to the training and test sets according to an 8:2 ratio, with the number of images in the training and test sets set based on the anomaly category. The photovoltaic panel electrical data and photovoltaic panel environmental parameters corresponding to the annotated images were also assigned simultaneously.

3. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 1, characterized in that, The method for obtaining an anomaly detection model for photovoltaic panels based on information from the training set includes: The photovoltaic panel image information in the training set is registered to achieve alignment of the photovoltaic panel images and maximum information preservation. Obtain the visible light and infrared light weight vectors within the photovoltaic panel image, and perform channel-dimensional mutual enhancement based on MLP and Softmax to obtain the enhanced feature maps of visible light and infrared light; The enhanced feature maps of visible light and infrared light are mixed based on MLP, and normalized cross-weight calibration is performed based on the Sigmoid function to obtain the enhanced features after secondary enhancement. An anomaly detection model is obtained by co-training the enhanced features based on prototype image contrast regularization.

4. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 3, characterized in that, The registration process for photovoltaic panel image information in the training set, to achieve alignment and maximum information preservation of the photovoltaic panel images, includes: The low-resolution visible light image in the photovoltaic panel image is upsampled according to the width ratio of the infrared image to obtain a width-matched visible light image. The equation for determining the width-matched visible light image is as follows: ， in, This represents the new visible light image after resizing; Upsample() represents the upsampling algorithm; I vis This represents a low-resolution visible light image; s represents the scaling factor; W ir Indicates the width of the infrared image; W vis Indicates the width of the infrared image; The coordinate system of the size-matched visible light image is adjusted based on the scaling factor to obtain new coordinates. The equation for determining the new coordinates is: ， in, Relative to I vis The original coordinates; This represents the new coordinates after scaling. Obtain the center of the bounding box in the infrared image and the center of the bounding box in the scaled visible light image, and determine the equation as follows: ， Among them, C ir Indicates the center of the infrared image annotation box; This indicates the center of the bounding box in the scaled visible light image; These represent the coordinates of the infrared image bounding box; These represent the coordinates of the bounding box in the visible light image; Obtain the overlapping region of the scaled visible light image and infrared image, and obtain the coordinates of the maximum cropping window of the infrared image. The equation is then determined as follows: ， in, These represent the maximum and minimum coordinates of the maximum cropping window in the infrared image, respectively. H represents the maximum and minimum coordinates of the overlapping region, respectively; ir Indicates the height of the infrared image; W vis-crop This represents the maximum cropping window for the infrared image; Based on the maximum cropping coordinate window of the infrared image, the maximum cropping coordinate window of the visible light image is obtained, and the equation is determined as follows: ， Among them, W vis-crop The maximum cropping coordinate window of the visible light image is represented by v; v represents the translational deviation vector between the center of the infrared image bounding box and the center of the scaled visible light image bounding box, with the equation being: .

5. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 3, characterized in that, The process of obtaining visible and infrared light weight vectors within the photovoltaic panel image and performing channel-dimensional mutual enhancement based on MLP and Softmax to obtain enhanced feature maps of visible and infrared light includes: Based on the channel attention mechanism, weight vectors for visible light and infrared light images are generated. The equation for determining the weight vectors is as follows: ， in, Represents the weight vector of a visible light image; Represents the weight vector of an infrared image; This represents the channel attention mechanism algorithm; F vis-align Represents a visible light image feature map; F ir-align Represents the feature map of an infrared light image; Based on MLP, mixed features are obtained, and the equation for determining the mixed features is: ， Among them, F mix This indicates mixed features; MLP() represents the MLP algorithm equation; Concat() represents concatenation; Based on the channel attention mechanism, cross-modal weights for hybrid features are generated. The equation for determining the cross-modal weights is as follows: ， in, Indicates cross-modal weights; Based on Softmax, the cross-modal weights are calibrated and enhanced to obtain the enhanced feature vector, with the equation as follows: ， in, This represents the feature vector after visible light image enhancement; This represents the feature vector after infrared image enhancement; Based on the enhanced feature vector, the enhanced feature map is obtained, and the equation for determining the enhanced feature map is: ， in, This represents the feature map after enhancement of a visible light image; This represents the feature map after infrared light image enhancement.

6. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 3, characterized in that, The process of mixing the enhanced feature maps of visible and infrared light based on MLP and performing normalized cross-weight calibration based on the Sigmoid function to obtain the enhanced features after secondary enhancement includes: Based on MLP, the enhanced feature maps of visible light and infrared light are mixed, and the mixing equation is: ， Among them, M cross Represents the space diagram after mixing; Normalized cross-weight calibration is performed based on the Sigmoid function to obtain the features after secondary enhancement. The equation for determining the features after secondary enhancement is as follows: ， in, This represents the visible light feature map after secondary enhancement; This represents the infrared light feature map after secondary enhancement; This represents the visible light space map after normalized cross-weight calibration. This represents the infrared spatial map after normalized cross-weight calibration. The equations for determining the visible light spatial map and the infrared light spatial map after normalized cross-weight calibration are as follows: ， Among them, M vis Represents the visible light space diagram; M ir Represents an infrared light spatial diagram; The equations for determining the visible light space map and the infrared light space map are: ； The enhanced features are then fused to achieve bidirectional enhancement of both dimensionality and spatial features. The fusion equation is as follows: ， Among them, F fused Indicates the result of the fusion process; P pred H represents a structured tensor; det ( ) represents the detector head network equation.

7. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 3, characterized in that, The method of co-training the enhanced features based on prototype image contrast regularization to obtain the anomaly detection model includes: Obtain the final bounding box coordinates of the original infrared image and the upsampled visible light image. The equation for determining the final bounding box coordinates is: ， in, Represents the final bounding box coordinates of the original infrared image; Represents the final bounding box coordinates of the upsampled visible light image; This indicates the cropping window prediction box; This represents the coordinate offset of the top-left corner of the original infrared image cropping window; This represents the offset corresponding to the cropping window of the upsampled visible light image; The original image is cropped based on the final bounding box coordinates, and the precise feature region corresponding to each anomaly in the cropped original image is obtained. Then, average pooling is performed. The processing equation is as follows: ， in, Represents the visible light prototype of the i-th detected anomaly; Represents the infrared light prototype of the i-th detected anomaly; AvgPool() represents the average pooling algorithm; RoIAlign() represents the RoIAlign algorithm; This represents the feature map of the upsampled visible light image; This represents the original visible light image after cropping. Represents the feature map of the original infrared light image; This refers to the original infrared image after the incident. Based on the detected abnormal visible and infrared light prototypes, the weights of edges between nodes of the same type are obtained, and the equation is determined as follows: ， Among them, A ij z represents the weight of edges between nodes of the same type. i z represents the prototype of the i-th detected anomaly. j This represents the prototype of the j-th detected anomaly; The detected anomalies are subjected to prototype graph regularization to obtain the sum of regularized losses. The sum of losses equation is as follows: ， in, This indicates that the visible light prototype is used to query the loss in the infrared image; This indicates that the infrared light prototype is used to query the loss in the visible light image; vis,i represents the visible light prototype separated from the graphic information; ir,i represents the infrared light prototype separated from the graphic information; Indicates the loss ratio; N represents the total number of nodes; L gcr Indicates the total loss; The total model loss is determined by the following equation: ， Among them, L total L represents the total loss of the model; yolo This indicates the loss in YOLO detection; The hyperparameter representing the contribution of the balancing loss term.

8. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 1, characterized in that, The step of obtaining high-dimensional features for photovoltaic panel loss prediction based on the anomaly detection model includes: Based on the input information from visible and infrared light, image texture information containing the defect location is obtained. The equation for determining the image texture information is: ， Among them, F rgb This represents visible light image texture information containing the location of the defect; F ir This represents the infrared image texture information containing the location of the defect; F CNN Represents the feature extraction backbone network; X rgb Indicates visible light input; Represents the visible light weighting parameter; X ir Indicates external infrared light input; P represents the infrared weighting parameter; pos Represents the sinusoidal position encoding matrix; Based on a binary mask, background information in the image texture information containing the defect location is removed to obtain the defect feature image; Thermodynamic parameters within the defect feature image are obtained, and when the thermodynamic parameters and the defect feature image correspond, multimodal fusion features after spatiotemporal alignment and background suppression are obtained; Based on a multilayer perceptron, affine transformation coefficients are dynamically generated from environmental boundary conditions. These coefficients are then fused with the multimodal fusion features that have undergone spatiotemporal alignment and background suppression through a Hadamard product to obtain high-dimensional features for photovoltaic panel loss prediction.

9. The photovoltaic panel loss prediction method based on the fusion of visual images and physical parameters according to claim 1, characterized in that, The step of eliminating errors in the high-dimensional features of the loss prediction based on semiconductor physical constraints to obtain accurate photovoltaic panel loss prediction results includes: To obtain the ideal parameters of the photovoltaic panel under the current environmental conditions, the equation for determining the ideal parameters of the photovoltaic panel is: ， in, Indicates the ideal parameters of a photovoltaic panel; f phy ( ) represents the SDM physical equation; G represents radiance; T represents temperature; datasheet represents the photovoltaic panel data sheet; I ph Represents the photogenerated current; I0 ​​represents the diode reverse saturation current; R s Represents series resistance; R sh This represents the parallel resistance; n represents the diode ideality factor; Based on the high-dimensional characteristics of the photovoltaic panel's loss prediction, residual parameters are predicted, and the residual parameter prediction equation is as follows: ， in, H represents the predicted residual parameter. regression ( ) represents the regression head network; F final This represents the high-dimensional characteristics of photovoltaic panel loss prediction. The ideal parameters of the photovoltaic panel are corrected based on the predicted residual parameters to obtain the photovoltaic panel loss prediction curve.

Images