Photovoltaic module early fault monitoring and early warning method and system based on deep learning

Abstract

The invention discloses a photovoltaic module early-stage fault monitoring and early-warning method and system based on deep learning, and the system employs component-level multi-source sensing to capture I-V characteristics, temperature field distribution and environment data in real time, employs extracted inflection point I-V data to replace the total data for uploading, and achieves the early-stage fault monitoring and early-warning of a photovoltaic module. An early fault is accurately diagnosed through fusion of a CNN-LSTM hybrid network and spatial-temporal characteristics, a confidence coefficient threshold is dynamically corrected and used for graded early warning, and different treatment measures are adopted for different response grades. According to the method, the problems of component-level fault leak detection, high environment false alarm rate, bandwidth redundancy and non-planned shutdown are solved, the early fault detection rate is effectively improved, and meanwhile, the bandwidth consumption is reduced.

Description

Technical Field

[0001] This invention relates to a fault monitoring and early warning method and system, specifically to a method and system for early fault monitoring and early warning of photovoltaic modules based on deep learning. Background Technology

[0002] A photovoltaic (PV) array comprises several parallel strings of PV modules, each string consisting of multiple PV modules connected in series. The DC output of each PV module is connected in parallel to a string inverter. The PV module early fault monitoring and warning system identifies early faults such as hot spot effects, microcracks in solar cells, and PID degradation based on real-time collected electrical, thermal, and environmental parameters of the PV array, in order to avoid power generation losses and safety accidents. Existing PV module early fault monitoring and warning systems have the following technical problems:

[0003] (1) Use string current and voltage sensors and discrete temperature probes to monitor coarse particle size.

[0004] Current and voltage sensors are only installed in the string combiner box, which cannot locate faults in individual photovoltaic modules, resulting in more than 30% of early latent faults going undetected; temperature monitoring relies on sparsely arranged contact probes, which cannot capture the spatial distribution of local hot spots.

[0005] (2) Fault alarms are achieved by relying on threshold comparison or simple regression models, but data analysis is lagging behind.

[0006] Using a fixed threshold cannot distinguish between environmental fluctuations and real faults, with a false alarm rate as high as 42% in rainy weather; time series analysis relies only on a simple LSTM network, and the recognition rate for spatial faults such as microcracks in battery cells is less than 60%.

[0007] (3) Inefficient system architecture.

[0008] Uploading all raw sensor data consumes a significant amount of bandwidth.

[0009] (4) The fault response is only a single relay trip, which is prone to causing unplanned shutdowns due to false alarms. Summary of the Invention

[0010] Purpose of the invention: The purpose of this invention is to provide a method and system for early fault monitoring and warning of photovoltaic modules based on deep learning, so as to solve one or more technical problems existing in the prior art.

[0011] Technical solution: The present invention provides a method for early fault monitoring and warning of photovoltaic modules based on deep learning, comprising:

[0012] (1) Collect IV characteristic curve data, surface thermal imaging map and environmental data of each photovoltaic module;

[0013] (2) Add a unified timestamp to each set of collected data. At the same time, associate each set of collected data with the corresponding component ID and location information; extract the inflection point I-V data of the I-V characteristic curve.

[0014] (3) Preprocess and perform time-space alignment on the timestamped data stream; segment the single-component thermal map according to the component border ROI of the photovoltaic module surface thermal image, and obtain the temperature field matrix of the corresponding photovoltaic module according to the single-component thermal map; fuse the inflection point I-V data, temperature field matrix, and environmental data at the same time step into a multi-modal sample.

[0015] (4) Invoke the trained CNN-LSTM hybrid neural network, take the multi-modal sample as the input, output the fault category probability vector, and calculate the fault confidence CF.

[0016] The CNN-LSTM hybrid neural network includes a CNN branch, an LSTM branch, and an output layer: The CNN branch uses an improved ResNet-18 backbone, and inserts a Squeeze-and-Excitation channel attention module in each residual block; the input of the CNN branch is a three-channel tensor obtained by stacking "current voltage value, temperature field matrix", and the output is a spatial feature vector; the LSTM branch contains two layers of bidirectional LSTM; the input of the LSTM branch is a time series composed of the feature data output by the CNN branch at each moment, and the output is a temporal feature vector; the output layer performs feature concatenation on the spatial feature vector and the temporal feature vector to obtain a fused feature, and then passes through two layers of fully connected networks and Softmax activation to output the fault category probability vector P = (p_normal, p_degradation, p_hot-spot), and calculate the fault confidence CF = 1 - p_normal; p_normal represents the probability of the component operating normally, p_degradation represents the probability of the component suffering from degradation faults, and p_hot-spot represents the probability of hot spot faults.

[0017] (5) Generate a hierarchical response instruction according to the fault confidence CF and the dynamic confidence thresholds CF1 to CF3.

[0018] CF ≥ CF1: Trigger a first-level response and disconnect the DC output circuit of the faulty photovoltaic module.

[0019] CF2 ≤ CF < CF1: Trigger a second-level response, give an audible and visual alarm, and push a maintenance work order to the operation and maintenance management system.

[0020] CF3 ≤ CF < CF2: Trigger a third-level response, add the component ID to the "observation queue", re-evaluate the status of the component every once in a while, highlight the location of the component in the photovoltaic array thermal map, and at the same time generate a preliminary maintenance suggestion by combining the expert rule base and historical cases.

[0021] CF <CF3: It is determined that the operation is normal and no response is triggered;

[0022] The confidence threshold is dynamically corrected according to Δ = k (target false alarm rate - current false alarm rate), and the current false alarm rate is obtained through statistics; Δ is the change amount of the confidence threshold; k is a coefficient that controls the proportional relationship between the false alarm rate and the change amount of the confidence threshold.

[0023] Further, in step (2), the I-V characteristic curve is subjected to Savitzky-Golay smoothing processing, and then the inflection point I-V data of the I-V characteristic curve is extracted by the second derivative zero-crossing method to replace the original I-V characteristic curve data, and the missing points are corrected by neighborhood linear interpolation.

[0024] Further, in step (3), the data preprocessing includes denoising, missing value imputation, outlier filtering, and normalization.

[0025] Further, in step (5), the component position is inversely calculated based on the pixel coordinates of the component border ROI in the temperature field matrix and the pre-established "component physical coordinate - pixel coordinate mapping table" to obtain the array row-column-component ID; the preliminary maintenance suggestions are generated after retrieval according to the pre-constructed expert knowledge base in combination with the fault label.

[0026] Further, in step (5), CF1 is initially configured as 0.90, CF2 is initially configured as 0.70, and CF3 is initially configured as 0.50.

[0027] An early fault monitoring and warning system for photovoltaic modules based on deep learning according to the present invention includes a multi-source sensor module, an edge data acquisition unit, a cloud analysis and warning platform, and a warning execution terminal;

[0028] The multi-source sensor module is used to collect the I-V characteristic curve data, surface thermal imaging diagrams, and environmental data of each photovoltaic module;

[0029] The edge data acquisition unit is connected to the multi-source sensor module and is used to perform A / D conversion on the collected analog signals; add a unified timestamp to each group of collected data, and at the same time, associate each group of collected data with the corresponding component ID and position information; extract the inflection point I-V data of the I-V characteristic curve for replacing the original I-V characteristic curve data and uploading it to the cloud analysis and warning platform;

[0030] The cloud analysis and early warning platform is used to preprocess the time-stamped data stream and perform time-space alignment; segment the thermal imaging map of the photovoltaic module surface according to the ROI of the module border to obtain the single-module thermal map, and obtain the temperature field matrix of the corresponding photovoltaic module according to the single-module thermal map; fuse the inflection point I-V data, temperature field matrix and environmental data at the same time step into a multi-modal sample; call the trained CNN-LSTM hybrid neural network, use the multi-modal sample as the input, output the fault category probability vector and calculate the fault confidence CF; generate a hierarchical response instruction according to the fault confidence CF and the dynamic confidence threshold CF1 to CF3:

[0031] CF≥CF1: Trigger a first-level response; CF2≤CF<CF1: Trigger a second-level response; CF3≤CF<CF2: Trigger a third-level response; CF<CF3: Determine normal operation and do not trigger a response;

[0032] The confidence threshold is dynamically corrected according to Δ=k(target false alarm rate - current false alarm rate), the current false alarm rate is obtained through statistics, and Δ is the change amount of the confidence threshold; k is a coefficient that controls the proportional relationship between the false alarm rate and the change amount of the confidence threshold;

[0033] The early warning execution terminal is used to disconnect the DC output circuit of the faulty photovoltaic module when a first-level response is triggered; when a second-level response is triggered, give an audible and visual alarm and push a maintenance work order to the operation and maintenance management system; when a third-level response is triggered, add the component ID to the "observation queue", re-evaluate the status of the component every once in a while, highlight the position of the component in the photovoltaic array thermal map, and generate preliminary maintenance suggestions in combination with the expert rule base and historical cases.

[0034] Further, the multi-source sensor module includes a current sensor, a voltage sensor, an infrared thermal imaging sensor and an environmental sensor group;

[0035] The current sensor adopts a Hall or shunt current sensor, and is connected in series to the DC output circuit of each photovoltaic module to collect component-level current data in real time;

[0036] The voltage sensor adopts an isolated voltage sensor and is connected in parallel across both ends of each photovoltaic module to collect component-level voltage data in real time; the component-level voltage data and the component-level current data together constitute the I-V characteristic curve data;

[0037] The infrared thermal imaging sensor is installed on a two-degree-of-freedom electrically adjustable bracket above the photovoltaic array, and obtains the thermal imaging map of each photovoltaic module surface according to the preset scanning trajectory and set time period;

[0038] The environmental sensor group includes an irradiance meter, an air temperature and humidity meter and an ultrasonic wind speed meter, and is used to synchronously collect environmental data.

[0039] Furthermore, the edge data acquisition unit is an integrated industrial control box, which contains a multiplexer, a high-precision ADC circuit, an isolated RS-485 bus, a GPS synchronization module, a microcontroller, and a 4G / 5G narrowband IoT module.

[0040] The input terminals of the multiplexer are connected to a current sensor, a voltage sensor, and an environmental sensor group, respectively, to switch different analog signal channels and output to a high-precision ADC circuit.

[0041] The high-precision ADC circuit performs A / D conversion on the analog signal output from the multiplexer and sends the digitized result to the microcontroller.

[0042] The isolated RS-485 bus is connected to the microcontroller via opto-isolation and is used for differential serial communication with the infrared thermal imaging sensor.

[0043] The GPS synchronization module provides the microcontroller with 1PPS pulse and UTC time information;

[0044] The microcontroller is used to control the channel switching of the multiplexer and the ADC sampling timing; it buffers and formats the data stream from the high-precision ADC circuit and the isolated RS-485 bus, and adds a unified timestamp to each group of acquired data based on the pulse. At the same time, it adds corresponding component ID and location information to each group of acquired data; it performs Savitzky-Golay smoothing on the IV characteristic curve, and then extracts the inflection point IV data of the IV characteristic curve using the second derivative zero-crossing method. Missing points are corrected by neighborhood linear interpolation. The extracted inflection point IV data is used to replace the original IV characteristic curve data and uploaded to the cloud analysis and early warning platform.

[0045] The 4G / 5G narrowband IoT module is used to upload timestamped data streams to the cloud-based analysis and early warning platform.

[0046] Furthermore, the cloud-based analysis and early warning platform includes a data preprocessing module, a temperature field extraction module, a multimodal spatiotemporal fusion module, a deep learning fault diagnosis module, and an early warning decision module;

[0047] The data preprocessing module preprocesses the timestamped data stream and resamples the sampling sequence composed of IV data at each inflection point and the environmental data sequence to the same frequency based on the timestamps.

[0048] The temperature field extraction module first performs perspective correction based on the calibrated extrinsic parameter matrix for the surface thermal imaging image of the photovoltaic module, then divides the image by the ROI of the module frame to obtain a single module thermal image, and uniformly downsamples the single module thermal image to obtain the temperature field matrix of the corresponding photovoltaic module.

[0049] The multimodal spatiotemporal fusion module uses timestamps as a reference and employs sliding window linear interpolation to upsample the temperature field matrix sequence to the same frequency as the sampling sequence composed of inflection point IV data at each time step, and fuses the inflection point IV data, temperature field matrix and environmental data at the same time step into a multimodal sample.

[0050] The deep learning fault diagnosis module has a built-in trained CNN-LSTM hybrid neural network. The input is multimodal samples, and the output is a fault category probability vector and a fault confidence CF is calculated. The fault category probability vector P = (p_normal, p_degradation, p_hot-spot), where p_normal represents the probability of the component operating normally, p_degradation represents the probability of the component decaying, and p_hot-spot represents the probability of hot spot failure. The fault confidence CF = 1 - p_normal.

[0051] The early warning decision module is used to generate graded response instructions based on the fault confidence level CF and dynamic confidence thresholds CF1 to CF3 and push them to the early warning execution terminal.

[0052] Furthermore, the early warning execution terminal is a fault handling gateway, including a relay control circuit, an audible and visual alarm, and a mobile APP, and also has an interface for connecting to the operation and maintenance management system;

[0053] When a Level 1 response is triggered, the fault handling gateway receives the component ID and disconnect command from the cloud analysis and early warning platform, drives the relay control circuit to cut off the DC output circuit of the corresponding photovoltaic module, and displays a pop-up message in the mobile APP stating "Component ID-xxx has been isolated".

[0054] When a Level 2 response is triggered, the cloud-based analysis and early warning platform automatically generates a standardized maintenance work order based on the component ID, array row and column number, and predicted fault type, and pushes it to the operation and maintenance management system. The operation and maintenance management system then pushes the work order information to the duty screen in the operation and maintenance center and the mobile APP in real time. At the same time, the fault handling gateway drives the audible and visual alarm installed in the operation and maintenance center to issue an alarm for a set duration.

[0055] When a Level 3 response is triggered, the cloud-based analysis and early warning platform adds the component ID to the "observation queue" and reassesses the component's status at regular intervals. The mobile app highlights the component's location on the array heat map and automatically generates preliminary maintenance suggestions based on the expert rule base and historical cases.

[0056] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0057] (1) Component-level monitoring improves fault location accuracy by 20 times compared to string-level monitoring.

[0058] (2) Multimodal data fusion combined with CNN-LSTM hybrid neural network, dual-branch structure simultaneously captures spatial thermal anomalies and temporal performance degradation, reducing the false alarm rate for rainy weather from 42% to 5.3%, and increasing the detection rate of faults such as microcracks in battery cells from below 60% to 92.4%.

[0059] (3) Extracting inflection point IV data to replace the full data upload effectively reduces bandwidth.

[0060] (4) The confidence threshold is dynamically corrected and the response level is automatically selected according to the fault confidence. This can suppress the false triggering of fixed thresholds in severe weather (such as rainy weather), avoid blind tripping, improve the operation and maintenance efficiency by 3 times, and reduce unplanned downtime by 70%.

[0061] In summary, this invention solves the problems of high missed detection rate of latent faults, high false alarm rate of fixed thresholds, large bandwidth pressure of full data transmission, and delayed fault handling caused by single electrical parameter detection in traditional photovoltaic monitoring, thus effectively improving the early fault detection rate. Attached Figure Description

[0062] Figure 1 This is the CNN-LSTM hybrid neural network structure in the embodiments of the present invention;

[0063] Figure 2 This is a schematic diagram of the early warning decision-making process in an embodiment of the present invention;

[0064] Figure 3 This is an architecture diagram of a photovoltaic module early fault monitoring and warning system based on deep learning, provided by an embodiment of the present invention.

[0065] Figure 4 This is the data acquisition and transmission process of the photovoltaic module early fault monitoring and early warning system in this embodiment of the invention;

[0066] Figure 5 This is the data preprocessing and fusion process of the photovoltaic module early fault monitoring and early warning system in this embodiment of the invention. Detailed Implementation

[0067] The invention will now be further described with reference to the accompanying drawings.

[0068] Example 1

[0069] Example 1 provides a method for early fault monitoring and warning of photovoltaic modules based on deep learning, including the following steps:

[0070] (1) Collect IV characteristic curve data, surface thermal imaging map and environmental data of each photovoltaic module. The environmental data includes irradiance, ambient temperature, ambient humidity and wind speed.

[0071] (2) A unified timestamp is added to each set of collected data. Simultaneously, each set of collected data is associated with its corresponding component ID and location information to ensure that all collected data can accurately identify its corresponding photovoltaic component and its location within the photovoltaic array. This information is crucial for subsequent data analysis, fault diagnosis, and localization, especially in large-scale photovoltaic arrays. By providing a component ID and location index for each data point, accurate references can be provided for subsequent time-space alignment and multimodal data fusion.

[0072] The IV characteristic curves were smoothed using Savitzky-Golay, and then the inflection point IV data of the IV characteristic curves were extracted using the second derivative zero-crossing method to replace the original IV characteristic curve data. Missing points were corrected using neighborhood linear interpolation.

[0073] (3) Preprocessing and temporal-spatial alignment are performed on the timestamped data stream. Data preprocessing includes noise reduction, missing value imputation, outlier filtering, and normalization. Temporal alignment is achieved through a unified timestamp to ensure that data from different sensors are synchronized in time. Spatial alignment is achieved through a pre-established "component physical coordinate-pixel coordinate mapping table" to convert the pixel coordinates of the thermal image into the actual position of the photovoltaic module in the photovoltaic array.

[0074] A single-module thermal image is obtained by segmenting the surface thermal image of the photovoltaic module according to the ROI of the module border. The temperature field matrix of the corresponding photovoltaic module is then obtained from the single-module thermal image. The inflection point IV data, temperature field matrix, and environmental data at the same time step are fused into a multimodal sample. By aligning the data from different sensors in a unified spatial coordinate system, it is ensured that the multimodal data can accurately correspond to the location of each photovoltaic module, thereby improving the accuracy of fault diagnosis.

[0075] (4) Call the trained CNN-LSTM hybrid neural network, take multimodal samples as input, output the fault category probability vector and calculate the fault confidence CF.

[0076] like Figure 1As shown in the figure, the CNN-LSTM hybrid neural network includes a CNN branch, an LSTM branch, and an output layer: The CNN branch uses an improved ResNet-18 backbone, and a Squeeze-and-Excitation channel attention module is inserted into each residual block. The channel attention mechanism can enhance the image feature extraction ability. The input of the CNN branch is a three-channel tensor obtained by stacking "current-voltage values, temperature field matrix", and the output is a 256-dimensional spatial feature vector. The LSTM branch contains two layers of bidirectional LSTM with 128 units each. The input of the LSTM branch is a time series composed of the feature data output by the CNN branch at each moment, and the output is a 256-dimensional time series feature vector. The output layer performs feature concatenation on the spatial feature vector and the time series feature vector to obtain a 512-dimensional fusion feature, and then passes through two fully connected networks (Dropout 0.3) and Softmax activation to output the fault category probability vector P = (p_normal, p_degradation, p_hot-spot), and calculates the fault confidence CF = 1 - p_normal. p_normal represents the probability of normal operation of the component, p_degradation represents the probability of degradation failure of the component, and p_hot-spot represents the probability of hot spot failure.

[0077] Training process of the CNN-LSTM hybrid neural network: A total of 24,000 measured and labeled data from 8 photovoltaic power plants in Jiangsu from January 2023 to April 2025 are used as the training set, and the training / validation / test sets are divided according to 70% / 15% / 15%; The optimizer is Adam, the learning rate is 1×10 -4 , the batch size is 64, the maximum is 100 epochs, and the early stopping tolerance is 10 epochs; For class imbalance, focal loss (γ = 2) is used.

[0078] (5) As Figure 2 shown, according to the fault confidence CF and the dynamic confidence thresholds CF1 to CF3, a hierarchical response instruction is generated. CF1 is initially configured as 0.90, CF2 is initially configured as 0.70, and CF3 is initially configured as 0.50.

[0079] CF ≥ CF1: Trigger a first-level response and disconnect the DC output circuit of the faulty photovoltaic component;

[0080] CF2 ≤ CF < CF1: Trigger a second-level response, give an audible and visual alarm and push a maintenance work order to the operation and maintenance management system;

[0081] CF3 ≤ CF < CF2: Trigger a third-level response, add the component ID to the "observation queue", re-evaluate the status of the component every 10 minutes, highlight the position of the component in the photovoltaic array heat map, and at the same time generate preliminary maintenance suggestions by combining the expert rule base and historical cases;

[0082] CF<CF3: It is determined to be operating normally and no response is triggered.

[0083] Statistically calculate the false alarm rate of rainy and cloudy conditions in the past 7 days every day, and dynamically correct each confidence threshold according to Δ=k(target false alarm rate - current false alarm rate). Δ is the change amount of the confidence threshold, which is used to dynamically adjust the confidence threshold. k is a coefficient that controls the proportional relationship between the false alarm rate and the change amount of the confidence threshold, and determines the response speed of the system to the change of the false alarm rate.

[0084] The position of the component is inversely calculated based on the pixel coordinates of the component border ROI in the temperature field matrix and the pre-established "component physical coordinate - pixel coordinate mapping table" to obtain the array row - column - component ID.

[0085] The preliminary maintenance suggestions are generated after retrieval according to the pre-constructed expert knowledge base combined with the fault labels.

[0086] Embodiment 2

[0087] As Figures 1 to 5 shown, Embodiment 2 provides a photovoltaic component early fault monitoring and warning system based on deep learning, including a multi-source sensor module, an edge data acquisition unit, a cloud analysis and warning platform, and a warning execution terminal.

[0088] (1) Multi-source sensor module

[0089] The multi-source sensor module is used to collect the I-V characteristic curve data, surface thermal imaging diagrams, and environmental data of each photovoltaic component. Specifically, the multi-source sensor module includes a current sensor, a voltage sensor, an infrared thermal imaging sensor, and an environmental sensor group.

[0090] The current sensor adopts a Hall-type or shunt-type current sensor, which is connected in series to the DC output loop of each photovoltaic component to collect component-level current data in real time. The measurement range of the current sensor is 0 - 15A, the accuracy is ±0.5%, and millisecond-level current sampling is achieved.

[0091] The voltage sensor adopts an isolated voltage sensor, which is connected in parallel across both ends of each photovoltaic component to collect component-level voltage data in real time. The measurement range of the voltage sensor is 0 - 50V, the accuracy is ±0.5%, and millisecond-level voltage sampling is achieved. The component-level voltage data and the component-level current data together constitute the I-V characteristic curve data.

[0092] The infrared thermal imaging sensor is installed on a two-degree-of-freedom electric adjustable bracket (pitch 0° - 45°, azimuth ±90°, positioning accuracy ≤0.5°, this two-degree-of-freedom electric adjustable bracket is an existing technology) above the photovoltaic array, and obtains the surface thermal imaging diagrams of each photovoltaic component according to the preset scanning trajectory and a 5-minute cycle.

[0093] The environmental sensor array includes a radiometer, an air temperature and humidity meter, and an ultrasonic anemometer, used to collect environmental data synchronously.

[0094] (II) Edge Data Acquisition Unit

[0095] The edge data acquisition unit is connected to the multi-source sensor module to perform 24-bit A / D conversion on the acquired analog signals; a unified timestamp is added to each set of acquired data, and each set of acquired data is associated with the corresponding component ID and location information; the inflection point IV data of the IV characteristic curve is extracted and used to replace the original IV characteristic curve data to be uploaded to the cloud analysis and early warning platform.

[0096] Specifically, the edge data acquisition unit is an integrated industrial control box, which contains a multiplexer, a high-precision ADC circuit, an isolated RS-485 bus, a GPS synchronization module, a microcontroller (MCU), and a 4G / 5G narrowband IoT module.

[0097] The input terminals of the multiplexer are connected to a current sensor, a voltage sensor, and an environmental sensor group, respectively, to switch different analog signal channels and output to a high-precision ADC circuit.

[0098] The high-precision ADC circuit adopts a 24-bit Σ-Δ architecture to perform A / D conversion on the analog signal output from the multiplexer and send the digitized result to the microcontroller MCU.

[0099] The isolated RS-485 bus is connected to the microcontroller (MCU) via opto-isolation and is used for differential serial communication with the infrared thermal imaging sensor to ensure anti-interference and electrical safety for long-distance data transmission.

[0100] The GPS synchronization module provides the microcontroller (MCU) with 1PPS (Pulse-Per-Second) pulses and UTC time information.

[0101] The microcontroller controls the channel switching of the multiplexer and the ADC sampling timing. It buffers and formats the data streams from the high-precision ADC circuit and the isolated RS-485 bus, adding a unified timestamp to each group of acquired data based on the pulses. Simultaneously, it adds corresponding component IDs and location information to each group of acquired data, ensuring that each data point accurately identifies the corresponding photovoltaic module and its physical location within the photovoltaic array. This is crucial for subsequent data analysis, fault diagnosis, and maintenance. Furthermore, it performs Savitzky-Golay smoothing on the IV characteristic curves, then extracts the inflection point IV data using the second derivative zero-crossing method, correcting missing points with neighborhood linear interpolation. The extracted inflection point IV data is used to replace the original IV characteristic curve data uploaded to the cloud-based analysis and early warning platform, effectively reducing uplink bandwidth compared to uploading the entire dataset.

[0102] The extraction and replacement functions of the inflection point I-V data are performed by the edge data acquisition unit, which helps reduce the amount of data uploaded and improve bandwidth efficiency. The temperature field matrix involves the analysis and processing of thermal imaging maps, which requires complex calculations and analyses of large-scale image data. Especially, it is necessary to segment according to the component border ROI (Region of Interest) and extract the corresponding temperature field data from it. Such image processing tasks require high computing power, and edge devices usually cannot provide sufficient processing power to complete complex image analysis tasks. Therefore, the analysis and processing of thermal imaging maps need to be performed by the cloud analysis and warning platform.

[0103] The 4G / 5G narrowband IoT module is used to upload the time-stamped data stream to the cloud analysis and warning platform, and the transmission protocol uses the MQTT-SN lightweight protocol.

[0104] (III) Cloud Analysis and Warning Platform

[0105] The cloud analysis and warning platform is used to preprocess and time-space align the time-stamped data stream. Data preprocessing includes denoising, missing value imputation, outlier filtering, and normalization. Time alignment is achieved through a unified timestamp to ensure the synchronization of data from different sensors in time. Space alignment is achieved through a pre-established "component physical coordinate - pixel coordinate mapping table" to convert the pixel coordinates from the infrared thermal imaging sensor into the actual position of the photovoltaic module in the array.

[0106] The thermal imaging map of the photovoltaic module surface is segmented according to the component border ROI to obtain a single-component thermal map, and the temperature field matrix of the corresponding photovoltaic module is obtained according to the single-component thermal map. The inflection point I-V data, temperature field matrix, and environmental data at the same time step are fused into a multi-modal sample.

[0107] Call the trained CNN-LSTM hybrid neural network, take the multi-modal sample as the input, output the fault category probability vector and calculate the fault confidence CF; generate a hierarchical response instruction according to the fault confidence CF and the dynamic confidence thresholds CF1 to CF3. CF1 is initially configured as 0.90, CF2 is initially configured as 0.70, and CF3 is initially configured as 0.50:

[0108] CF≥CF1: Trigger a first-level response; CF2≤CF<CF1: Trigger a second-level response; CF3≤CF<CF2: Trigger a third-level response; CF<CF3: Determine normal operation and do not trigger a response;

[0109] The false alarm rate under rainy conditions over the past 7 days is calculated daily, and the confidence thresholds are dynamically adjusted according to Δ = k(target false alarm rate - current false alarm rate). Δ represents the change in the confidence threshold, used for dynamic adjustment. k is a coefficient that controls the proportional relationship between the false alarm rate and the change in the confidence threshold, determining the system's response speed to changes in the false alarm rate.

[0110] Specifically, the cloud-based analysis and early warning platform includes a data preprocessing module, a temperature field extraction module, a multimodal spatiotemporal fusion module, a deep learning fault diagnosis module, and an early warning decision module.

[0111] The data preprocessing module preprocesses the timestamped data stream, including noise reduction, missing value imputation, 3-σ outlier filtering, and 0-1 normalization. It also resamples the sampling sequence composed of inflection point IV data to 1Hz and the environmental data sequence to 1Hz based on the timestamps. Furthermore, the data preprocessing module is responsible for temporal and spatial alignment of the data, ensuring that data from different sensors accurately corresponds to the physical location of the photovoltaic modules. Specifically, temporal alignment is achieved through a unified timestamp; the pixel coordinates of the thermal image are calculated using a pre-established "module physical coordinate-pixel coordinate mapping table" to obtain the array row and column numbers and module IDs, thereby ensuring spatial data alignment and providing accurate spatial references for subsequent data fusion and fault diagnosis.

[0112] The temperature field extraction module first performs perspective correction based on the calibrated extrinsic parameter matrix for the 640×512 pixel thermal image of the photovoltaic module surface uploaded by the infrared thermal imaging sensor. Then, it divides the module into individual module thermal images according to the ROI of the module border and downsamples the individual module thermal images into a 32×32 pixel matrix, which is the temperature field matrix of the corresponding photovoltaic module.

[0113] The multimodal spatiotemporal fusion module uses timestamps as a reference and employs sliding window linear interpolation to upsample the 0.2Hz temperature field matrix sequence to 1Hz. It also fuses the inflection point IV data, temperature field matrix, and environmental data at the same time step into a multimodal sample, achieving cross-source spatiotemporal alignment.

[0114] The deep learning fault diagnosis module has a built-in trained CNN-LSTM hybrid neural network. The input is multimodal samples, and the output is a fault category probability vector and a fault confidence CF is calculated. The fault category probability vector P = (p_normal, p_degradation, p_hot-spot), where p_normal represents the probability of the component operating normally, p_degradation represents the probability of the component decaying, and p_hot-spot represents the probability of hot spot failure. The fault confidence CF = 1 - p_normal.

[0115] The early warning decision module is used to generate graded response instructions based on the fault confidence level CF and dynamic confidence thresholds CF1 to CF3 and push them to the early warning execution terminal.

[0116] (iv) Early warning execution terminal

[0117] The early warning execution terminal is used to disconnect the DC output circuit of the faulty photovoltaic module when a Level 1 response is triggered; when a Level 2 response is triggered, it will trigger an audible and visual alarm and push a maintenance work order to the operation and maintenance management system; when a Level 3 response is triggered, it will add the module ID to the "observation queue", re-evaluate the status of the module at regular intervals, highlight the location of the module in the photovoltaic array heat map, and generate preliminary maintenance suggestions by combining the expert rule base and historical cases.

[0118] Specifically, the early warning execution terminal is a fault handling gateway, which includes a relay control circuit, an audible and visual alarm, and a mobile APP, and also has an interface for connecting to the maintenance management system (CMMS).

[0119] The relay control circuit adopts the existing photovoltaic DC side circuit breaking control circuit, including optocoupler isolators, Darlington transistor arrays, and solid-state relay (SSR) or magnetic latching relay coils.

[0120] Optocouplers (such as PC817, ADuM1201, etc.) are used to achieve electrical isolation between the MCU-GPIO and the high-voltage side.

[0121] Darlington transistor arrays (such as the ULN2003A) are used to amplify isolated 3.3V / 5V logic levels to a current level capable of driving relay coils.

[0122] Solid-state relay (SSR) or magnetic latching relay coils are controlled by the collector current of a Darlington transistor array to enable rapid switching on and off of the DC output circuit of photovoltaic modules.

[0123] When a Level 1 response is triggered, the fault handling gateway receives the component ID and disconnect command from the cloud-based analysis and early warning platform, drives the relay control circuit to cut off the DC output circuit of the corresponding photovoltaic module, and displays a pop-up message in the mobile app stating "Component ID-xxx has been isolated".

[0124] When a Level 2 response is triggered, the cloud-based analysis and early warning platform automatically generates a standardized maintenance work order (including fields such as site, array, component, fault type, confidence level, and suggested time limit) based on the component ID, array row and column number, and predicted fault type. The work order is then pushed to the operation and maintenance management system via HTTPS API. The operation and maintenance management system then pushes the work order information to the operation and maintenance center's duty screen and mobile APP in real time. At the same time, the fault handling gateway drives the audible and visual alarm installed in the operation and maintenance center to issue a continuous 5-second alarm.

[0125] When a Level 3 response is triggered, the cloud-based analysis and early warning platform adds the component ID to the "observation queue" and reassesses the component's status every 10 minutes. The mobile app highlights the component's location on the array heat map and automatically generates preliminary maintenance suggestions based on the expert rule base and historical cases.

[0126] The following is an example of preliminary repair suggestions:

[0127] • If the predicted fault is “potential hot spot”, it is recommended to check the junction box and local shielding.

[0128] • If the predicted fault is “power decay”, it is recommended to check for welding fatigue and retest the IV curve.

[0129] The component position is calculated by back-calculating the pixel coordinates of the component's ROI in the temperature field matrix and the pre-established "component physical coordinate-pixel coordinate mapping table" to obtain the array row-column-component ID.

[0130] Preliminary repair suggestions are generated after searching a pre-built expert knowledge base and combining it with fault tags.

Claims

1. A method for early fault monitoring and warning of photovoltaic modules based on deep learning, characterized in that, Including: (1) Collecting I-V characteristic curve data, surface thermal imaging maps, and environmental data of each photovoltaic module; (2) Adding a unified timestamp to each set of collected data. At the same time, associating each set of collected data with the corresponding module ID and location information; extracting the inflection point I-V data of the I-V characteristic curve; (3) Preprocessing the timestamped data stream and performing time-space alignment; segmenting the surface thermal imaging map of the photovoltaic module according to the ROI of the module border to obtain a single-module thermal map, and obtaining the temperature field matrix of the corresponding photovoltaic module from the single-module thermal map; fusing the inflection point I-V data, temperature field matrix, and environmental data at the same time step into a multi-modal sample; (4) Invoking a trained CNN-LSTM hybrid neural network, using the multi-modal sample as input, outputting a fault category probability vector, and calculating the fault confidence CF; The CNN-LSTM hybrid neural network includes a CNN branch, an LSTM branch, and an output layer: The CNN branch uses an improved ResNet-18 backbone, and inserts a Squeeze-and-Excitation channel attention module in each residual block; the input of the CNN branch is a three-channel tensor stacked by "current voltage value, temperature field matrix", and the output is a spatial feature vector; the LSTM branch contains two layers of bidirectional LSTM; the input of the LSTM branch is a time series composed of the feature data output by the CNN branch at each moment, and the output is a temporal feature vector; the output layer performs feature splicing on the spatial feature vector and the temporal feature vector to obtain a fused feature, and then passes through two layers of fully connected networks and Softmax activation to output a fault category probability vector P=(p_normal, p_degradation, p_hot-spot), and calculates the fault confidence CF = 1 - p_normal; p_normal represents the probability of the module operating normally, p_degradation represents the probability of the module suffering from degradation faults, and p_hot-spot represents the probability of hot spot faults; (5) Generating a hierarchical response instruction according to the fault confidence CF and dynamic confidence thresholds CF1 to CF3; CF ≥ CF1: Trigger a first-level response to disconnect the DC output loop of the faulty photovoltaic module; CF2 ≤ CF < CF1: Trigger a second-level response, with an audible and visual alarm and push a maintenance work order to the operation and maintenance management system; CF3 ≤ CF < CF2: Trigger a third-level response, add the module ID to the "observation queue", re-evaluate the status of the module at regular intervals, highlight the location of the module in the photovoltaic array thermal map, and generate preliminary maintenance suggestions in combination with the expert rule base and historical cases; CF < CF3: Determine as normal operation, without triggering a response; The confidence threshold is dynamically corrected according to Δ = k(target false alarm rate - current false alarm rate), and the current false alarm rate is obtained through statistics; Δ is the change amount of the confidence threshold; k is a coefficient that controls the proportional relationship between the false alarm rate and the change amount of the confidence threshold.

2. The method for early fault monitoring and warning of photovoltaic modules based on deep learning according to claim 1, characterized in that, In step (2), the I-V characteristic curve is smoothed using Savitzky-Golay smoothing, and then the inflection point I-V data of the I-V characteristic curve is extracted by the second derivative zero-crossing method to replace the original I-V characteristic curve data, and the missing points are corrected by neighborhood linear interpolation.

3. The method for early fault monitoring and warning of photovoltaic modules based on deep learning according to claim 1, characterized in that, In step (3), data preprocessing includes denoising, missing value imputation, outlier filtering, and normalization.

4. The method for early fault monitoring and warning of photovoltaic modules based on deep learning according to claim 1, characterized in that, In step (5), the component position is calculated inversely according to the pixel coordinates of the component border ROI in the temperature field matrix and the pre-established "component physical coordinate - pixel coordinate mapping table" to obtain the array row-column-component ID; the preliminary maintenance suggestions are generated after retrieval according to the pre-constructed expert knowledge base combined with the fault label.

5. The method for early fault monitoring and warning of photovoltaic modules based on deep learning according to claim 1, characterized in that, In step (5), the initial configuration of CF1 is 0.90, the initial configuration of CF2 is 0.70, and the initial configuration of CF3 is 0.

50.

6. A photovoltaic module early fault monitoring and warning system based on deep learning, characterized in that, It includes a multi-source sensor module, an edge data acquisition unit, a cloud analysis and warning platform, and a warning execution terminal; The multi-source sensor module is used to collect the I-V characteristic curve data, surface thermal imaging map, and environmental data of each photovoltaic module. The edge data acquisition unit is connected to the multi-source sensor module and is used to perform A / D conversion on the collected analog signals; add a unified timestamp to each group of collected data, and at the same time, associate each group of collected data with the corresponding component ID and location information; extract the inflection point I-V data of the I-V characteristic curve for replacing the original I-V characteristic curve data and uploading it to the cloud analysis and warning platform. The cloud analysis and warning platform is used to preprocess the time-stamped data stream and perform time-space alignment; segment the surface thermal imaging map of the photovoltaic module according to the component border ROI to obtain a single-component thermal map, and obtain the temperature field matrix of the corresponding photovoltaic module according to the single-component thermal map; fuse the inflection point I-V data, temperature field matrix, and environmental data at the same time step into a multi-modal sample; call the trained CNN-LSTM hybrid neural network, use the multi-modal sample as the input, output the fault category probability vector and calculate the fault confidence CF; generate a hierarchical response instruction according to the fault confidence CF and the dynamic confidence thresholds CF1 to CF3: CF≥CF1: Trigger a first-level response; CF2≤CF<CF1: Trigger a second-level response; CF3≤CF<CF2: Trigger a third-level response; CF<CF3: Determine normal operation and do not trigger a response; The confidence threshold is dynamically corrected according to Δ=k(target false alarm rate - current false alarm rate), the current false alarm rate is obtained through statistics, and Δ is the change amount of the confidence threshold; k is a coefficient that controls the proportional relationship between the false alarm rate and the change amount of the confidence threshold; The warning execution terminal is used to disconnect the DC output circuit of the faulty photovoltaic module when a first-level response is triggered; when a second-level response is triggered, give an audible and visual alarm and push a maintenance work order to the operation and maintenance management system; when a third-level response is triggered, add the component ID to the "observation queue", re-evaluate the component status every once in a while, highlight the position of the component in the photovoltaic array thermal map, and at the same time generate preliminary maintenance suggestions in combination with the expert rule base and historical cases.

7. The photovoltaic module early fault monitoring and warning system based on deep learning according to claim 6, characterized in that, The multi-source sensor module includes a current sensor, a voltage sensor, an infrared thermal imaging sensor, and an environmental sensor group; The current sensor is a Hall effect or shunt current sensor, which is connected in series to the DC output circuit of each photovoltaic module to collect module-level current data in real time. The voltage sensor is an isolated voltage sensor, which is connected in parallel to both ends of each photovoltaic module to collect module-level voltage data in real time; Component-level voltage data and component-level current data together constitute the IV characteristic curve data; The infrared thermal imaging sensor is mounted on a two-degree-of-freedom electrically adjustable bracket above the photovoltaic array, and acquires thermal images of the surface of each photovoltaic module according to a preset scanning trajectory and a set time period. The environmental sensor group includes an irradiance meter, an air temperature and humidity meter, and an ultrasonic anemometer, which are used to collect environmental data synchronously.

8. The photovoltaic module early fault monitoring and warning system based on deep learning according to claim 7, characterized in that, The edge data acquisition unit is an integrated industrial control box, which contains a multiplexer, a high-precision ADC circuit, an isolated RS-485 bus, a GPS synchronization module, a microcontroller, and a 4G / 5G narrowband IoT module. The input terminals of the multiplexer are connected to a current sensor, a voltage sensor, and an environmental sensor group, respectively, to switch different analog signal channels and output to a high-precision ADC circuit. The high-precision ADC circuit performs A / D conversion on the analog signal output from the multiplexer and sends the digitized result to the microcontroller. The isolated RS-485 bus is connected to the microcontroller via opto-isolation and is used for differential serial communication with the infrared thermal imaging sensor. The GPS synchronization module provides the microcontroller with 1PPS pulse and UTC time information; The microcontroller is used to control the channel switching of the multiplexer and the ADC sampling timing; The data streams from the high-precision ADC circuit and the isolated RS-485 bus are buffered and formatted, and a unified timestamp is added to each group of acquired data based on the pulse. At the same time, corresponding component IDs and location information are added to each group of acquired data. The IV characteristic curve is smoothed by Savitzky-Golay, and then the inflection point IV data of the IV characteristic curve is extracted by the second derivative zero-crossing method. Missing points are corrected by neighborhood linear interpolation. The extracted inflection point IV data is used to replace the original IV characteristic curve data and uploaded to the cloud analysis and early warning platform. The 4G / 5G narrowband IoT module is used to upload timestamped data streams to the cloud-based analysis and early warning platform.

9. The photovoltaic module early fault monitoring and warning system based on deep learning according to claim 8, characterized in that, The cloud-based analysis and early warning platform includes a data preprocessing module, a temperature field extraction module, a multimodal spatiotemporal fusion module, a deep learning fault diagnosis module, and an early warning decision module. The data preprocessing module preprocesses the timestamped data stream and resamples the sampling sequence composed of IV data at each inflection point and the environmental data sequence to the same frequency based on the timestamps. The temperature field extraction module first performs perspective correction based on the calibrated extrinsic parameter matrix for the surface thermal imaging image of the photovoltaic module, then divides the image by the ROI of the module frame to obtain a single module thermal image, and uniformly downsamples the single module thermal image to obtain the temperature field matrix of the corresponding photovoltaic module. The multimodal spatiotemporal fusion module uses timestamps as a reference and employs sliding window linear interpolation to upsample the temperature field matrix sequence to the same frequency as the sampling sequence composed of inflection point IV data at each time step, and fuses the inflection point IV data, temperature field matrix and environmental data at the same time step into a multimodal sample. The deep learning fault diagnosis module has a built-in trained CNN-LSTM hybrid neural network. The input is multimodal samples, and the output is a fault category probability vector and a fault confidence CF is calculated. The fault category probability vector P = (p_normal, p_degradation, p_hot-spot), where p_normal represents the probability of the component operating normally, p_degradation represents the probability of the component decaying, and p_hot-spot represents the probability of hot spot failure. The fault confidence CF = 1 - p_normal. The early warning decision module is used to generate graded response instructions based on the fault confidence level CF and dynamic confidence thresholds CF1 to CF3 and push them to the early warning execution terminal.

10. The photovoltaic module early fault monitoring and warning system based on deep learning according to claim 9, characterized in that, The early warning execution terminal is a fault handling gateway, which includes a relay control circuit, an audible and visual alarm, and a mobile APP, and also has an interface for connecting to the operation and maintenance management system. When a Level 1 response is triggered, the fault handling gateway receives the component ID and disconnect command from the cloud analysis and early warning platform, drives the relay control circuit to cut off the DC output circuit of the corresponding photovoltaic module, and displays a pop-up message in the mobile APP stating "Component ID-xxx has been isolated". When a Level 2 response is triggered, the cloud-based analysis and early warning platform automatically generates a standardized maintenance work order based on the component ID, array row and column number, and predicted fault type, and pushes it to the operation and maintenance management system. The operation and maintenance management system then pushes the work order information to the operation and maintenance center's duty screen and mobile APP in real time. The fault handling gateway simultaneously drives the audible and visual alarm installed in the operation and maintenance center to issue an alarm for a set duration. When a Level 3 response is triggered, the cloud-based analysis and early warning platform adds the component ID to the "observation queue" and reassesses the component's status at regular intervals. The mobile app highlights the component's location on the array heatmap and automatically generates preliminary maintenance suggestions based on the expert rule base and historical cases.

Images