New energy power generation monitoring system and method

Abstract

The invention discloses a new energy power generation monitoring system and method, and relates to the technical field of new energy power generation, and the system comprises the following components: a multi-modal data collection module which integrates unmanned plane high-definition shooting, a temperature sensor, a vibration sensor and a high-sensitivity sound sensor; according to the photovoltaic panel state monitoring method, the photovoltaic panel surface details, real-time temperature vibration data and operation sound are captured in an all-around mode, an innovative multi-mode data comprehensive analysis technology is introduced, all-around and high-precision monitoring of the state of the power generation equipment is achieved, and a traditional monitoring method usually only depends on a single data type, so that the monitoring efficiency is greatly improved. The multi-modal data acquisition module integrates unmanned aerial vehicle high-definition shooting, a temperature sensor, a vibration sensor and a high-sensitivity sound sensor, image, temperature, vibration and sound multi-source data of the photovoltaic panel can be acquired at the same time, the comprehensive analysis of the multi-modal data makes up for the deficiency of a single data source, and the multi-modal data acquisition module can be used for acquiring the multi-source data of the photovoltaic panel. And mutual verification and cross verification among data are also carried out.

Description

[0001] Compared with the prior art, the new energy power generation monitoring system and method have the following beneficial effects:

[0002] 1. The present invention realizes all-round and high-precision monitoring of the status of power generation equipment by introducing innovative multimodal data comprehensive analysis technology. Traditional monitoring methods often rely only on a single data type, while the multimodal data acquisition module of the present invention integrates drone high-definition photography, temperature sensors, vibration sensors and high-sensitivity sound sensors, and can simultaneously obtain multi-source data such as image, temperature, vibration and sound of photovoltaic panels. This comprehensive analysis of multimodal data not only makes up for the shortcomings of a single data source, but also significantly improves the accuracy and reliability of fault identification through mutual verification and cross-validation between data.

[0003] 2. The present invention uses advanced image recognition, temperature and vibration data analysis, and sound spectrum analysis technology to perform deep processing and feature extraction on the collected multi-source data. By constructing a fusion model, the image, sound, temperature, and vibration multimodal data are aligned according to timestamps, and their respective features that are useful for fault diagnosis are extracted to form a comprehensive feature vector. This deep fusion of multimodal data not only achieves a comprehensive description of the equipment status, but also realizes early detection, accurate positioning, and cause diagnosis of faults through intelligent algorithms.

[0004] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0006] Figure 1 It is a flow chart of a new energy power generation monitoring system;

[0007] Figure 2 The figure is a flow chart of a new energy power generation monitoring method. DETAILED DESCRIPTION

[0008] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0009] Embodiment 1:

[0010] This embodiment provides a new energy power generation monitoring system, which integrates drone high-definition photography, temperature sensors, vibration sensors and high-sensitivity sound sensors to achieve detailed capture of the surface of photovoltaic panels and real-time monitoring of equipment operation. The data processing and analysis module uses image recognition, temperature and vibration data analysis, and sound spectrum analysis technology to comprehensively extract and integrate multi-source data features and build an efficient fault diagnosis model. In addition, the system has early warning and decision support functions, can automatically generate warning information and provide customized emergency response plans, and provide a strong decision-making basis for operation and maintenance personnel.

[0011] The multimodal data acquisition module uses drones with high-definition imaging capabilities and professional-grade aerial cameras. Regular flight plans are formulated, the drone flight route and altitude are set, and the flight is carried out according to the planned route. The high-definition camera captures the surface image of the photovoltaic panel in real time, paying special attention to the vulnerable parts at the edges and joints. High-precision temperature sensors and vibration sensors are installed at key parts of the photovoltaic panel. The sensors collect temperature data and vibration physical quantities in real time and transmit them to the data acquisition terminal wirelessly. High-sensitivity sound sensors are installed in key equipment areas in the power station. The sensors continuously collect sound signals when the equipment is running, and all collected data are transmitted to the data processing center in real time through wireless communication technology. A data center is established, and high-performance servers and storage devices are configured to receive and process various data.

[0012] The image recognition technology in the data processing and analysis module performs denoising on the collected images, extracts edge information in the images, obtains the edge contour of the photovoltaic panel surface, analyzes the texture features of the photovoltaic panel surface, calculates the image brightness mean L, edge clarity (represented by E, which can be measured by the proportion of edge pixels to total pixels), color difference (represented by D, which can be calculated based on the difference in color histograms) and uniformity (represented by H, which can be approximated by calculating the variance of image pixel values), and designs a comprehensive evaluation function as the decision basis for the classifier. The formula of the classifier is: Z = aL + bE + cD + dH, where a, b, c, d are weight coefficients, and a + b + c + d = 1. A large number of images of normal photovoltaic panel surfaces are collected as training samples, and the comprehensive evaluation function values ​​of these samples are calculated, and their value range is determined. Use these training samples to train the classifier, including the optimization adjustment of the weight coefficient. For the surface image of the photovoltaic panel to be detected, divide it into several small areas, apply the comprehensive evaluation function to each small area, calculate its Z value, and if the Z value of a small area is not If the area is within the range, it is judged as an abnormal area, containing damage, defects and foreign objects.

[0013] The temperature and vibration data analysis in the data processing and analysis module collects the temperature data sequence T={T1,T2,…,T n}, using the vibration sensor to collect the vibration amplitude data sequence V = {V1, V2, ..., V m}, preprocess the temperature data and vibration data, calculate the temperature data features, and the mean: Standard Deviation: Temperature deviation sequence: Vibration data feature calculation, mean: Standard Deviation: Vibration deviation sequence Define a correlation index I TV Evaluate the correlation between temperature anomalies and specific vibration characteristics, which is calculated as: According to historical data and expert experience, a reasonable loss of connection threshold τ is set and the correlation index I is calculated. TV , and compare it with the threshold τ. If I TV >τ, it is considered that there is a correlation between abnormal temperature and specific vibration characteristics, indicating that there are overheating, looseness, and imbalance problems inside the equipment. Further combined with the specific manifestations of abnormal temperature areas and vibration characteristics, detailed fault diagnosis is performed.

[0014] The sound spectrum analysis in the data processing and analysis module collects the sound signal when the equipment is running, obtains the analog sound signal, converts the analog sound signal into a digital signal through an analog-to-digital converter, calculates the sound spectrum, extracts useful features from the spectrum graph, and obtains the sound spectrum from the spectrum graph. i (f) Useful features are extracted to form the feature vector S = {S1, S2, ..., S k}, where S i Represents a certain characteristic value of the sound signal. When the device is operating normally, a standard sound characteristic vector S is established and stored. normal ={S n1 ,S n2 ,…,S nk}, calculate the difference vector D between the real-time sound feature vector and the standard sound feature vector = {D1, D2, ..., Dk}, wherein a comprehensive difference index E is defined to quantify the difference between the real-time sound signal and the normal sound signal. The formula of the comprehensive difference index E is: A comprehensive difference threshold τ is set, and it is considered that there is a significant difference between the real-time sound signal and the normal sound signal, indicating that the equipment has abnormal noise and mechanical failure.

[0015] The data fusion and fault diagnosis module, based on the data of data analysis, deeply integrates the multi-source data of image, sound, temperature, and vibration and aligns them according to the timestamp, extracts the visual features of edge, texture, and color from the image data, extracts the acoustic features of spectrum, energy distribution, and harmonic ratio from the sound data, extracts the thermal features of temperature change rate and temperature gradient from the temperature data, and extracts the vibration features of frequency, amplitude, and phase from the vibration data. The various extracted features are normalized, and the multimodal data are fused by majority voting to form a comprehensive feature vector. According to the characteristics and application scenarios of new energy power generation equipment, a fault diagnosis model is constructed, and the model is trained using the fused multimodal data. The trained model is evaluated using an independent test data set to verify its performance in actual applications. According to the evaluation feedback, the model is further optimized, and the optimized model is deployed in the actual application environment of the new energy power station, the multimodal data of the equipment is monitored in real time, and the model is used for fault prediction and diagnosis. The results of the data processing and analysis module are combined to achieve accurate positioning of the fault.

[0016] The early warning and decision support module receives the real-time output results from the data fusion and fault diagnosis module. The module has a set of preset thresholds and rules to determine the early warning level reached by the current equipment status. When the received data exceeds the preset threshold, the module automatically determines that the corresponding early warning level has been reached and prepares to build early warning information. The module has a built-in plan library that stores emergency response plans for different fault types and severities. Taking into account the severity of the fault, the location of occurrence, and the current resource status, the module customizes the found plans, summarizes the warning information, fault details, and customized emergency response plan related information, and displays it to the operation and maintenance personnel.

[0017] Embodiment 2:

[0018] A new energy power generation monitoring system and method in this embodiment particularly emphasizes data fusion and fault diagnosis.

[0019] Collect the image, sound, temperature and vibration data of the equipment, align and preprocess the timestamps to improve the data quality, extract the edge, texture and color visual features from the image data, extract the spectrum, energy distribution and harmonic ratio acoustic features from the sound data, extract the temperature change rate and temperature gradient thermal features from the temperature data, and extract the frequency, amplitude and phase vibration features from the vibration data.

[0020] The multimodal data is fused using majority voting fusion. Let the image feature classification result be C I , the sound feature classification result is C s , the temperature feature classification result is C T , the vibration feature classification result is C V For each sample, calculate the number of votes for each category that is classified as this category by different modal data. k ,Votes k =N I,k +N S,k +N T,k +N V,k , where N I,k is the number of votes of the image features to classify the sample into k categories, C I = k, otherwise N I,k =0, similarly, calculate N S,k 、N T,k 、N V,k , the final classification result C final The category with the most votes, i.e. C final = argmax k Votes k .

[0021] By constructing a fault diagnosis model to identify the equipment status, let the given training data set be T = {(x1, y1), (x2, y2), …, (x n ,y n )}, where x i is the fused multimodal data feature vector, y i ∈{-1,1} is the corresponding category label. The model tries to find a hyperplane: w·xb=0, where w is the normal vector and b is the displacement term. The goal of the model is to maximize the classification interval, that is, The optimization objective is expressed as: minimize At the same time, the constraints y i (w T x i +b)≥1, for i=1,2,…,n, introduce the Lagrange multiplier α i ≥0, transform the original optimization problem into a dual form, and solve it to obtain the optimal Lagrange multiplier The final classification decision function is: Among them, b * By satisfying y j (w T x j +b) = 1 for any support vector x j to calculate.

[0022] Use test data to evaluate the trained SVM model, optimize the model based on the evaluation results, and deploy the optimized SVM model to the actual application environment to monitor the status of new energy power generation equipment in real time. Once anomalies and potential faults are found, an early warning is triggered immediately, and corresponding fault details and emergency handling suggestions are provided.

[0023] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A new energy power generation monitoring system, characterized in that: The system consists of the following components: Multimodal data acquisition module: Integrates drone high-definition photography, temperature sensors, vibration sensors and high-sensitivity sound sensors to capture all-round surface details of photovoltaic panels, real-time temperature and vibration data, and operating sounds; Data processing and analysis module: including image recognition technology, temperature and vibration data analysis and sound spectrum analysis: The image recognition technology performs denoising on the collected images, extracts edge information in the images, analyzes the texture features of the photovoltaic panel surface, identifies its contour and shape features, and trains a classifier to distinguish between a normal photovoltaic panel surface and an area with damage, defects, and foreign matter obstruction; The temperature and vibration data analysis acquires the temperature data and vibration data of the equipment, extracts the statistical features of the temperature data, extracts the time domain features, frequency domain features and time-frequency domain features from the vibration signal, analyzes the correlation between the temperature data and the vibration data, identifies the association between the temperature anomaly and the specific vibration features, and performs fault diagnosis in combination with the temperature anomaly area and the vibration features; The sound spectrum analysis pre-processes the collected sound signals, converts the analog sound signals into digital signals, calculates the sound spectrum, extracts useful features from the spectrum graph, compares the extracted features with the sound feature library when the equipment is operating normally, and identifies abnormal noise and early signs of mechanical failure; Data fusion and fault diagnosis module: Build a fusion model, combine image recognition results, physical data analysis results and sound spectrum analysis results to form a comprehensive description of the equipment status. Use this model in combination with the data processing and analysis module to achieve early detection, accurate positioning and cause diagnosis of faults; Early warning and decision support module: According to the results of the data fusion and fault diagnosis module, early warning information is automatically generated, and corresponding emergency treatment plans are given according to the type and severity of the fault to provide decision support for operation and maintenance personnel.

2. A new energy power generation monitoring system according to claim 1, characterized in that: The multimodal data acquisition module uses a drone equipped with a high-definition camera to regularly capture surface images of photovoltaic panel equipment to capture subtle damage and color anomalies, deploys temperature sensors and vibration sensors at key locations to collect temperature and vibration physical quantity data in real time, installs high-sensitivity sound sensors in the power station to collect sound signals when the equipment is running, transmits all data to a data processing center via wireless transmission, and stores all collected data.

3. A new energy power generation monitoring system according to claim 1, characterized in that: The image recognition technology in the data processing and analysis module distinguishes the normal photovoltaic panel surface from the area with damage, defects and foreign matter by training the classifier. The formula of the classifier is: Z = aL + bE + cD + dH, where a, b, c, d are weight coefficients, and a + b + c + d = 1, where L represents the brightness mean of the image, E represents the edge clarity of the image, D represents the color difference of the image, and H represents the uniformity of the image. By collecting a large number of normal photovoltaic panel surface images, the value range of its comprehensive evaluation function Z is calculated and set as When the Z value of an area image is not within this range, the area is determined to be an abnormal area.

4. A new energy power generation monitoring system according to claim 1, characterized in that: The temperature and vibration data analysis in the data processing and analysis module determines the association between temperature anomaly and specific vibration characteristics through correlation indicators. Let T = {T1, T2, ..., T n } is the temperature data sequence, and the mean of the temperature data is: The standard deviation of the temperature data is: Let V = {V1, V2, …, V m } is the vibration amplitude data sequence, the mean of the vibration amplitude data is: Standard deviation of vibration amplitude data: The temperature deviation sequence is: The vibration deviation sequence is: Define a correlation index I TV , the calculation formula is: Among them, w ij is the weight coefficient, setting the association threshold I threshold , when I TV More than I threshold When the temperature anomaly is detected, it is considered that there is a correlation between the temperature anomaly and the specific vibration characteristics.

5. A new energy power generation monitoring system according to claim 1, characterized in that: The sound spectrum analysis in the data processing and analysis module identifies the difference between abnormal noises through comprehensive difference indicators. Suppose the collected real-time sound signal feature vector is S = {S1, S2, ..., S k }, where S i Represents a certain characteristic value of the sound signal. The standard sound feature vector stored in the sound feature library when the device is operating normally is S normal ={S n1 ,S n2 ,…,S nk }, calculate the difference vector difference vector D = {D1, D2, ..., D k }, where represents the absolute value of the difference between the real-time sound feature and the normal sound feature at the i-th feature, and the comprehensive difference index E is calculated. The formula of the comprehensive difference index E is: A comprehensive difference threshold τ is set, and it is considered that there is a significant difference between the real-time sound signal and the normal sound signal.

6. A new energy power generation monitoring system according to claim 1, characterized in that: The data fusion and fault diagnosis module deeply integrates multi-source data such as image, sound, temperature, and vibration and aligns them according to timestamps, extracts features useful for fault diagnosis from each type of data, fuses multimodal data using majority voting fusion to form a comprehensive feature vector, builds a fault diagnosis model based on application scenarios and data characteristics to identify equipment status, trains the model using the fused multimodal data, evaluates the trained model using test data, optimizes the model based on the evaluation results, deploys the optimized model in an actual application environment, and combines image recognition results, physical data analysis results, and sound spectrum analysis results to accurately locate faults.

7. A new energy power generation monitoring system according to claim 1, characterized in that: The data fusion and fault diagnosis module uses majority voting fusion to fuse multimodal data. Suppose the image feature classification result is C I , the sound feature classification result is C s , the temperature feature classification result is C T , the vibration feature classification result is C V For each sample, calculate the number of votes for each category that is classified as this category by different modal data. For category k, the number of votes is Votes k =N I,k +N S,k +N T,k +N V,k , where N I,k is the number of votes of the image features to classify the sample into k categories, C I = k, otherwise N I,k =0, similarly, calculate N S,k 、N T,k 、N V,k , the final classification result C final The category with the most votes.

8. A new energy power generation monitoring system according to claim 1, characterized in that: The data fusion and fault diagnosis module constructs a fault diagnosis model to identify the equipment status. Assume that the given training data set is T = {(x1, y1), (x2, y2), ..., (x n ,y n )},in is the fused multimodal data feature vector, y i ∈{-1,1} is the corresponding category label, and the model tries to find a hyperplane: w T x+b=0, where w is the normal vector and b∈R is the displacement term. The goal of the model is to maximize the classification interval, which is defined as Among them, ∥·∥ represents the norm of the vector, and the optimization objective is expressed as: minimize At the same time, the constraints y i (w T x i +b)≥1, for i=1,2,…,n, by introducing the Lagrange multiplier α i ≥0, the above optimization problem can be transformed into a dual form, and the optimal Lagrange multiplier can be obtained by solving this dual problem. The final classification decision function is: Among them, b * By satisfying y j (w T x j +b)=1 is calculated for any support vector xj.

9. A new energy power generation monitoring system according to claim 1, characterized in that: The early warning and decision support module receives the output results from the data fusion and fault diagnosis module, and determines the early warning level reached by the current state according to the preset thresholds and rules. When the received data exceeds the preset threshold, it automatically determines the corresponding early warning level reached, searches for the corresponding emergency response plan in the system's plan library, and customizes the plan according to the severity of the fault, the location of occurrence, and the current resource status factors, and summarizes the early warning information, fault details, and emergency response plan related information and displays it to the operation and maintenance personnel.

10. A new energy power generation monitoring method, characterized in that: The specific steps of this method are: Data collection: Use drones equipped with high-definition cameras to regularly photograph the surface of photovoltaic panels to obtain high-resolution image data. Install temperature sensors and vibration sensors at key locations of power generation equipment to collect temperature and vibration information of the equipment in real time. Install sound sensors in power stations to capture sound signals when the equipment is running. Data processing and feature extraction: including image processing, physical data analysis and sound spectrum analysis: The image processing performs preprocessing on the collected image data and uses image recognition technology to detect damage and defects on the surface of the photovoltaic panel equipment; The physical data analysis performs filtering processing on the temperature data and the vibration data to remove noise interference and extract key characteristic parameters reflecting the equipment status; The sound spectrum analysis performs spectrum analysis on the sound signal to obtain a sound spectrum diagram, and combines it with electrical data to identify early signs of abnormal noise and mechanical failure; Fault diagnosis and early warning: Combine image recognition results, physical data analysis results, and sound spectrum analysis results to use algorithm models to intelligently diagnose the operating status of the equipment. By comparing the data features under normal conditions, the abnormal state is identified and the fault type and location are determined. Early warning and decision-making: Early warning information is automatically generated based on the fault diagnosis results, and corresponding emergency treatment plans are given according to the fault type and severity, so that operation and maintenance personnel can take actions based on the early warning information and decision support suggestions. A new energy power generation monitoring system and method Technical Field The present invention relates to the field of renewable energy power generation technology, and in particular to a renewable energy power generation monitoring system and method. With the continuous growth of global demand for clean energy, new energy power generation technology, especially solar photovoltaic power generation technology, has been widely used and rapidly developed. In order to ensure the stable and efficient operation of new energy power generation equipment, real-time monitoring and fault diagnosis of its status are particularly important. This is not only related to power generation efficiency and economic benefits, but also directly affects the safety and stability of the power grid and the reliability of energy supply. Therefore, the development of efficient and accurate power generation equipment monitoring technology has become a hot spot and difficulty in current research. Traditional methods of monitoring power generation equipment mostly rely on a single data type for analysis, such as image recognition technology used to detect surface damage of photovoltaic panels, and temperature and vibration sensor data used to determine internal equipment faults. However, in a complex and changeable power generation environment, a single data source is often difficult to fully reflect the true status of the equipment, which can easily lead to monitoring blind spots and misjudgments. Image recognition alone may not be able to accurately determine the performance degradation of photovoltaic panels behind shadows, and single temperature and vibration data may produce false alarms due to interference from environmental factors. In addition, traditional methods lack mutual verification and cross-validation mechanisms between multi-source data, making it difficult to ensure the accuracy and reliability of fault identification. Therefore, a new energy power generation monitoring system and method is developed, which adopts advanced image recognition, temperature and vibration data analysis, and sound spectrum analysis technology to perform deep processing and feature extraction on multi-source data, and realizes deep fusion and intelligent analysis of multi-modal data by building a fusion model. This innovative method not only makes up for the shortcomings of a single data source, but also significantly improves the accuracy and reliability of fault identification through mutual confirmation and cross-validation between data, providing a strong guarantee for the stable operation of new energy power generation equipment. Summary of the invention The purpose of the present invention is to make up for the shortcomings of the existing technology and provide a new energy power generation monitoring system and method, which can realize all-round and high-precision monitoring of the status of power generation equipment by integrating multimodal data acquisition modules and advanced intelligent analysis technologies. It not only breaks through the limitations of traditional monitoring methods in data uniformity, but also greatly improves the accuracy and timeliness of fault identification through the deep fusion and cross-validation of multi-source data. In order to solve the above technical problems, the present invention provides the following technical solutions: On the one hand, a new energy power generation monitoring system includes the following components: Multimodal data acquisition module: Integrates drone high-definition photography, temperature sensors, vibration sensors and high-sensitivity sound sensors to capture all-round surface details of photovoltaic panels, real-time temperature and vibration data, and operating sounds; Data processing and analysis module: including image recognition technology, temperature and vibration data analysis and sound spectrum analysis: The image recognition technology performs denoising on the collected images, extracts edge information in the images, analyzes the texture features of the photovoltaic panel surface, identifies its contour and shape features, and trains a classifier to distinguish between a normal photovoltaic panel surface and an area with damage, defects, and foreign matter obstruction; The temperature and vibration data analysis acquires the temperature data and vibration data of the equipment, extracts the statistical features of the temperature data, extracts the time domain features, frequency domain features and time-frequency domain features from the vibration signal, analyzes the correlation between the temperature data and the vibration data, identifies the association between the temperature anomaly and the specific vibration features, and performs fault diagnosis in combination with the temperature anomaly area and the vibration features; The sound spectrum analysis pre-processes the collected sound signals, converts the analog sound signals into digital signals, calculates the sound spectrum, extracts useful features from the spectrum graph, compares the extracted features with the sound feature library when the equipment is operating normally, and identifies abnormal noise and early signs of mechanical failure; Data fusion and fault diagnosis module: Build a fusion model, combine image recognition results, physical data analysis results and sound spectrum analysis results to form a comprehensive description of the equipment status. Use this model in combination with the data processing and analysis module to achieve early detection, accurate positioning and cause diagnosis of faults; Early warning and decision support module: According to the results of the data fusion and fault diagnosis module, early warning information is automatically generated, and corresponding emergency treatment plans are given according to the type and severity of the fault to provide decision support for operation and maintenance personnel. Furthermore, the multimodal data acquisition module uses a drone equipped with a high-definition camera to regularly capture surface images of photovoltaic panel equipment to capture subtle damage and color anomalies, deploys temperature sensors and vibration sensors at key locations to collect temperature and vibration physical quantity data in real time, installs high-sensitivity sound sensors in the power station to collect sound signals when the equipment is running, transmits all data to a data processing center via wireless transmission, and stores all collected data. Furthermore, the image recognition technology in the data processing and analysis module distinguishes the normal photovoltaic panel surface from the area with damage, defects and foreign matter occlusion by training a classifier. The formula of the classifier is: Z = aL + bE + cD + dH, where a, b, c, d are weight coefficients, and a + b + c + d = 1, where L represents the brightness mean of the image, E represents the edge clarity of the image, D represents the color difference of the image, and H represents the uniformity of the image. By collecting a large number of normal photovoltaic panel surface images, the value range of its comprehensive evaluation function Z is calculated and set as When the Z value of an area image is not within this range, the area is determined to be an abnormal area. Furthermore, the temperature and vibration data analysis in the data processing and analysis module determines the association between temperature anomaly and specific vibration characteristics through the correlation index, assuming that T = {T1, T2, ..., T n } is the temperature data sequence, and the mean of the temperature data is: The standard deviation of the temperature data is: Let V = {V1, V2, …, V m } is the vibration amplitude data sequence, the mean of the vibration amplitude data is: Standard deviation of vibration amplitude data: The temperature deviation sequence is: The vibration deviation sequence is: Define a correlation index I TV , the calculation formula is: Among them, w ij is the weight coefficient, setting the association threshold I threshold , when I TV More than I threshold When the temperature is abnormal, it is considered that there is a correlation between the temperature anomaly and the specific vibration characteristics. Furthermore, the sound spectrum analysis in the data processing and analysis module identifies the difference between abnormal noises by comprehensive difference indicators. Suppose the collected real-time sound signal feature vector is S = {S1, S2, ..., S k }, where S i Represents a certain characteristic value of the sound signal. The standard sound feature vector stored in the sound feature library when the device is operating normally is S normal ={S n1 ,S n2 ,…,S nk }, calculate the difference vector difference vector D = {D1, D2, ..., D k }, where represents the absolute value of the difference between the real-time sound feature and the normal sound feature at the i-th feature, and the comprehensive difference index E is calculated. The formula of the comprehensive difference index E is: A comprehensive difference threshold τ is set, and it is considered that there is a significant difference between the real-time sound signal and the normal sound signal. Furthermore, the data fusion and fault diagnosis module deeply integrates multi-source data such as images, sounds, temperature, and vibrations and aligns them according to timestamps, extracts features useful for fault diagnosis from each type of data, fuses multimodal data using majority voting fusion to form a comprehensive feature vector, builds a fault diagnosis model based on application scenarios and data characteristics to identify equipment status, trains the model using the fused multimodal data, evaluates the trained model using test data, optimizes the model based on the evaluation results, deploys the optimized model in an actual application environment, and combines image recognition results, physical data analysis results, and sound spectrum analysis results to accurately locate the fault. Furthermore, the data fusion and fault diagnosis module uses majority voting fusion to fuse multimodal data. Suppose the image feature classification result is C I , the sound feature classification result is C s , the temperature feature classification result is C T , the vibration feature classification result is C V For each sample, calculate the number of votes for each category that is classified as this category by different modal data. For category k, the number of votes is Among them, N I,k is the number of votes of the image features to classify the sample into k categories, C I = k, otherwise N I,k =0, similarly, calculate N S,k 、N T,k 、N V,k , the final classification result C final The category with the most votes. Furthermore, the data fusion and fault diagnosis module constructs a fault diagnosis model to identify the equipment status. Assume that the given training data set is T = {(x1, y1), (x2, y2), ..., (x n ,y n )},in is the fused multimodal data feature vector, y i ∈{-1,1} is the corresponding category label, and the model tries to find a hyperplane: w T x+b=0, where w is the normal vector and b∈R is the displacement term. The goal of the model is to maximize the classification interval, which is defined as Among them, ∥·∥ represents the norm of the vector, and the optimization objective is expressed as: minimize At the same time, the constraints y i (w T x i +b)≥1, for i=1,2,…,n, by introducing the Lagrange multiplier α i ≥0, the above optimization problem can be transformed into a dual form, and the optimal Lagrange multiplier can be obtained by solving this dual problem. The final classification decision function is: Among them, b * By satisfying For any support vector x j to calculate. Furthermore, the early warning and decision support module receives the output results from the data fusion and fault diagnosis module, and determines the early warning level reached by the current state according to preset thresholds and rules. When the received data exceeds the preset threshold, it automatically determines the corresponding early warning level reached, searches for the corresponding emergency response plan in the system's plan library, and customizes the plan according to the severity of the fault, the location of occurrence, and the current resource status factors, and summarizes the early warning information, fault details, and emergency response plan related information and displays it to the operation and maintenance personnel. On the other hand, a new energy power generation monitoring method is provided, wherein the specific steps of the method are: Data collection: Use drones equipped with high-definition cameras to regularly photograph the surface of photovoltaic panels to obtain high-resolution image data. Install temperature sensors and vibration sensors at key locations of power generation equipment to collect temperature and vibration information of the equipment in real time. Install sound sensors in power stations to capture sound signals when the equipment is running. Data processing and feature extraction: including image processing, physical data analysis and sound spectrum analysis: The image processing performs preprocessing on the collected image data and uses image recognition technology to detect damage and defects on the surface of the photovoltaic panel equipment; The physical data analysis performs filtering processing on the temperature data and the vibration data to remove noise interference and extract key characteristic parameters reflecting the equipment status; The sound spectrum analysis performs spectrum analysis on the sound signal to obtain a sound spectrum diagram, and combines it with electrical data to identify early signs of abnormal noise and mechanical failure; Fault diagnosis and early warning: Combine image recognition results, physical data analysis results, and sound spectrum analysis results to use algorithm models to intelligently diagnose the operating status of the equipment. By comparing the data features under normal conditions, the abnormal state is identified and the fault type and location are determined. Early warning and decision-making: Early warning information is automatically generated based on the fault diagnosis results, and corresponding emergency treatment plans are given according to the fault type and severity, so that operation and maintenance personnel can take actions based on the early warning information and decision support suggestions.

Images