Photovoltaic field area unmanned aerial vehicle inspection system

Abstract

The invention discloses a photovoltaic field area unmanned aerial vehicle inspection system, which relates to the field of unmanned aerial vehicle inspection, and specifically comprises a data acquisition and preprocessing module, a data fusion module, a data processing module, a data storage module, a data processing module and a data processing module, and is characterized in that the data acquisition and preprocessing module is used for acquiring environmental parameter data and carrying out dynamic data cleaning and preprocessing to obtain preprocessed environmental parameter data; carrying out weight adjustment on the basis of the preprocessed environmental parameter data, and carrying out cross-modal attention mechanism processing and synchronous fusion processing on the basis of the environmental parameter data subjected to weight adjustment to obtain cross-modal fusion features and synchronous fusion features; by introducing adaptive adjustment of environmental parameter data, cross-modal data fusion, anomaly detection and a self-learning optimization module, the high-precision inspection capability in a complex environment is realized, a data acquisition strategy is dynamically adjusted based on different environmental states, weight distribution and time synchronization fusion are performed on video images, thermal imaging and audio data, and the accuracy of inspection is improved. And the fusion precision of the multi-modal data and the accuracy of anomaly detection are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle inspection, and in particular to a photovoltaic field unmanned aerial vehicle inspection system. Background Art

[0002] In the daily operation and maintenance of photovoltaic fields, drone inspection systems have gradually been widely used to monitor and manage various environmental parameters and equipment operating status within the field. Traditional photovoltaic field drone inspection systems usually patrol and monitor the field by collecting environmental parameters such as temperature, humidity, light intensity, and wind speed.

[0003] However, due to the large area and complex environment of photovoltaic fields, especially in low light and high noise conditions, the effectiveness and accuracy of traditional inspection systems are subject to many limitations, resulting in insufficient comprehensiveness and real-time inspection data. In addition, inspections in photovoltaic fields require accurate identification of abnormal activities and rapid response. Traditional drone inspection systems are difficult to meet these high-efficiency and high-precision requirements. Secondly, the existing system has weak processing capabilities in multimodal data fusion, feature analysis and anomaly detection, and is unable to fully utilize multiple data sources such as video images, thermal imaging and audio for accurate judgment, which can easily result in underreporting or false alarms. Summary of the invention

[0004] The purpose of the present invention is to address the shortcomings of the prior art and provide a photovoltaic field drone inspection system, which aims to increase the dynamic adaptive adjustment of environmental parameter data acquisition, cross-modal data fusion, multi-dimensional composite judgment and self-learning optimization modules, so that the system can automatically adjust the acquisition strategy under different environmental conditions, and dynamically adjust the weights of multimodal data and synchronize time fusion, thereby improving the accuracy and comprehensiveness of the data. At the same time, the system significantly improves the detection effect of abnormal activities and reduces missed reports and false alarms through the combination of graph modeling and graph neural networks, convolutional neural networks and recurrent neural networks. Finally, the system introduces a self-learning optimization mechanism based on historical feedback data, which can automatically adjust the scoring weights and rule parameters to achieve adaptive optimization of environmental changes, greatly improving the intelligence level and safety assurance capabilities of the drone inspection system.

[0005] To this end, the present application provides a photovoltaic field drone inspection system, including the following modules:

[0006] The data acquisition and preprocessing module is used to obtain environmental parameter data, and perform dynamic data cleaning and preprocessing to obtain preprocessed environmental parameter data.

[0007] The data fusion module performs weight adjustment based on the preprocessed environmental parameter data, and performs cross-modal attention mechanism processing and synchronous fusion processing based on the weight-adjusted environmental parameter data to obtain cross-modal fusion features and synchronous fusion features.

[0008] The feature analysis and anomaly detection module performs graph structure modeling based on the obtained cross-modal fusion features and synchronous fusion features to obtain an activity graph, performs feature extraction and anomaly detection based on the activity graph, and performs preliminary screening based on the data after anomaly detection to obtain preliminary scoring data.

[0009] The multi-dimensional composite judgment module performs multi-level threat scoring and composite judgment based on the obtained preliminary scoring data, executes the rule engine on abnormal data, and optimizes the scoring weights and rule parameters based on historical alarm feedback and environmental parameter data.

[0010] The feedback and remote monitoring module visualizes abnormal data, dynamically updates the comprehensive threat score, and introduces remote control.

[0011] The self-learning optimization module is used to analyze historical feedback data for self-optimization iteration.

[0012] In some specific embodiments, the data acquisition and preprocessing module specifically includes:

[0013] Collect environmental parameter data based on drones and obtain environmental parameter data.

[0014] Among them, environmental parameter data include: temperature, humidity, light intensity and wind speed.

[0015] Adjust the collection strategy based on the different environmental conditions of the drone.

[0016] Among them, different environments include: low light environment and high noise environment.

[0017] Among them, if it is in a low-light environment, the image resolution and frame rate are increased to enhance image clarity. If it is in a high-noise environment, the audio acquisition frequency is reduced to improve the signal-to-noise ratio of the acquisition.

[0018] Dynamic data cleaning is performed based on the obtained environmental parameter data, and multi-level quality screening is performed on the collected data to obtain environmental parameter data after dynamic data cleaning.

[0019] Among them, dynamic data cleaning is to filter the video images, thermal imaging and audio data separately. For video images, low-light enhancement filtering is applied to eliminate noise caused by insufficient light. For audio data, irrelevant background noise is eliminated through signal-to-noise ratio filtering.

[0020] It should be noted here that since dynamic data cleaning is a technology well known to people in this field, it will not be elaborated in detail here.

[0021] Preprocessing is performed on the environmental parameter data after dynamic data cleaning, and the preprocessed environmental parameter data is obtained.

[0022] Among them, preprocessing includes multimodal synchronization and time alignment.

[0023] Among them, multimodal synchronization and time alignment is the time synchronization and alignment processing of video, thermal imaging and audio.

[0024] It should be noted that, since preprocessing is a technology well known to those skilled in the art, it will not be described in detail here.

[0025] In some specific implementations, the data fusion module specifically includes:

[0026] Weight adjustment is performed based on the preprocessed environmental parameter data.

[0027] Among them, the weight adjustment is based on the environmental parameter data in low-light environment or high-noise environment to dynamically allocate the weight of each modal data.

[0028] Among them, in low-light environments, the weight of thermal imaging data is increased to enhance the recognition of object shape, and in high-noise environments, the weight of audio data is appropriately reduced. Specifically:

[0029]

[0030] Where: w v Represents the weight of video image data, w t represents the weight of thermal imaging data, w a represents the weight of the audio data, L represents the light intensity, L is represented by a standardized value from 0 to 1, 0 represents no light at all, 1 represents a high light environment, N represents the noise intensity, N is represented by a standardized value from 0 to 1, 0 represents no noise at all, 1 represents a high noise environment, and the sum of the weights satisfies w v +w t +w a =1.

[0031] Among them, Z is the normalization coefficient, so that w v +w t +w a =1, specifically:

[0032]

[0033] Among them, when the light intensity L is low, the system tends to increase the weight of the video image, the weight w vThe 1-L item was added.

[0034] Among them, when the noise intensity N is low, the weight of the video image is further improved, combining the 1-N terms.

[0035] Among them, when the light intensity L is high, the system automatically increases the weight of thermal imaging to compensate for the defects of video images in high-light environments.

[0036] Among them, when the noise intensity N is low, the thermal imaging data also has good stability, so the weighted term includes 1-N.

[0037] Among them, when the light intensity L is low, the audio weight is enhanced, which can supplement the insufficient video image in low-light environment.

[0038] Among them, when the noise intensity N is high, the audio weight is further improved, so that the system's adaptability to high noise environments is enhanced.

[0039] A cross-modal attention mechanism is used to process the weighted environmental parameter data and obtain cross-modal fusion features.

[0040] Among them, the cross-modal attention mechanism processing is to extract the feature vectors of each modality from video images, thermal imaging and audio data to construct the initial feature matrix.

[0041] Among them, based on the eigenvector of each mode, the correlation weights between the eigenvectors of each mode are calculated to form a weight matrix.

[0042] Among them, the weight matrix is ​​used to perform weighted fusion on the feature vectors of each modality to generate the final fusion feature, and finally obtain the cross-modal fusion feature.

[0043] Based on the weight-adjusted environmental parameter data, synchronous fusion processing is performed to obtain synchronous fusion features.

[0044] Among them, the synchronous fusion processing is to time-stamp the video image, thermal imaging and audio data.

[0045] Among them, based on the timestamp mark, the environmental parameter data is time-aligned.

[0046] Among them, after time alignment processing, the environmental parameter data are fused into a complete set of synchronous fusion features to obtain synchronous fusion features.

[0047] In some specific implementations, the feature analysis and anomaly detection module specifically includes:

[0048] Graph structure modeling is performed based on the obtained cross-modal fusion features and synchronous fusion features.

[0049] Among them, graph structure modeling is to model the graph structure of active objects. Active objects are represented as nodes in the graph, and the interactions between active objects are represented as edges.

[0050] Among them, an activity graph is constructed based on nodes and edges, and an activity graph is obtained.

[0051] Based on the obtained activity graph, a graph neural network is used to extract related features from the neighboring nodes of each node, update the node status information, and obtain the feature vector of the activity object, specifically:

[0052]

[0053] Where: is the feature vector of node i at the k+1th layer, W (k) is the weight matrix of the kth layer, used to adjust the node features, is the feature vector of node i at layer k, is the set of neighbor nodes of node i, α ij is the attention weight between node i and node j, which is used to adjust the influence of adjacent nodes on the target node, and σ is the activation function, which is used to increase nonlinearity.

[0054] Based on the obtained feature vectors of the active objects, a combined model of convolutional neural network and recurrent neural network is used to perform anomaly detection and obtain abnormal data.

[0055] Among them, the convolutional neural network is responsible for extracting the local spatial relationship of features and identifying whether there are abnormalities in the space of the active object. Specifically:

[0056]

[0057] Where: f conv is the spatial feature after convolution, indicating the spatial relationship between the current object and the surrounding objects, w mn is the convolution kernel weight, x ij are the values ​​in the input feature matrix.

[0058] Among them, the recurrent neural network is used to capture the temporal dynamic changes of features and identify abnormal behaviors of active objects that last for a long time or occur repeatedly, specifically:

[0059] h t =σ(W h h t-1 +W x x t +b)

[0060] Where: h t is the implicit state of the current time step, W h and W xis the weight matrix, x t is the input feature of the current time step.

[0061] A preliminary score is performed based on the obtained abnormal data, and activity object data with a higher score is obtained.

[0062] Among them, the preliminary scoring is based on the spatial location, duration and behavior type characteristics of the object, and the preliminary screening of abnormal data is performed to obtain preliminary scoring data, specifically:

[0063] S initial =w p ·f p +w d ·f d +w f ·f f

[0064] Where: S initial is the preliminary anomaly score, f p is the spatial position score, f d is the temporal feature score, f f is the behavior type score, w p 、w d ,w f is the weight coefficient of the named feature.

[0065] In some specific implementations, the multi-dimensional composite judgment module specifically includes:

[0066] Based on the preliminary scoring data obtained, a multi-level threat score is performed on each abnormal data, and a comprehensive threat score is obtained, which is as follows:

[0067] S threat =w s ·f s +w t ·f t +w b ·f b

[0068] Where: S threat is a comprehensive threat score used to assess the overall risk level of an activity, s is the spatial position score, indicating the distance of the activity from the photovoltaic panel, f t is the time feature score, indicating the duration of the activity, f b is the behavior type score, indicating the specific type of activity, and w s 、w t 、w b is the weight coefficient of each feature, which is used to balance the influence of different dimensions.

[0069] Based on the obtained comprehensive threat score, the rule engine makes a compound judgment on the characteristics of the activity.

[0070] The rule engine includes: Rule 1 and Rule 2.

[0071] Among them, rule 1, if S threat >threshold and close to photovoltaic panels, it is marked as a high threat activity.

[0072] The threshold value is set as follows: during night time, the threshold value is adjusted to 70; during day time, the threshold value is adjusted to 85.

[0073] Among them, rule two, if the activity lasts for a long time and the behavior frequency is high, it is marked as suspected anomaly.

[0074] Prioritize each activity based on compound analysis of the rule engine results and the comprehensive threat score.

[0075] Among them, high-priority events will trigger immediate alarms, while low-priority events will enter a state to be observed.

[0076] Automatically adjust scoring weights and rule parameters based on historical alarm feedback and environmental parameter data, specifically:

[0077]

[0078] Where: is the updated spatial weight, α is the learning rate, which is used to adjust the update speed, and R is the reward value based on feedback, which indicates the judgment effect of the system.

[0079] In some specific implementations, the feedback and remote monitoring module specifically includes:

[0080] Based on the obtained comprehensive threat score, the key information of the abnormal data is intuitively displayed on the visualization interface in the form of icons, colors and warning marks.

[0081] Among them, key information includes: activity location, threat score, and activity trajectory.

[0082] Based on the dynamically updated comprehensive threat score, the abnormal data information on the monitoring interface is refreshed in real time.

[0083] Among them, when an active object is detected close to the photovoltaic panel or the threat score reaches a high threat level, the position, trajectory and threat score information are updated.

[0084] Based on the obtained comprehensive threat score data, the abnormal data is divided into three levels: high threat, medium threat and low threat.

[0085] Among them, for high-threat activities, immediate alerts will be triggered and highlighted on the interface.

[0086] Among them, early warning alerts are sent for medium-threat activities.

[0087] Among them, low-threat activities will enter observation status.

[0088] Based on the preset threat score threshold, when the comprehensive threat score exceeds the threshold, an alert message will be pushed to the operator, including the activity location, threat score, and handling suggestions.

[0089] Remote control is introduced, and the operator adjusts the drone's patrol route based on the comprehensive threat score.

[0090] Among them, the operator can control the drone to stay in the threat area for key monitoring or command the drone to track active objects.

[0091] Data is recorded based on all operator intervention instructions and the system's responses.

[0092] In some specific implementations, the self-learning optimization module specifically includes:

[0093] Using an optimization mechanism and based on historical abnormal activity detection results, a comprehensive threat score S is assigned to each activity. threat Conduct analysis and scoring, and determine whether it reflects actual risks.

[0094] An optimization mechanism is adopted and attribution analysis is performed based on environmental parameter data.

[0095] Among them, attribution analysis is to analyze whether the scoring accuracy changes under different light intensities and noise levels.

[0096] Based on the historical feedback data showing that the spatial position score performs poorly in high threat detection, the spatial position score weight w is reduced s Optimize, specifically:

[0097]

[0098] Where: represents the optimized spatial position score weight, R is the target score, and α is the learning rate, which controls the update speed.

[0099] Based on time score f t and behavior type score f b The weight w t and w b Perform independent optimization to increase the time score weight or behavior type score weight if historical feedback data shows that long-term activities or specific behaviors are more threatening.

[0100] Based on the latest feedback data and environmental parameters, when the setting reaches 7 days, the spatial location score weight, time score weight, behavior type score weight and threshold are updated.

[0101] Among them, an adaptive optimization mechanism will be formed based on updating the spatial location scoring weight, time scoring weight, behavior type scoring weight and threshold.

[0102] Among them, based on the updated scores and threshold settings of the optimization mechanism, tuning suggestions are generated for operators to review and manually adjust weights or thresholds.

[0103] In summary, the present application provides a photovoltaic field drone inspection system, a data acquisition and preprocessing module, which is used to obtain environmental parameter data, and perform dynamic data cleaning and preprocessing to obtain preprocessed environmental parameter data, a data fusion module, which performs weight adjustment based on the preprocessed environmental parameter data, and performs cross-modal attention mechanism processing and synchronous fusion processing based on the weight-adjusted environmental parameter data to obtain cross-modal fusion features and synchronous fusion features, a feature analysis and anomaly detection module, which performs graph structure modeling based on the obtained cross-modal fusion features and synchronous fusion features to obtain an activity graph, performs feature extraction and anomaly detection based on the activity graph, and performs preliminary screening based on the data after anomaly detection. , obtain preliminary scoring data, and realize high-precision inspection capabilities in complex environments by introducing adaptive adjustment of environmental parameter data, cross-modal data fusion, anomaly detection and self-learning optimization modules. The data collection strategy is dynamically adjusted based on different environmental states, and the video image, thermal imaging and audio data are weighted and time-synchronized to effectively improve the fusion accuracy of multimodal data and the accuracy of anomaly detection. In addition, through multi-dimensional composite judgment and self-learning optimization mechanism, the scoring weights and rule parameters are optimized based on historical feedback data, so that it can adapt to environmental changes during the inspection process, continuously improve the stability and intelligence level of detection, thereby greatly improving the monitoring efficiency and safety assurance capabilities of photovoltaic fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 It is an overall flow chart of a photovoltaic field drone inspection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0105] Please refer to Figure 1 , which shows the process of an embodiment of a photovoltaic field drone inspection system according to the present disclosure

[0106] like Figure 1 As shown, a photovoltaic field drone inspection system includes the following modules:

[0107] The data acquisition and preprocessing module is used to obtain environmental parameter data, and perform dynamic data cleaning and preprocessing to obtain preprocessed environmental parameter data.

[0108] The data fusion module performs weight adjustment based on the preprocessed environmental parameter data, and performs cross-modal attention mechanism processing and synchronous fusion processing based on the weight-adjusted environmental parameter data to obtain cross-modal fusion features and synchronous fusion features.

[0109] The feature analysis and anomaly detection module performs graph structure modeling based on the obtained cross-modal fusion features and synchronous fusion features to obtain an activity graph, performs feature extraction and anomaly detection based on the activity graph, and performs preliminary screening based on the data after anomaly detection to obtain preliminary scoring data.

[0110] The multi-dimensional composite judgment module performs multi-level threat scoring and composite judgment based on the obtained preliminary scoring data, executes the rule engine on abnormal data, and optimizes the scoring weights and rule parameters based on historical alarm feedback and environmental parameter data.

[0111] The feedback and remote monitoring module visualizes abnormal data, dynamically updates the comprehensive threat score, and introduces remote control.

[0112] The self-learning optimization module is used to analyze historical feedback data for self-optimization iteration.

[0113] In some specific embodiments, the data acquisition and preprocessing module specifically includes:

[0114] Collect environmental parameter data based on drones and obtain environmental parameter data.

[0115] Among them, environmental parameter data include: temperature, humidity, light intensity and wind speed.

[0116] Adjust the collection strategy based on the different environmental conditions of the drone.

[0117] Among them, different environments include: low light environment and high noise environment.

[0118] Among them, if it is in a low-light environment, the image resolution and frame rate are increased to enhance image clarity. If it is in a high-noise environment, the audio acquisition frequency is reduced to improve the signal-to-noise ratio of the acquisition.

[0119] Dynamic data cleaning is performed based on the obtained environmental parameter data, and multi-level quality screening is performed on the collected data to obtain environmental parameter data after dynamic data cleaning.

[0120] Among them, dynamic data cleaning is to filter the video images, thermal imaging and audio data separately. For video images, low-light enhancement filtering is applied to eliminate noise caused by insufficient light. For audio data, irrelevant background noise is eliminated through signal-to-noise ratio filtering.

[0121] It should be noted here that since dynamic data cleaning is a technology well known to people in this field, it will not be elaborated in detail here.

[0122] Preprocessing is performed on the environmental parameter data after dynamic data cleaning, and the preprocessed environmental parameter data is obtained.

[0123] Among them, preprocessing includes multimodal synchronization and time alignment.

[0124] Among them, multimodal synchronization and time alignment is the time synchronization and alignment processing of video, thermal imaging and audio.

[0125] It should be noted that, since preprocessing is a technology well known to those skilled in the art, it will not be described in detail here.

[0126] In some specific implementations, the data fusion module specifically includes:

[0127] Weight adjustment is performed based on the preprocessed environmental parameter data.

[0128] Among them, the weight adjustment is based on the environmental parameter data in low-light environment or high-noise environment to dynamically allocate the weight of each modal data.

[0129] Among them, in low-light environments, the weight of thermal imaging data is increased to enhance the recognition of object shape, and in high-noise environments, the weight of audio data is appropriately reduced. Specifically:

[0130]

[0131] Where: w v Represents the weight of video image data, w t represents the weight of thermal imaging data, w a represents the weight of the audio data, L represents the light intensity, L is represented by a standardized value from 0 to 1, 0 represents no light at all, 1 represents a high light environment, N represents the noise intensity, N is represented by a standardized value from 0 to 1, 0 represents no noise at all, 1 represents a high noise environment, and the sum of the weights satisfies w v +w t +w a =1.

[0132] Among them, Z is the normalization coefficient, so that w v +w t +w a =1, specifically:

[0133]

[0134] Among them, when the light intensity L is low, the system tends to increase the weight of the video image, the weight w v The 1-L item was added.

[0135] Among them, when the noise intensity N is low, the weight of the video image is further improved, combining the 1-N terms.

[0136] Among them, when the light intensity L is high, the system automatically increases the weight of thermal imaging to compensate for the defects of video images in high-light environments.

[0137] Among them, when the noise intensity N is low, the thermal imaging data also has good stability, so the weighted term includes 1-N.

[0138] Among them, when the light intensity L is low, the audio weight is enhanced, which can supplement the insufficient video image in low-light environment.

[0139] Among them, when the noise intensity N is high, the audio weight is further improved, so that the system's adaptability to high noise environments is enhanced.

[0140] A cross-modal attention mechanism is used to process the weighted environmental parameter data and obtain cross-modal fusion features.

[0141] Among them, the cross-modal attention mechanism processing is to extract the feature vectors of each modality from video images, thermal imaging and audio data to construct the initial feature matrix.

[0142] Among them, based on the eigenvector of each mode, the correlation weights between the eigenvectors of each mode are calculated to form a weight matrix.

[0143] Among them, the weight matrix is ​​used to perform weighted fusion on the feature vectors of each modality to generate the final fusion feature, and finally obtain the cross-modal fusion feature.

[0144] Based on the weight-adjusted environmental parameter data, synchronous fusion processing is performed to obtain synchronous fusion features.

[0145] Among them, the synchronous fusion processing is to time-stamp the video image, thermal imaging and audio data.

[0146] Among them, based on the timestamp mark, the environmental parameter data is time-aligned.

[0147] Among them, after time alignment processing, the environmental parameter data are fused into a complete set of synchronous fusion features to obtain synchronous fusion features.

[0148] In some specific implementations, the feature analysis and anomaly detection module specifically includes:

[0149] Graph structure modeling is performed based on the obtained cross-modal fusion features and synchronous fusion features.

[0150] Among them, graph structure modeling is to model the graph structure of active objects. Active objects are represented as nodes in the graph, and the interactions between active objects are represented as edges.

[0151] Among them, an activity graph is constructed based on nodes and edges, and an activity graph is obtained.

[0152] Based on the obtained activity graph, a graph neural network is used to extract related features from the neighboring nodes of each node, update the node status information, and obtain the feature vector of the activity object, specifically:

[0153]

[0154] Where: is the feature vector of node i at the k+1th layer, W (k) is the weight matrix of the kth layer, used to adjust the node features, is the feature vector of node i at layer k, is the set of neighbor nodes of node i, α ij is the attention weight between node i and node j, which is used to adjust the influence of adjacent nodes on the target node, and σ is the activation function, which is used to increase nonlinearity.

[0155] Based on the obtained feature vectors of the active objects, a combined model of convolutional neural network and recurrent neural network is used to perform anomaly detection and obtain abnormal data.

[0156] Among them, the convolutional neural network is responsible for extracting the local spatial relationship of features and identifying whether there are abnormalities in the space of the active object. Specifically:

[0157]

[0158] Where: f conv is the spatial feature after convolution, indicating the spatial relationship between the current object and the surrounding objects, w mn is the convolution kernel weight, x ij are the values ​​in the input feature matrix.

[0159] Among them, the recurrent neural network is used to capture the temporal dynamic changes of features and identify abnormal behaviors of active objects that last for a long time or occur repeatedly, specifically:

[0160] h t =σ(W h h t-1 +Wx x t +b)

[0161] Where: h t is the implicit state of the current time step, W h and W x is the weight matrix, x t is the input feature of the current time step.

[0162] A preliminary score is performed based on the obtained abnormal data, and activity object data with a higher score is obtained.

[0163] Among them, the preliminary scoring is based on the spatial location, duration and behavior type characteristics of the object, and the preliminary screening of abnormal data is performed to obtain preliminary scoring data, specifically:

[0164] S initial =w p ·f p +w d ·f d +w f ·f f

[0165] Where: S initial is the preliminary anomaly score, f p is the spatial position score, f d is the temporal feature score, f f is the behavior type score, w p 、w d ,w f is the weight coefficient of the named feature.

[0166] In some specific implementations, the multi-dimensional composite judgment module specifically includes:

[0167] Based on the preliminary scoring data obtained, a multi-level threat score is performed on each abnormal data, and a comprehensive threat score is obtained, which is as follows:

[0168] S threat =w s ·f s +w t ·f t +w b ·f b

[0169] Where: S threat is a comprehensive threat score used to assess the overall risk level of an activity, s is the spatial position score, indicating the distance of the activity from the photovoltaic panel, f t is the time feature score, indicating the duration of the activity, f b is the behavior type score, indicating the specific type of activity, and ws 、w t 、w b is the weight coefficient of each feature, which is used to balance the influence of different dimensions.

[0170] Based on the obtained comprehensive threat score, the rule engine makes a compound judgment on the characteristics of the activity.

[0171] The rule engine includes: Rule 1 and Rule 2.

[0172] Among them, rule 1, if S threat >threshold and close to photovoltaic panels, it is marked as a high threat activity.

[0173] The threshold value is set as follows: during night time, the threshold value is adjusted to 70; during day time, the threshold value is adjusted to 85.

[0174] Among them, rule two, if the activity lasts for a long time and the behavior frequency is high, it is marked as suspected anomaly.

[0175] Prioritize each activity based on compound analysis of the rule engine results and the comprehensive threat score.

[0176] Among them, high-priority events will trigger immediate alarms, while low-priority events will enter a state to be observed.

[0177] Automatically adjust scoring weights and rule parameters based on historical alarm feedback and environmental parameter data, specifically:

[0178]

[0179] Where: is the updated spatial weight, α is the learning rate, which is used to adjust the update speed, and R is the reward value based on feedback, which indicates the judgment effect of the system.

[0180] In some specific implementations, the feedback and remote monitoring module specifically includes:

[0181] Based on the obtained comprehensive threat score, the key information of the abnormal data is intuitively displayed on the visualization interface in the form of icons, colors and warning marks.

[0182] Among them, key information includes: activity location, threat score, and activity trajectory.

[0183] Based on the dynamically updated comprehensive threat score, the abnormal data information on the monitoring interface is refreshed in real time.

[0184] Among them, when an active object is detected close to the photovoltaic panel or the threat score reaches a high threat level, the position, trajectory and threat score information are updated.

[0185] Based on the obtained comprehensive threat score data, the abnormal data is divided into three levels: high threat, medium threat and low threat.

[0186] Among them, for high-threat activities, immediate alerts will be triggered and highlighted on the interface.

[0187] Among them, early warning alerts are sent for medium-threat activities.

[0188] Among them, low-threat activities will enter observation status.

[0189] Based on the preset threat score threshold, when the comprehensive threat score exceeds the threshold, an alert message will be pushed to the operator, including the activity location, threat score, and handling suggestions.

[0190] Remote control is introduced, and the operator adjusts the drone's patrol route based on the comprehensive threat score.

[0191] Among them, the operator can control the drone to stay in the threat area for key monitoring or command the drone to track active objects.

[0192] Data is recorded based on all operator intervention instructions and the system's responses.

[0193] In some specific implementations, the self-learning optimization module specifically includes:

[0194] Using an optimization mechanism and based on historical abnormal activity detection results, a comprehensive threat score S is assigned to each activity. threat Conduct analysis and scoring, and determine whether it reflects actual risks.

[0195] An optimization mechanism is adopted and attribution analysis is performed based on environmental parameter data.

[0196] Among them, attribution analysis is to analyze whether the scoring accuracy changes under different light intensities and noise levels.

[0197] Based on the historical feedback data showing that the spatial position score performs poorly in high threat detection, the spatial position score weight w is reduced s Optimize, specifically:

[0198]

[0199] Where: represents the optimized spatial position score weight, R is the target score, and α is the learning rate, which controls the update speed.

[0200] Based on time score f t and behavior type score f b The weight w t and w bPerform independent optimization to increase the time score weight or behavior type score weight if historical feedback data shows that long-term activities or specific behaviors are more threatening.

[0201] Based on the latest feedback data and environmental parameters, when the setting reaches 7 days, the spatial location score weight, time score weight, behavior type score weight and threshold are updated.

[0202] Among them, an adaptive optimization mechanism will be formed based on updating the spatial location scoring weight, time scoring weight, behavior type scoring weight and threshold.

[0203] Among them, based on the updated scores and threshold settings of the optimization mechanism, tuning suggestions are generated for operators to review and manually adjust weights or thresholds.

[0204] In the above content, in actual application, the environmental parameter data of the photovoltaic field is first collected by drones, including basic data such as temperature, humidity, light intensity and wind speed. Based on the acquired environmental parameter data, the system automatically adjusts the collection strategy according to the environmental status of the drone. In low-light environments, the image clarity is enhanced by improving the image resolution and frame rate, while in high-noise environments, the signal-to-noise ratio of the collection is improved by reducing the audio collection frequency, thereby effectively improving the quality of data collection. After the data collection is completed, the system performs dynamic data cleaning on the collected data, and performs multi-level quality screening on the video images, thermal imaging and audio data respectively, to eliminate noise points and irrelevant background noise caused by insufficient light, and finally obtain high-quality environmental parameter data after dynamic data cleaning.

[0205] Based on the preprocessed environmental parameter data, the system dynamically adjusts the weights through the data fusion module to reasonably distribute the weights of multimodal data in different environments. The system dynamically allocates the weights of video images, thermal imaging and audio data according to the environmental parameter data in low-light and high-noise environments. In low-light environments, the system automatically increases the weight of thermal imaging data to enhance the recognition of object morphology, while in high-noise environments, the system appropriately reduces the weight of audio data to ensure the validity of other data sources. The data fusion module further extracts the feature vectors of video images, thermal imaging and audio data through the cross-modal attention mechanism, constructs the initial feature matrix, calculates the correlation weights between the features of each modality and performs weighted fusion to obtain high-precision cross-modal fusion features.

[0206] After data fusion is completed, the system performs synchronous fusion processing on the weighted environmental parameter data. First, the video image, thermal imaging and audio data are time-stamped to ensure the consistency of multimodal data on the timeline. Then, based on the timestamp, the data of different modes are time-aligned to synchronize them at the same time point. Finally, the data after time alignment is fused into a complete set of synchronous fusion features, providing reliable input data for subsequent anomaly detection and analysis.

[0207] Based on the obtained cross-modal fusion features and synchronous fusion features, the system models the graph structure of the active objects in the venue. The system represents each active object as a node in the graph, and the interaction relationship between different objects as an edge in the graph, thereby constructing an activity graph. Based on the activity graph, the system uses a graph neural network to extract associated features from the neighboring nodes of each node, updates the status information of the node, and obtains the feature vector of the active object. The feature vector of the active object is analyzed through a combined model of a convolutional neural network and a recurrent neural network to extract spatial and temporal features, thereby realizing the detection of abnormal activities and identifying abnormal data with potential risks.

[0208] For detected abnormal data, the system performs multi-level threat scoring based on the preliminary scoring data to generate a comprehensive threat score S threat The score is based on the spatial location score, time feature score and behavior type score of the object to evaluate the overall risk level of the activity object. On this basis, the system makes a composite judgment through the rule engine. Specifically, the rule engine contains two main rules: First, when the comprehensive threat score S threat When the threshold is exceeded and the activity is close to the photovoltaic panel, the activity is marked as a high-threat activity. Secondly, for activities with a long duration and high frequency of behavior, they are marked as suspected abnormal events. The system assigns priority to each activity based on the results of the rule engine and the comprehensive threat score. High-priority activities will trigger an alarm, and low-priority activities will enter a state to be observed.

[0209] The system automatically optimizes the scoring weights and rule parameters based on historical alarm feedback data and environmental parameter data, thereby continuously improving the detection accuracy and adaptability of the system. Specifically, the self-learning optimization module first analyzes the historical feedback data, and optimizes the weights of the spatial location score, time feature score, and behavior type score based on the scoring results. In terms of rule thresholds, the system sets a lower threshold of 70 at night and a higher threshold of 85 during the day to ensure detection sensitivity in different time periods. In addition, the system automatically updates the scoring model based on real-time feedback to ensure that it maintains the best effect under a variety of environmental changes, thereby improving the intelligence level of photovoltaic field inspections and safety monitoring effects.

Claims

1. A photovoltaic field drone inspection system, characterized in that: Includes the following modules: The data acquisition and preprocessing module is used to obtain environmental parameter data, and perform dynamic data cleaning and preprocessing to obtain preprocessed environmental parameter data; A data fusion module is used to adjust the weights based on the preprocessed environmental parameter data, and to perform cross-modal attention mechanism processing and synchronous fusion processing based on the weight-adjusted environmental parameter data to obtain cross-modal fusion features and synchronous fusion features; The feature analysis and anomaly detection module is used to perform graph structure modeling based on the obtained cross-modal fusion features and synchronous fusion features, obtain activity graphs, perform feature extraction and anomaly detection based on the activity graphs, and perform preliminary screening based on the data after anomaly detection to obtain preliminary scoring data; A multi-dimensional composite judgment module, which is used to perform multi-level threat scoring and composite judgment based on the obtained preliminary scoring data, execute the rule engine on abnormal data, and optimize the scoring weights and rule parameters based on historical alarm feedback and environmental parameter data; Feedback and remote monitoring module, used to visualize abnormal data, dynamically update comprehensive threat scores and introduce remote control; The self-learning optimization module is used to analyze historical feedback data for self-optimization iteration.

2. A photovoltaic field drone inspection system according to claim 1, characterized in that: Data acquisition and preprocessing module, including: Collect environmental parameter data based on drones and obtain environmental parameter data; Based on the obtained environmental parameter data, dynamic data cleaning is performed, and the collected data is subjected to multi-level quality screening to obtain environmental parameter data after dynamic data cleaning; Preprocessing is performed on the environmental parameter data after dynamic data cleaning, and the preprocessed environmental parameter data is obtained.

3. A photovoltaic field drone inspection system according to claim 1, characterized in that: Data fusion module, including: Weight adjustment is performed based on the preprocessed environmental parameter data; Among them, the weight adjustment dynamically allocates the weight of each modal data based on the environmental parameter data in a low-light environment or a high-noise environment; In low-light environments, the weight of thermal imaging data is increased to enhance the recognition of object shape, and in high-noise environments, the weight of audio data is appropriately reduced; Perform cross-modal attention mechanism processing based on the weighted environmental parameter data and obtain cross-modal fusion features; Based on the weight-adjusted environmental parameter data, synchronous fusion processing is performed to obtain synchronous fusion features.

4. A photovoltaic field drone inspection system according to claim 1, characterized in that: Feature analysis and anomaly detection module, including: Graph structure modeling is performed based on the obtained cross-modal fusion features and synchronous fusion features; Among them, graph structure modeling is to model the graph structure of active objects. Active objects are represented as nodes in the graph, and the interactions between active objects are represented as edges. Among them, an activity graph is constructed based on nodes and edges, and an activity graph is obtained; Based on the obtained activity graph, a graph neural network is used to extract related features from the neighboring nodes of each node, update the node status information, and obtain the feature vector of the activity object, specifically: Where: is the feature vector of node i at the k+1th layer, W (k) is the weight matrix of the kth layer, used to adjust the node features, is the feature vector of node i at layer k, is the set of neighbor nodes of node i, α ij is the attention weight between node i and node j, which is used to adjust the influence of adjacent nodes on the target node, and σ is the activation function, which is used to increase nonlinearity; Based on the obtained feature vectors of the active objects, a combined model of convolutional neural network and recurrent neural network is used to perform anomaly detection and obtain abnormal data; Among them, the convolutional neural network is responsible for extracting the local spatial relationship of features and identifying whether there are abnormalities in the space of the active object. Specifically: Where: f conv is the spatial feature after convolution, indicating the spatial relationship between the current object and the surrounding objects, w mn is the convolution kernel weight, x ij are the values ​​in the input feature matrix; Among them, the recurrent neural network is used to capture the temporal dynamic changes of features and identify abnormal behaviors of active objects that last for a long time or occur repeatedly, specifically: h t =σ(W h h t-1 +W x x t +b) Where: h t is the implicit state of the current time step, W h and W x is the weight matrix, x t is the input feature of the current time step; Perform preliminary scoring based on the obtained abnormal data, and obtain activity object data with higher scores; Among them, the preliminary scoring is based on the spatial position, duration and behavior type characteristics of the object, and the abnormal data is preliminarily screened to obtain preliminary scoring data.

5. The photovoltaic field drone inspection system according to claim 1, characterized in that: Multi-dimensional composite judgment module, including: Based on the preliminary scoring data obtained, a multi-level threat score is performed on each abnormal data, and a comprehensive threat score is obtained, which is as follows: S threat =w s ·f s +w t ·f t +w b ·f b Where: S threat is a comprehensive threat score used to assess the overall risk level of an activity, s is the spatial position score, indicating the distance of the activity from the photovoltaic panel, f t is the time feature score, indicating the duration of the activity, f b is the behavior type score, indicating the specific type of activity, and w s 、w t 、w b is the weight coefficient of each feature, which is used to balance the influence of different dimensions; Based on the obtained comprehensive threat score, the rule engine makes a composite judgment on the characteristics of the activity; The rule engine includes: Rule 1 and Rule 2; Among them, rule 1, if S threat > threshold and close to photovoltaic panels, it is marked as a high-threat activity; Among them, the threshold is set to be greater than 0; Among them, rule two, if the activity lasts for a long time and the behavior frequency is high, it is marked as suspected anomaly. Prioritize each activity based on compound analysis of the rule engine results and the comprehensive threat score; Among them, high-priority events will trigger immediate alarms, while low-priority events will enter a state of waiting for observation; Automatically adjust scoring weights and rule parameters based on historical alert feedback and environmental parameter data.

6. A photovoltaic field drone inspection system according to claim 1, characterized in that: Feedback and remote monitoring module, including: Based on the obtained comprehensive threat score, the key information of abnormal data is intuitively displayed on the visualization interface in the form of icons, colors and warning marks; Based on the dynamically updated comprehensive threat score, the abnormal data information on the monitoring interface is refreshed in real time; Among them, when an active object is detected close to the photovoltaic panel or the threat score reaches a high threat level, the location, trajectory and threat score information are updated; Based on the obtained comprehensive threat score data, the abnormal data is divided into three levels: high threat, medium threat and low threat; Among them, for high-threat activities, an immediate alert will be triggered and highlighted on the interface; Among them, early warning alerts are sent for medium-threat activities; Among them, low-threat activities will enter observation status; Based on the preset threat score threshold, when the comprehensive threat score exceeds the threshold, an alert message will be pushed to the operator, including the activity location, threat score and handling suggestions; Introducing remote control, the operator adjusts the drone’s patrol route based on the comprehensive threat score; Data is recorded based on all operator intervention instructions and the system's responses.

7. A photovoltaic field drone inspection system according to claim 1, characterized in that: Self-learning optimization module, including: Using an optimization mechanism and based on historical abnormal activity detection results, a comprehensive threat score S is assigned to each activity. threat Conduct analysis and scoring, and determine whether the actual risks are reflected; Adopt optimization mechanisms and conduct attribution analysis based on environmental parameter data; Based on the historical feedback data showing that the spatial position score performs poorly in high threat detection, the spatial position score weight w is reduced s Optimize, specifically: Where: represents the optimized spatial position score weight, R is the target score, and α is the learning rate, which controls the update speed; Based on time score f t and behavior type score f b The weight w t and w b Perform independent optimization to increase the time score weight or behavior type score weight if historical feedback data shows that long-term activities or specific behaviors are more threatening; Based on the latest feedback data and environmental parameters, when the arrival time is set to be greater than 0 days, the spatial location score weight, time score weight, behavior type score weight and threshold are updated; Among them, an adaptive optimization mechanism will be formed based on updating the spatial location scoring weight, time scoring weight, behavior type scoring weight and threshold.

Images