Remote intelligent patrol centralized monitoring system and method for hydropower station

Abstract

The invention discloses a hydropower station remote intelligent patrol centralized monitoring system and method, and relates to the technical field of hydropower station intelligent operation and maintaining.The hydropower station remote intelligent patrol centralized monitoring system comprises a multi-modal data fusion and perception module, an intelligent diagnosis and prediction module, a patrol resource optimization and cooperation module and a database; according to the method, data for comprehensively describing the state of equipment is generated, the GMM is used for dynamically detecting anomalies, an improved GNN and LSTM-Transformer mixed model is combined for recognizing fault modes and predicting faults, tasks are dynamically distributed according to the importance of the equipment and fault risks, meanwhile, instructions are pushed in real time through 5G communication, edge calculation and the LZ77 compression technology, the anomalies of the equipment can be accurately recognized, and the reliability of the equipment is improved. The accuracy of equipment fault mode identification and equipment fault prediction is guaranteed, the utilization rate of inspection resources is improved, and the timeliness of field operation is also guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance of hydropower stations, and particularly relates to a remote intelligent inspection centralized monitoring system and method for hydropower stations. BACKGROUND

[0002] Hydropower stations are important energy bases for the country, and their safe operation is of great significance for ensuring energy supply and social stability. With the development of AI intelligent analysis and cloud computing technologies, technical support is provided for the construction of a remote intelligent inspection centralized monitoring system for hydropower stations.

[0003] The traditional remote intelligent inspection centralized monitoring system and method for hydropower stations analyzes meter identification and appearance inspection through video monitoring, identifies device fault modes and predicts device faults through frameworks such as OpenCV, and uses a fixed cycle mode to inspect each device. Obviously, this remote intelligent inspection centralized monitoring system and method for hydropower stations has the following shortcomings: 1. The video monitoring of the traditional remote intelligent inspection centralized monitoring system and method for hydropower stations can only perform simple appearance analysis, lacks the ability to perceive the internal state of the device, and cannot accurately identify device abnormalities.

[0004] 2. The traditional remote intelligent inspection centralized monitoring system and method for hydropower stations identifies device fault modes and predicts device faults through frameworks such as OpenCV. However, frameworks such as OpenCV are sensitive to factors such as light changes and weather conditions, and have a high false positive rate in actual application, which cannot guarantee the accuracy of device fault mode identification and device fault prediction.

[0005] 3. The traditional remote intelligent inspection centralized monitoring system and method for hydropower stations uses a fixed cycle mode to inspect each device, which cannot be dynamically adjusted according to the actual state of the device, reduces the utilization rate of inspection resources, and there is a significant delay in data interaction between the mobile terminal and the background system, which seriously affects the timeliness of field operations. SUMMARY

[0006] In view of the above technical deficiencies, the present application aims to provide a remote intelligent inspection centralized monitoring system and method for hydropower stations.

[0007] To solve the above technical problems, the present application adopts the following technical solutions: In a first aspect, the present application provides a remote intelligent inspection centralized monitoring system for hydropower stations, comprising the following modules: a multi-modal data fusion and perception module, an intelligent diagnosis and prediction module, an inspection resource optimization and coordination module, and a database.

[0008] The multi-modal data fusion and perception module comprises a data acquisition unit and a data fusion unit.

[0009] The data acquisition unit is used for collecting appearance images, temperature distribution, running sound and physical parameters of each device in real time.

[0010] The data fusion unit is used for extracting image features, thermal features, voiceprint features and time sequence features of physical parameters of each device, and organically combining them to generate data describing the state of each device in all directions, and presenting them to the operation and maintenance personnel.

[0011] The intelligent diagnosis and prediction module comprises an anomaly detection unit and a fault mode identification and prediction unit.

[0012] The anomaly detection unit is used for setting a dynamic anomaly detection threshold for the GNN, and detecting whether each device is abnormal.

[0013] The fault mode identification and prediction unit is used for abstracting each device and its operation data into a graph structure when there is an abnormal device, identifying the fault mode of each abnormal device, and predicting the device fault of each device when there is no abnormal device.

[0014] The inspection resource optimization and cooperation module comprises a task priority scheduling unit and a lightweight data interaction unit.

[0015] The task priority scheduling unit is used for collecting each operation parameter and inspection resource state of each device in real time, generating a priority list, and dynamically adjusting the inspection cycle.

[0016] The lightweight data interaction unit is used for pushing instructions in real time through a 5G network.

[0017] The database is used for storing weight matrices, bias vectors of each layer of the MLP network, standard reference values, maximum values, benchmark thresholds of each working condition parameter, and preset inspection rules, and the importance of each device to the overall operation of the hydropower station and the fault data of each historical fault.

[0018] In a second aspect, the present application provides a hydropower station remote intelligent inspection centralized monitoring method, comprising the following steps: S1, data acquisition: collecting appearance images, temperature distribution, running sound and physical parameters of each device in real time.

[0019] S2, data fusion: extracting image features, thermal features, voiceprint features and time sequence features of physical parameters of each device, and organically combining them to generate data describing the state of each device in all directions, and presenting them to the operation and maintenance personnel.

[0020] S3, anomaly detection: setting a dynamic anomaly detection threshold for the GNN, and detecting whether each device is abnormal.

[0021] S4, fault mode identification and prediction: when there is an abnormal device, each device and its operation data are abstracted into a graph structure, and the fault mode of each abnormal device is identified, and when there is no abnormal device, the device fault of each device is predicted.

[0022] S5, task priority scheduling: real-time collection of each operation parameter and inspection resource state of each device, generation of a priority list, and dynamic adjustment of the inspection cycle.

[0023] S6, lightweight data interaction: real-time pushing of instructions through a 5G network.

[0024] The beneficial effects of the present application are: 1, the present application provides a remote intelligent inspection centralized monitoring system and method for a hydropower station, which combines appearance image, temperature distribution, running sound and physical parameters to generate data describing the state of the device in all directions, and uses GMM to dynamically detect abnormalities, combines improved GNN and LSTM-Transformer hybrid model to identify fault mode and predict fault, and dynamically allocates tasks according to the importance of the device and the fault risk, and uses 5G communication, edge computing and LZ77 compression technology to push instructions in real time, which can accurately identify the abnormality of the device, ensure the accuracy of the device fault mode identification and device fault prediction, improve the utilization rate of inspection resources, and also ensure the timeliness of the on-site operation.

[0025] 2, the present application extracts the image features, thermal features, voiceprint features and time sequence features of the physical parameters of each device, and then normalizes the image feature vectors, thermal feature vectors, voiceprint feature vectors and time sequence feature vectors of the physical parameters of each device, then splices the normalized image feature vectors, thermal feature vectors, voiceprint feature vectors and time sequence feature vectors of the physical parameters of each device in order, to obtain the fusion feature vector of each device, and then calculates the Mahalanobis distance between the real-time feature vector and the normal model to dynamically generate an abnormality detection threshold, which can accurately identify the abnormality of the device.

[0026] 3, the present application abstracts each device and each operation parameter of each device at each time into a dynamic attribute graph, and improves the GNN, identifies the fault mode of each abnormal device through the improved GNN, and inputs each operation parameter of each device at each time into the LSTM in time sequence, to obtain the dependency relationship between each operation parameter of each device, and takes the dependency relationship between each operation parameter of each device as the input of the Transformer, and the Transformer obtains the global features and correlation patterns of the dependency relationship between each operation parameter of each device according to the dependency relationship between each operation parameter of each device, which ensures the accuracy of the device fault mode identification and the device fault prediction.

[0027] 4, Obtain the operation of each device at each time, and monitor the current state of the inspection resource, and obtain the importance of the overall operation of the hydropower station from the database, and evaluate the risk of each device by combining the historical fault data of each device, and assign a priority score to each device, establish a device priority list according to the priority score, and cannot obtain the preset inspection rule from the database, set a reasonable inspection cycle for each device according to the preset inspection rule and the device priority list, and improve the utilization rate of the inspection resource.

[0028] 5, The application uses 5G communication technology and edge computing technology, sets an edge computing node in the hydropower station site, and uses an efficient and low-complexity LZ77 compression algorithm to compress the device operation data and inspection task instructions, uses an industry standard MQTT protocol as a data interaction protocol between the mobile terminal and the background system, unifies the data transmission format as JSON, and constructs a real-time message pushing mechanism according to WebSocket to timely communicate between the background system and the inspection personnel, and guarantee the timeliness of the on-site operation. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the premise of these drawings.

[0030] Figure 1 The system structure connection diagram of the present application.

[0031] Figure 2 The multi-modal data fusion and perception module architecture diagram in the present application.

[0032] Figure 3 The intelligent diagnosis and prediction module flow chart in the present application.

[0033] Figure 4 The method implementation step flow chart of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0035] Please refer to Figure 1As shown, the application provides a remote intelligent inspection centralized monitoring system for a hydropower station, comprising: a multi-modal data fusion and perception module, an intelligent diagnosis and prediction module, an inspection resource optimization and coordination module, and a database.

[0036] The multi-modal data fusion and perception module is connected to the intelligent diagnosis and prediction module, the intelligent diagnosis and prediction module is connected to the inspection resource optimization and coordination module, and the database is connected to the intelligent diagnosis and prediction module and the inspection resource optimization and coordination module.

[0037] As shown in Figure 2 The multi-modal data fusion and perception module comprises a data acquisition unit and a data fusion unit.

[0038] The data acquisition unit is used to acquire real-time images of the appearance of each device, temperature distribution, operating sound, and physical parameters.

[0039] It should be noted that a variety of advanced sensor devices are used to acquire images of the appearance of each device, temperature distribution, operating sound, and physical parameters, such as high-definition cameras, infrared thermal imagers, voiceprint acquisition devices, intelligent inspection robots, and temperature and humidity sensors, vibration sensors, and current sensors.

[0040] It should also be noted that the physical parameters include current and voltage, etc.

[0041] The data fusion unit is used to extract image features, thermal features, voiceprint features, and time sequence features of physical parameters of each device, and to combine them organically to generate data that comprehensively describes the state of the device and presents it to the operation and maintenance personnel.

[0042] In a specific embodiment, the data fusion unit has the following specific process: first, acquire real-time images of the appearance of each device, temperature distribution, operating sound, and physical parameters, and extract image features, thermal features, voiceprint features, and time sequence features of physical parameters of each device; second, perform standardization processing on the image feature vectors, thermal feature vectors, voiceprint feature vectors, and time sequence feature vectors of physical parameters of each device; and third, sequentially splice the standardized image feature vectors, thermal feature vectors, voiceprint feature vectors, and time sequence feature vectors of physical parameters of each device to obtain the fusion feature vector F of each device. a F = [I' a T' a A' a S' a ], where I' a represents the image feature vector of the a-th device after standardization processing, T' a represents the thermal feature vector of the a-th device after standardization processing, A' aa th device after standardization, S' a a th device after standardization, S'

[0043] Wherein, MLP is a multi-layer perception machine.

[0044] It should be noted that the image features are extracted based on a convolutional neural network, the thermal features are extracted by an infrared image segmentation algorithm, the voiceprint features are extracted by using a mel-frequency cepstral coefficient, and the time sequence features of the physical parameters are extracted by using a time sequence analysis algorithm.

[0045] It should also be noted that the image feature vector, the thermal feature vector, the voiceprint feature vector, and the time sequence feature vector of the physical parameters of each device are standardized, so that In the formula, I a , T a , A a , S a respectively represent the image feature vector, the thermal feature vector, the voiceprint feature vector, and the time sequence feature vector of the a th device, max(I a ), min(I a ) respectively represent the maximum and minimum values of the image feature vector of the a th device, max(T a ), min(T a ) respectively represent the maximum and minimum values of the thermal feature vector of the a th device, max(A a ), min(A a ) respectively represent the maximum and minimum values of the voiceprint feature vector of the a th device, max(S a ), min(S a ) respectively represent the maximum and minimum values of the time sequence feature vector of the physical parameters of the a th device.

[0046] In the above, the fusion feature vector of each device is nonlinearly transformed by the MLP network containing three hidden layers to obtain the final feature vector of each device, and the specific process is as follows: h1=σ(W1F a +b1), h2=σ(W2h1+b2), h3=σ(W3h2+b3), Z a =W4h3+b4.

[0047] In the formula, σ represents a ReLU activation function, and Z aThe final feature vector of the a-th device is represented by W1, W2, W3, and W4 represent the weight matrices of the first, second, third, and fourth layers of the MLP network, b1, b2, b3, and b4 represent the bias vectors of the first, second, third, and fourth layers of the MLP network, h1, h2, and h3 represent the outputs of the first, second, and third hidden layers of the MLP network.

[0048] Referring to Figure 3 As shown in the figure, the intelligent diagnosis and prediction module includes an anomaly detection unit and a fault mode identification and prediction unit.

[0049] The anomaly detection unit is configured to set a dynamic anomaly detection threshold for the GNN and detect whether each device is abnormal.

[0050] It should be noted that the GNN is used to detect whether each device is abnormal, and the GNN is a Gaussian mixture model.

[0051] In a specific embodiment, the dynamic anomaly detection threshold is set for the GNN, and the specific process is as follows: the real-time feature vector is obtained, the Mahalanobis distance between the real-time feature vector and the normal model is calculated, and the dynamic anomaly detection threshold is generated. The adjustment formula of the anomaly detection threshold is: In the formula, τ0 represents a reference threshold, T and L represent the working condition parameters of the current working condition, T max and L max represent the maximum value of the allowable working condition parameter T and the maximum value of the allowable working condition parameter L, and α and β represent learnable parameters. T0 and L0 represent the standard reference value of the working condition parameter T and the standard reference value of the working condition parameter L, respectively, and τ represents the adjusted anomaly detection threshold.

[0052] It should be noted that the working condition parameters include temperature and pressure.

[0053] It should also be noted that the normal model refers to a model obtained by jointly modeling the multi-modal feature vectors of the devices in a normal operating state using a Gaussian mixture model.

[0054] The fault mode identification and prediction unit is configured to abstract each device and its operating data into a graph structure when there is an abnormal device, identify the fault mode of each abnormal device, and predict the device fault of each device when there is no abnormal device.

[0055] In a specific embodiment, the fault mode identification and prediction unit specifically processes as follows: each device and each operating parameter of each device at each time is abstracted into a dynamic attribute graph, and is recorded as Gt, where Gt=(V, E, At), V represents a node set of the attribute graph, each element in the node set represents a device, E represents an edge set of the attribute graph, each element in the edge set represents a connection relationship between devices, At represents a node attribute matrix of the attribute graph, the node attribute matrix contains each operating parameter of each device at the tth time, t represents the number of each time, and t is a positive integer.

[0056] It should be noted that the operating parameters include temperature, pressure, vibration frequency, current and voltage, etc.

[0057] When there is an abnormal device, the abnormal devices are referred to as abnormal devices, and the GNN is improved, and the improved GNN is used to identify the fault mode of the abnormal devices, and when there is no abnormal device, the LSTM and the Transformer are mixed to obtain a hybrid model, and the hybrid model is used to predict the device fault of each device.

[0058] In the above, the GNN is improved, and the specific process is as follows: a multi-head improved spatial attention is used, first, the query vector of the ith node and the key vector and the value vector of the jth neighbor node of the ith node are calculated, then In the formula, respectively represent the query vector of the ith node, the key vector of the jth neighbor node, and the value vector of the jth neighbor node under the kth attention head, h i , h j respectively represent the input feature vector of the ith node and the input feature vector of the jth neighbor node, respectively represent the learnable parameter matrix of the kth attention head, q represents the abbreviation of query, k' represents the abbreviation of key, v represents the abbreviation of value, j represents the number of each neighbor node, k represents the number of each attention head, and k and j are positive integers.

[0059] Then, the feature aggregation part outputs the weighted sum of neighbor features of each attention head, and splices the outputs of each attention head, then Where N(i) represents a neighbor node set of the ith node, represents the attention weight coefficient of the ith node and the jth neighbor node thereof under the kth attention head, K represents the total number of attention heads, h i ' represents the output feature vector of the ith node after splicing;

[0060] Finally, the historical feature sequence of the ith node is taken as the input of time convolution, and a gating signal is generated, and feature fusion is performed.

[0061] It should be noted that Z t = sigma'(U z [H t ||M t ]+b z ), wherein Z t represents the gating signal at the t th moment, U z represents a learnable matrix, H t represents the spatial attention output at the t th moment, M t represents the time convolution output at the t th moment, and b z represents a bias vector, and sigma' represents a Sigmoid function, wherein z represents a symbol mark, indicating that U z is a learnable matrix weight related to the gating signal, and b z is a bias vector related to the gating signal.

[0062] It should also be noted that wherein g represents an identifier, U g represents a learnable matrix, b g represents a bias vector, represents the fusion feature vector at the t th moment.

[0063] In the above, the mixing of the LSTM and the Transformer is specifically as follows: each running parameter of each device at each moment is sequentially input into the LSTM according to the time sequence, the dependency relationship between each running parameter of each device is obtained, and the dependency relationship between each running parameter of each device is taken as the input of the Transformer. The Transformer obtains the global feature and the correlation mode of the dependency relationship between each running parameter of each device according to the dependency relationship between each running parameter of each device.

[0064] The patrol resource optimization and cooperation module comprises a task priority scheduling unit and a lightweight data interaction unit.

[0065] The task priority scheduling unit is used for collecting each running parameter of each device and the patrol inspection resource state in real time, generating a priority list, and dynamically adjusting the patrol inspection period.

[0066] In a specific embodiment, the task priority scheduling unit specifically processes as follows: obtaining the running statements of each device at each time, monitoring the current state of the inspection resource, obtaining the importance of each device to the overall operation of the hydropower station from the database, evaluating the risk of each device by combining the historical fault data of each device, assigning a priority score to each device, establishing a device priority list according to the priority score, and setting a reasonable inspection cycle for each device according to the preset inspection rule and the device priority list.

[0067] It should be noted that the risk level of the hydropower station at each time of each device failure is obtained from the database, and the average value of the risk level of the hydropower station at each time of each device failure is calculated. The importance coefficient of each device to the overall operation of the hydropower station and the average value of the risk level of the hydropower station at each time of each device failure are added to obtain the risk coefficient of each device. The risk coefficient of each device is used as the assigned priority of each device, and the device priority list is arranged in order from high to low according to the assigned priority.

[0068] Among them, the importance coefficient of each device to the overall operation of the hydropower station is obtained by controlling the normal operation of other devices and the stop operation of a certain device. The risk level of the hydropower station at each time of each device failure is set by the relevant staff according to the loss of the hydropower station at each time of each device failure. The higher the loss of the hydropower station, the higher the risk level.

[0069] It should also be noted that the inspection cycle of the device under different assigned priorities is obtained from the preset inspection rule. The assigned priority of each device is compared with each assigned priority in the preset inspection rule. If the assigned priority of a certain device is the same as a certain assigned priority in the preset inspection rule, the inspection cycle of the device under the assigned priority is obtained from the preset inspection rule, which is used as the inspection cycle of the device. In this way, the inspection cycle of each device is obtained.

[0070] It should be explained that the higher the assigned priority, the shorter the inspection cycle.

[0071] The lightweight data interaction unit is used to push instructions in real time through a 5G network.

[0072] In a specific embodiment, the lightweight data interaction unit specifically processes as follows: using 5G communication technology and edge computing technology, setting an edge computing node on the spot of the hydropower station, and using an efficient and low-complexity LZ77 compression algorithm to compress the device operation data and inspection task instructions. At the same time, an industry-standard MQTT protocol is used as the data interaction protocol between the mobile terminal and the background system, the unified data transmission format is JSON, and a real-time message pushing mechanism is constructed according to WebSocket to timely communicate between the background system and the inspection personnel.

[0073] It should be noted that the use of 5G communication technology and edge computing technology can speed up data processing and transmission speed, the use of efficient and low complexity LZ77 compression algorithm can reduce the amount of data transmission, the use of industry standard MQTT protocol can ensure the accuracy and stability of data transmission, and the use of WebSocket to build real-time message push mechanism can push real-time messages to ensure that the inspection personnel can obtain key information in the first time, and improve the response speed and efficiency of inspection operation.

[0074] The database is used for storing the weight matrix and bias vector of each layer of the MLP network, the standard reference value, the maximum value, the reference threshold value and the preset inspection rule of each working condition parameter, and the importance of each device to the overall operation of the hydropower station and the fault data of each historical fault.

[0075] Please refer to Figure 4 As shown in the figure, the present application provides a centralized monitoring method for remote intelligent inspection of a hydropower station, comprising: S1, data acquisition: real-time acquisition of appearance images, temperature distribution, running sound and physical parameters of each device.

[0076] S2, data fusion: extracting image features, thermal features, voiceprint features and time sequence features of physical parameters of each device, and organically combining to generate data describing the state of the device in all directions, and presenting to the operation and maintenance personnel.

[0077] S3, anomaly detection: setting a dynamic anomaly detection threshold for GNN to detect whether each device is abnormal.

[0078] S4, fault mode identification and prediction: when there is an abnormal device, abstract each device and its operation data into a graph structure, identify the fault mode of each abnormal device, and when there is no abnormal device, predict the device fault of each device.

[0079] S5, task priority scheduling: real-time collection of each operation parameter and inspection resource state of each device, generation of a priority list, and dynamic adjustment of the inspection cycle.

[0080] S6, lightweight data interaction: real-time pushing of instructions through 5G network.

[0081] The embodiment of the application fuses appearance images, temperature distribution, running sound and physical parameters to generate data for comprehensively describing the equipment state, dynamically detects abnormalities by using GMM, combines the improved GNN and LSTM-Transformer hybrid model to identify fault modes and predict faults, dynamically allocates tasks according to the importance of the equipment and the fault risk, simultaneously uses 5G communication, edge computing and LZ77 compression technology to push instructions in real time, can accurately identify the abnormalities of the equipment, guarantees the accuracy of equipment fault mode identification and equipment fault prediction, improves the utilization rate of patrol resources, and guarantees the timeliness of on-site operation.

[0082] The above is only an example and description of the concept of the present application, and those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace, as long as the modifications or supplements do not deviate from the concept of the present application or exceed the scope defined in the specification, which shall belong to the protection scope of the present application.

Claims

1. A remote intelligent patrol centralized monitoring system for a hydropower station, characterized in that: Includes the following modules: The multimodal data fusion and perception module includes a data acquisition unit and a data fusion unit: The data acquisition unit is used to collect the appearance image, temperature distribution, operating sound and physical parameters of each device in real time; The data fusion unit is used to extract the image features, thermal features, voiceprint features and time series features of physical parameters of each device, and organically combine them to generate data that fully describes the device status and presents it to the operation and maintenance personnel; The intelligent diagnosis and prediction module includes an anomaly detection unit and a fault pattern recognition and prediction unit: The anomaly detection unit is used to set a dynamic anomaly detection threshold for the GNN to detect whether each device is abnormal; The fault pattern recognition and prediction unit is used to abstract each device and its operating data into a graph structure when an abnormal device exists, identify the fault pattern of each abnormal device, and predict the device failure of each device when no abnormal device exists; The patrol resource optimization and coordination module includes a task priority scheduling unit and a lightweight data interaction unit: The task priority scheduling unit is used to collect the operating parameters and inspection resource status of each device in real time, generate a priority list, and dynamically adjust the inspection cycle; The lightweight data interaction unit is used to push instructions in real time through the 5G network; The database is used to store the weight matrix, bias vector, standard reference value, maximum value, benchmark threshold and preset inspection rules of each operating parameter of each layer of the MLP network, as well as the importance of each equipment to the overall operation of the hydropower station and the fault data of each historical fault.

2. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 1, characterized in that: The data fusion unit has the following specific process: First, the real-time appearance image, temperature distribution, operating sound and physical parameters of each device are obtained, and the image features, thermal features, voiceprint features and time series features of the physical parameters of each device are extracted. Then, the image feature vector, thermal feature vector, voiceprint feature vector and time series feature vector of the physical parameters of each device are standardized. Then, the image feature vector, thermal feature vector, voiceprint feature vector and time series feature vector of the physical parameters of each device after the standardized processing are spliced ​​in sequence to obtain the fusion feature vector F of each device. a , then F a =[I′ a |T′ a |A′ a |S′ a ], where I′ a Represents the image feature vector of the ath device after normalization, T a Represents the thermal feature vector of the ath device after normalization, A a represents the voiceprint feature vector of the ath device after normalization, S a The time series feature vector representing the physical parameters of the ath device after standardization, where a represents the device number and is a positive integer, is then used to perform a nonlinear transformation on the fused feature vectors of each device through an MLP network containing three hidden layers to obtain the final feature vectors of each device. Finally, the final feature vectors of each device are input into a random forest classifier to classify the status of each device, and the classification results are presented to the operation and maintenance personnel.

3. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 2, characterized in that: The fusion feature vectors of each device are nonlinearly transformed by an MLP network containing three hidden layers to obtain the final feature vectors of each device. The specific process is as follows: h1=σ(W1F a +b1); h2=σ(W2h1+b2); h3=σ(W3h2+b3); Z a =W4h3+b4; Where σ represents the ReLU activation function, Z a represents the final feature vector of the ath device, W1 represents the weight matrix of the first layer of the MLP network, W2 represents the weight matrix of the second layer of the MLP network, W3 represents the weight matrix of the third layer of the MLP network, W4 represents the weight matrix of the fourth layer of the MLP network, b1 represents the bias vector of the first layer of the MLP network, b2 represents the bias vector of the second layer of the MLP network, b3 represents the bias vector of the third layer of the MLP network, b4 represents the bias vector of the fourth layer of the MLP network, h1 represents the output of the first hidden layer of the MLP network, h2 represents the output of the second hidden layer of the MLP network, and h3 represents the output of the third hidden layer of the MLP network.

4. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 1, characterized in that: The dynamic anomaly detection threshold is set for GNN. The specific process is as follows: Obtain the real-time feature vector, calculate the Mahalanobis distance between the real-time feature vector and the normal model, and dynamically generate the anomaly detection threshold. The adjustment formula for the anomaly detection threshold is: Where τ0 represents the reference threshold, T and L represent the working condition parameters of the current working condition, T max and L max Both represent the maximum value of the operating condition parameter T and the maximum value of the operating condition parameter L, α and β both represent learnable parameters, T0 and L0 represent the standard reference value of the operating condition parameter T and the standard reference value of the operating condition parameter L, respectively, and τ represents the adjusted anomaly detection threshold.

5. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 1, characterized in that: The specific process of the fault mode identification and prediction unit is as follows: Each device and its operating parameters at each moment are abstracted into a dynamic attribute graph, which is denoted by Gt. Then Gt = (V, E, At), where Gt represents the attribute graph at the tth moment, V represents the node set of the attribute graph, and each element in the node set represents each device. E represents the edge set of the attribute graph, and each element in the edge set represents the connection relationship between devices. At represents the node attribute matrix of the attribute graph, which contains the operating parameters of each device at the tth moment. t represents the number of each moment, and t is a positive integer. When there are abnormal devices, each abnormal device is called an abnormal device, and the GNN is improved to identify the failure mode of each abnormal device. When there are no abnormal devices, the LSTM and Transformer are mixed to obtain a hybrid model, and the hybrid model is used to predict the equipment failure of each device.

6. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 5, characterized in that: The specific process of improving GNN is as follows: Using multi-head improved spatial attention, we first calculate the query vector of the i-th node, as well as the key vector and value vector of the j-th neighbor node of the i-th node, then In the formula V i k They represent the query vector of the i-th node under the k-th attention head, the key vector of the j-th neighbor node, and the value vector of the j-th neighbor node, respectively. i 、h j Represent the input feature vector of the i-th node and the input feature vector of the j-th neighbor node respectively, They represent the learnable parameter matrices of the kth attention head, q represents the abbreviation of query, k′ represents the abbreviation of key, v represents the abbreviation of value, j represents the number of each neighbor node, and k represents the number of each attention head. k and j are positive integers. Then, in the feature aggregation part, each attention head outputs the weighted sum of neighbor features, and the outputs of each attention head are spliced ​​together. Where N(i) represents the set of neighbor nodes of the i-th node, represents the attention weight coefficient of the i-th node and its j-th neighbor node under the k-th attention head, K represents the total number of attention heads, and h i ′ represents the output feature vector of the i-th node after splicing is completed; Finally, the historical feature sequence of the i-th node is used as the input of the temporal convolution, and the gating signal is generated, and feature fusion is performed at the same time.

7. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 5, characterized in that: The specific process of mixing LSTM and Transformer is as follows: The operating parameters of each device at each moment are input into the LSTM in chronological order to obtain the dependency relationship between the operating parameters of each device. The dependency relationship between the operating parameters of each device is used as the input of the Transformer. Based on the dependency relationship between the operating parameters of each device, the Transformer obtains the global features and correlation patterns of the dependency relationship between the operating parameters of each device.

8. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 1, characterized in that: The task priority scheduling unit has the following specific process: Obtain the operation description of each device at each moment, and monitor the current status of inspection resources. At the same time, obtain the importance of each device to the overall operation of the hydropower station from the database, and evaluate the risk of each device in combination with the fault data of each device's historical faults. At the same time, assign a priority score to each device, and establish a device priority list based on the priority score. If the preset inspection rules cannot be obtained from the database, set a reasonable inspection cycle for each device based on the preset inspection rules and equipment priority list.

9. A remote intelligent patrol centralized monitoring system for a hydropower station according to claim 1, characterized in that: The specific process of the lightweight data interaction unit is as follows: Using 5G communication technology and edge computing technology, edge computing nodes are set up at the hydropower station site, and the efficient and low-complexity LZ77 compression algorithm is adopted to compress equipment operation data and inspection task instructions. At the same time, the industry-standard MQTT protocol is used as the data interaction protocol between mobile terminals and backend systems. The data transmission format is unified to JSON, and a real-time message push mechanism is built based on WebSocket to enable timely communication between the backend system and inspection personnel.

10. A method for remote intelligent patrol and centralized monitoring of a hydropower station using a remote intelligent patrol and centralized monitoring system for a hydropower station according to any one of claims 1 to 9, characterized in that: include: S1. Data collection: Real-time collection of appearance images, temperature distribution, operating sounds and physical parameters of each device; S2. Data Fusion: Extract the image features, thermal features, voiceprint features, and time series features of physical parameters of each device, organically combine them to generate data that fully describes the device status, and present it to the operation and maintenance personnel; S3. Anomaly detection: Set dynamic anomaly detection thresholds for GNN to detect whether each device is abnormal. S4. Fault pattern identification and prediction: When there are abnormal devices, each device and its operating data are abstracted into a graph structure to identify the failure mode of each abnormal device. If there are no abnormal devices, the device failure of each device is predicted; S5. Task priority scheduling: collect the operating parameters and inspection resource status of each device in real time, generate a priority list, and dynamically adjust the inspection cycle; S6. Lightweight data interaction: Push instructions in real time through the 5G network.

Images