Substation equipment detection and inspection method, device, equipment and medium

Abstract

The invention discloses a substation equipment detection and inspection method, device, equipment and medium, relates to the technical field of substation inspection, and is used for solving the technical problems of low defect identification accuracy and the like in the prior art. The method comprises the following steps: carrying out multi-modal data acquisition on target substation equipment to obtain target multi-source detection data; performing fault identification on the target multi-source detection data by adopting the trained fault identification model to obtain an identification result; wherein the trained fault recognition model comprises an LSTM (Long Short Term Memory) layer, a Transform layer, a feature fusion layer and an output layer, and the LSTM layer is parallel to the Transform layer; the recognition result is matched with fault mode data in a preset fault mode library, and whether the target substation equipment breaks down or not is determined; if it is determined that the target substation equipment breaks down, early warning is triggered, and a fault diagnosis report is generated, so that the defect recognition accuracy can be greatly improved through deep combination of a multi-modal data fusion technology and an intelligent algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of substation inspection, and provides a substation equipment detection and inspection method, device, equipment and medium. BACKGROUND

[0002] For substation equipment detection and inspection, manual inspection is currently used to detect and inspect substation equipment. However, since manual inspection relies on visual inspection, the single inspection cycle is long, and it is difficult to quickly respond to sudden abnormalities of the equipment, so the operation and maintenance efficiency is low. Secondly, since the detection process of manual inspection highly depends on the individual experience and technical level of the operation and maintenance personnel, the judgment of the equipment state is easily affected by subjective factors, so it is difficult to ensure the consistency and accuracy of the interpretation results. In addition, since manual inspection needs to frequently enter complex environments such as high-voltage live working, multiple safety hazards are faced in harsh working conditions or special areas, so there is a high operation safety risk.

[0003] In addition, the existing substation inspection defect recognition method also has many shortcomings. Specifically, first, the multi-source heterogeneous data fusion analysis capability is lacking, and lacks deep fusion algorithm support, which cannot meet the complex working condition detection requirements. Secondly, the three-dimensional scene interaction capability is insufficient, and the traditional display method based on two-dimensional monitoring pictures cannot intuitively present the spatial position relationship and three-dimensional layout of the equipment, there are visual angle blind areas and low abnormal point positioning efficiency. At the same time, the existing intelligent diagnosis algorithm also has shortcomings, for example, it lacks dynamic adaptability to the real-time running condition of the equipment, has high false alarm and missed alarm risks, and lacks deep mining capability for historical data. The predictive maintenance capability is missing, the alarm information reminding is delayed, and there is a lack of time series analysis and trend prediction capability for equipment state data.

[0004] Therefore, an efficient, self-adaptive and multi-modal fusion substation equipment inspection method is needed to realize substation equipment detection and inspection management. SUMMARY

[0005] The present application provides a substation equipment detection and inspection method, device, equipment and medium, which solves the technical problems of low defect recognition accuracy in the prior art.

[0006] In one aspect, a substation equipment detection and inspection method is provided, the method comprising: performing multi-modal data acquisition on a target substation equipment to obtain target multi-source detection data; obtaining an identification result, wherein the trained fault identification model comprises an LSTM layer, a Transformer layer, a feature fusion layer and an output layer, the LSTM layer and the Transformer layer are parallel, and the identification result comprises a device health score and a fault probability distribution; matching the identification result with fault mode data in a preset fault mode library to determine whether the target substation device has a fault; if it is determined that the target substation device has a fault, triggering an early warning and generating a fault diagnosis report.

[0007] Optionally, the step of collecting multi-modal data of the target substation device to obtain target multi-source detection data comprises: collecting multi-modal data of the target substation device according to various sensors installed on the surface and inside of the target substation device to obtain initial multi-source detection data; performing denoising processing on the initial multi-source detection data by using a preset wavelet threshold denoising algorithm to obtain the target multi-source detection data.

[0008] Optionally, the step of obtaining an identification result by using a trained fault identification model to perform fault identification on the target multi-source detection data comprises: performing calibration preprocessing on the target multi-source detection data by using a space-time alignment algorithm to obtain calibrated multi-source detection data; performing fault identification on the calibrated multi-source detection data by using the trained fault identification model to obtain the identification result.

[0009] Optionally, the step of obtaining an identification result by using a trained fault identification model to perform fault identification on the calibrated multi-source detection data comprises: extracting features of the calibrated multi-source detection data by using an LSTM layer in the trained fault identification model to obtain a local time sequence feature vector; extracting features of the calibrated multi-source detection data by using a Transformer layer in the trained fault identification model to obtain a global dependence feature vector; performing feature fusion on the local time sequence features and the global dependence features by using a feature fusion layer in the trained fault identification model to obtain a fusion feature vector; performing feature processing on the fusion features by using an output layer in the trained fault identification model to obtain the identification result.

[0010] Optionally, the step of matching the identification result with fault mode data in a preset fault mode library to determine whether the target substation equipment has a fault comprises: determining a target fault type according to the fault probability distribution in the identification result; matching a feature matching vector of the same fault type as the identification result from the preset fault mode library according to the target fault type; calculating a target similarity between the fusion feature vector and the feature matching vector; determining whether the target similarity exceeds a set similarity threshold; if it is determined that the target similarity exceeds the set similarity threshold, determining that the target substation equipment has a fault.

[0011] Optionally, after the trained fault identification model is used to identify faults in the target multi-source detection data to obtain an identification result, the method further comprises: constructing a virtual substation scene using a Unity 3D tool and a geographic information system (GIS); completing multi-modal interaction control between a user and the virtual substation scene according to multi-modal data collected by an intelligent wearable device; the multi-modal data is obtained through a gesture instruction or a voice instruction.

[0012] Optionally, after the identification result is matched with data in a preset fault mode library to determine whether to trigger a fault early warning, the method further comprises: establishing an end-to-end real-time communication connection according to WebRTC technology; synchronously transmitting multi-modal interaction control data, target substation equipment real-time data, and operation pictures of an operation and maintenance personnel to a dedicated client according to the real-time communication connection; in the dedicated client, using a marking tool to mark a fault position of the virtual substation scene, and guiding an on-site operation and maintenance personnel to handle the fault through voice guidance.

[0013] In one aspect, a substation equipment detection and inspection device is provided, and the device comprises: a multi-modal data acquisition unit configured to acquire multi-modal data of target substation equipment to obtain target multi-source detection data; an identification result obtaining unit configured to identify faults in the target multi-source detection data using a trained fault identification model to obtain an identification result; the trained fault identification model comprises an LSTM layer, a Transformer layer, a feature fusion layer, and an output layer, and the LSTM layer and the Transformer layer are parallel; the identification result comprises an equipment health score and a fault probability distribution. a fault determination unit configured to match the identification result with fault mode data in a preset fault mode library to determine whether the target substation equipment has a fault; a pre-warning and diagnosis unit configured to trigger a pre-warning and generate a fault diagnosis report if it is determined that the target substation equipment has a fault.

[0014] In one aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the above methods when executing the computer program.

[0015] In one aspect, a storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement any of the above methods.

[0016] Compared with the prior art, the application has the following beneficial effects: In the present application, first, multi-modal data of the target substation equipment can be collected to obtain target multi-source detection data; then, a trained fault identification model can be used to identify faults of the target multi-source detection data to obtain an identification result; wherein the trained fault identification model includes an LSTM layer, a Transformer layer, a feature fusion layer, and an output layer, and the LSTM layer and the Transformer layer are parallel; the identification result includes an equipment health score and a fault probability distribution; next, the identification result can be matched with fault mode data in a preset fault mode library to determine whether the target substation equipment has a fault; finally, if it is determined that the target substation equipment has a fault, a pre-warning can be triggered, and a fault diagnosis report can be generated.

[0017] Based on this, in the present application, since multi-modal data of the target substation equipment is collected and a trained fault identification model is used to identify faults of the collected target multi-source detection data, the present application can break through the limitations of single sensor or fixed threshold analysis by deep combination of multi-modal data fusion technology and intelligent algorithm, realize comprehensive perception and accurate judgment of the state of equipment appearance defects, abnormal operation parameters, etc., to effectively reduce missed detection and misjudgment, and significantly improve the accuracy and reliability of equipment defect identification. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments or the related art, the following will briefly introduce the drawings needed to be used in the embodiment or related art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without any creative effort.

[0019] Figure 1 An application scenario schematic diagram provided for an embodiment of the present application; Figure 2 An application scenario schematic diagram provided for an embodiment of the present application; Figure 3 An application scenario schematic diagram provided for an embodiment of the present application; Figure 4 An application scenario schematic diagram provided for an embodiment of the present application.

[0020] In the figure, 10 is a substation equipment detection and inspection device, 101 is a processor, 102 is a memory, 103 is an I / O interface, 104 is a database, 40 is a substation equipment detection and inspection device, 401 is a multi-modal data acquisition unit, 402 is an identification result obtaining unit, 403 is a fault determination unit, 404 is a pre-warning and diagnosis unit, 405 is a scene construction and interaction control unit, and 406 is a remote collaborative operation and maintenance unit. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application. The embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily without conflict. Moreover, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0022] For substation equipment detection and inspection, manual inspection is usually used to detect and inspect substation equipment. However, since manual inspection relies on visual inspection, the single inspection cycle is long, and it is difficult to respond quickly to sudden abnormalities of the equipment, so the operation and maintenance efficiency is low. Secondly, since the detection process of manual inspection highly depends on the individual experience and technical level of the operation and maintenance personnel, the judgment of the equipment state is easily affected by subjective factors, so it is difficult to ensure the consistency and accuracy of the interpretation results. In addition, since manual inspection needs to frequently enter complex environments such as high-voltage live working, multiple safety hazards are faced in harsh working conditions or special areas, so there is a high operation safety risk.

[0023] Furthermore, existing substation inspection defect identification methods have several shortcomings. Specifically, firstly, they lack the ability to fuse and analyze multi-source heterogeneous data, lacking deep fusion algorithms to meet the needs of complex operating conditions. Secondly, their 3D scene interaction capabilities are insufficient; traditional display methods based on 2D monitoring screens cannot intuitively present the spatial relationships and 3D layout of equipment, resulting in blind spots and low efficiency in anomaly point location. Simultaneously, existing intelligent diagnostic algorithms also have shortcomings. For example, they lack dynamic adaptability to real-time equipment operating conditions, leading to a high risk of false alarms and missed alarms, and lack the ability to deeply mine historical data. Predictive maintenance capabilities are lacking, alarm information is delayed, and there is a lack of time-series analysis and trend prediction capabilities for equipment status data.

[0024] Based on this, this application provides a method for substation equipment detection and inspection. In this method, firstly, multimodal data acquisition is performed on the target substation equipment to obtain target multi-source detection data. Then, a trained fault identification model is used to identify faults in the target multi-source detection data to obtain identification results. The trained fault identification model includes an LSTM layer, a Transformer layer, a feature fusion layer, and an output layer, with the LSTM layer and Transformer layer operating in parallel. The identification results include equipment health scores and fault probability distributions. Next, the identification results are matched with fault mode data in a preset fault mode library to determine whether the target substation equipment has experienced a fault. Finally, if a fault is determined in the target substation equipment, an early warning is triggered, and a fault diagnosis report is generated. Based on this, in this application, since multimodal data is collected from the target substation equipment and a trained fault identification model is used to identify faults in the collected multi-source detection data, this application can overcome the limitations of single sensor or fixed threshold analysis by deeply combining multimodal data fusion technology with intelligent algorithms. This enables comprehensive perception and accurate judgment of equipment appearance defects, abnormal operating parameters, and other states, effectively reducing missed detections and misjudgments, and significantly improving the accuracy and reliability of equipment defect identification.

[0025] After introducing the design concept of the embodiments of this application, the following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application can be applied. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0026] like Figure 1 The diagram shown is an application scenario illustration provided by an embodiment of this application. This application scenario may include substation equipment detection and inspection equipment 10.

[0027] The transformer substation equipment detection and inspection device 10 can be used to detect and inspect the transformer substation equipment, for example, can be a vehicle-mounted computer, a personal computer (PC), a server, a laptop, etc. The transformer substation equipment detection and inspection device 10 can include one or more processors 101, a memory 102, an I / O interface 103, and a database 104. Specifically, the processor 101 can be a central processing unit (CPU), or a digital processing unit, etc. The memory 102 can be a volatile memory, for example, a random-access memory (RAM); the memory 102 can also be a non-volatile memory, for example, a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 102 can be any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and accessible by a computer, but not limited to this. The memory 102 can be a combination of the above memories. The memory 102 can store part of the program instructions of the transformer substation equipment detection and inspection method provided in the embodiments of the present application, and when the processor 101 executes these program instructions, the steps of the transformer substation equipment detection and inspection method provided in the embodiments of the present application can be implemented to solve the technical problems of low defect identification accuracy of the existing transformer substation equipment detection and inspection method. The database 104 can be used to store the target multi-source detection data, the trained fault identification model, the identification result, and the fault mode data involved in the scheme provided in the embodiments of the present application.

[0028] In the embodiments of the present application, the transformer substation equipment detection and inspection device 10 can obtain the transformer substation equipment inspection instruction through the I / O interface 103, and then the processor 101 of the transformer substation equipment detection and inspection device 10 can solve the technical problems of low defect identification accuracy of the existing transformer substation equipment detection and inspection method according to the program instructions of the transformer substation equipment detection and inspection method provided in the embodiments of the present application in the memory 102. In addition, the target multi-source detection data, the trained fault identification model, the identification result, and the fault mode data can be stored in the database 104.

[0029] Of course, the method provided in the embodiments of the present application is not limited to the application scenarios shown in the above Figure 1 The embodiments of the present application are not limited to the application scenarios shown in the above Figure 1The functions that can be achieved by the various devices of the application scenarios shown will be described in subsequent method embodiments, and will not be described in detail here. In the following, the method of the embodiments of the application will be introduced in conjunction with the drawings.

[0030] As shown in Figure 2 , it is a flowchart of a substation equipment detection and inspection method provided by the embodiments of the application. The method can be executed by the substation equipment detection and inspection device 10 in Figure 1 . Specifically, the flow of the method is introduced as follows.

[0031] Step 201: Multi-modal data acquisition is performed on the target substation equipment to obtain target multi-source detection data.

[0032] Specifically, when performing "multi-modal data acquisition", first, the target substation equipment can be subjected to multi-modal data acquisition according to various sensors installed on the surface and inside of the target substation equipment, and various detection data can be summarized to obtain initial multi-source detection data. Then, a preset wavelet threshold denoising algorithm can be used to perform denoising processing on the initial multi-source detection data to obtain target multi-source detection data, which is convenient for the next step of data fusion processing.

[0033] In the embodiments of the application, the preset wavelet threshold denoising algorithm is an improved wavelet threshold denoising algorithm, and the improved wavelet threshold denoising algorithm can be represented by the following wavelet threshold function formula:

[0034] wherein, is the wavelet coefficient after denoising, is the original wavelet coefficient, sgn() is a sign function, is a threshold, is an adjustment parameter. Furthermore, the application can determine the optimal value by using an adaptive optimization algorithm to effectively remove white noise and pulse interference in the collected data.

[0035] Step 202: The trained fault identification model is used to identify the target multi-source detection data to obtain an identification result.

[0036] In the embodiments of the application, as shown in Figure 3 , it is a structure diagram of a fault identification model provided by the embodiments of the application. The trained fault identification model includes an LSTM layer, a Transformer layer, a feature fusion layer and an output layer, and the LSTM layer and the Transformer layer are parallel. The identification result includes a device health score and a fault probability distribution.

[0037] Specifically, in the process of adopting the trained fault identification model to perform fault identification on the target multi-source detection data and obtaining the identification result, the target multi-source detection data is obtained based on the "multi-modal data acquisition". First, the target multi-source detection data can be calibrated and preprocessed by using a space-time alignment algorithm to obtain calibrated multi-source detection data. That is, the target multi-source detection data can be processed by space-time alignment, wherein, in the process of time alignment, interpolation and synchronization can be performed based on the timestamp; in the process of space calibration, space unification mapping can be completed according to the spatial coordinate information of the target transformer station equipment installation; then, in order to improve the training stability and convergence efficiency of the subsequent fault identification model, the target multi-source detection data processed by space-time alignment can also be standardized, that is, different modal data can be normalized.

[0038] Then, the trained fault identification model (i.e., the improved LSTM-Transformer fusion neural network model) can be used to perform fault identification on the calibrated multi-source detection data to obtain the identification result, so as to improve the fusion effect and fault identification ability by taking into account the time sequence feature extraction and global context modeling ability.

[0039] That is, as shown in Figure 3 , first, the LSTM layer in the trained fault identification model is used to extract features from the calibrated multi-source detection data to obtain a local time sequence feature vector. In the embodiment of the present application, when the LSTM layer is used to obtain the local time sequence feature vector, it can be assumed that the input multi-source detection data is , which is recursively processed by the LSTM layer, and the forward calculation process is as follows:

[0040] Wherein, σ is the Sigmoid function; ⊙ is the Hadamard product; x_t is the input at time t; h_{t-1} is the hidden state at the previous time; c_{t-1} is the cell state at the previous time; i_t, f_t, o_t are the input gate, the forget gate and the output gate, respectively; c_t, h_t are the current cell and hidden state, respectively; tanh() is the hyperbolic tangent function; W_i, W_f, W_o and W_c are the weight matrices of the input gate, the forget gate, the output gate and the current cell, respectively.

[0041] Then, the Transformer layer in the trained fault identification model can be used to extract features from the calibrated multi-source detection data to obtain a global dependency feature vector. In the embodiment of the present application, when the Transformer layer is used to obtain the local time sequence feature vector, the Transformer Encoder (i.e., the Transformer encoder) can be introduced, and its execution process is as follows:

[0042] wherein Attention() is an attention mechanism, softmax() is a normalized exponential function, MultiHead() is multi-head, Concat() is a function for connecting two or more arrays, Q, K, and V are query matrix, key matrix, and value matrix respectively; d_k is the dimension of the key vector; √d_k is the dimension scaling factor of the key vector; h is the number of attention heads; W_i^Q, W_i^K, W_i^V, and W^O are trainable projection matrices.

[0043] Next, the local time sequence features and the global dependency features can be fused by using the feature fusion layer in the trained fault identification model to obtain a fusion feature vector. In the embodiments of the present application, the fusion formula used for feature fusion is as follows:

[0044] wherein H_LSTM and H_Trans are the outputs of the LSTM layer and the Transformer layer respectively; a is a trainable fusion coefficient, and a e [0, 1] is a trainable fusion coefficient.

[0045] Finally, the fusion features can be processed by using the output layer in the trained fault identification model to obtain an identification result. In the embodiments of the present application, the device health score and the fault probability distribution in the identification result can be represented by using the following formula: Device health score:

[0046] Fault probability distribution:

[0047] Further, after obtaining the identification result (S, P), (S, P) can be stored in association with the calibrated multi-source detection data and transmitted to a subsequent module.

[0048] Step 203: Matching the identification result with the fault mode data in the preset fault mode library to determine whether the target substation device has failed.

[0049] Specifically, first, the target fault type can be determined according to the fault probability distribution in the identification result; then, the feature matching vector of the same fault type as the identification result can be matched from the preset fault mode library (covering common fault types and feature matching vectors) according to the target fault type; next, the target similarity between the fusion feature vector and the feature matching vector can be calculated; then, it can be determined whether the target similarity exceeds the set similarity threshold; finally, if it is determined that the target similarity exceeds the set similarity threshold, it can be determined that the target substation equipment has a fault. On the contrary, if it is determined that the target similarity does not exceed the set similarity threshold, it can be determined that the target substation equipment has no fault.

[0050] Step 204: If it is determined that the target substation equipment has a fault, triggering a pre-warning and generating a fault diagnosis report.

[0051] Specifically, after triggering the pre-warning, the fault tree analysis (FTA) algorithm can be combined to deduce the fault cause layer by layer, generate a fault diagnosis report and push it to the relevant personnel.

[0052] In one possible implementation, in order to reduce the operation safety risk, in the embodiments of the present application, after the trained fault identification model is used to identify the faults of the target multi-source detection data and the identification result is obtained, “VR scene construction” and “multi-modal interaction control” can be performed.

[0053] Specifically, when performing “VR scene construction”, the Unity 3D tool and the geographic information system (GIS) can be used to construct a virtual substation scene, and the device state visualization effect can be realized through programming.

[0054] When performing “multi-modal interaction control”, multi-modal interaction control between the user and the virtual substation scene can be completed according to multi-modal data collected by the intelligent wearable device; wherein the multi-modal data is obtained through line of sight, gesture instruction or voice instruction.

[0055] Further, when gesture recognition is performed through a gesture instruction, multi-modal interaction control can be performed through the following steps: First, data collection can be performed. That is, the user's hand key point sequence K={k1, k2,…,k_T} can be captured through a depth camera combined with the Open Source Computer Vision Library (OpenCV).

[0056] Then, feature extraction can be performed. Namely, a Convolutional Neural Network (CNN) can be employed to extract local spatial feature vectors; and a Graph Convolutional Network (GCN) can be used to model the keypoint topological relationship.

[0057] Next, encoding output can be performed. Namely, a gesture feature vector g = GCN(CNN(K)) can be generated for subsequent interaction action mapping.

[0058] In addition, when voice recognition is performed through a voice instruction, multi-modal interaction control can be performed through the following steps: First, voice data acquisition can be performed. Namely, a voice stream V can be acquired in real time through a device microphone.

[0059] Then, automatic speech recognition (ASR) can be performed. Namely, an automatic speech recognition ASR module can be used to convert the voice stream V into a text sequence T = {t1, t2,…, t_n}.

[0060] Then, semantic analysis can be performed. Namely, a pre-trained language model based on Bidirectional Encoder Representations from Transformers (BERT) can be introduced to perform joint intent recognition and slot filling, and the recognition and filling process is as follows:

[0061] wherein T is the input text sequence {t1, t2,…, t_n}, E is a context representation matrix [E_[CLS], E1,…, E_n], E_[CLS] is a whole sentence feature vector, Ej is a feature vector of the jth word, ŷ is an intent classification probability, 、 are the intent recognition layer and slot labeling layer weights, respectively, is a jth word slot prediction probability vector, 、 is a bias term.

[0062] Next, instruction generation and execution can be performed. That is, instruction mapping, scene control, and execution feedback mechanism can be performed. Specifically, when instruction mapping is performed, the intent category y and the slot label {s_j} can be mapped into a control command C; when scene control is performed, the virtual scene can be operated by executing the command C, such as device query, abnormality positioning, and view switching; when the execution feedback mechanism is performed, natural language feedback and graphical prompts can be generated according to the execution result to enhance the user interaction experience.

[0063] In a possible implementation, in order to improve the training and collaboration efficiency, in the embodiment of the present application, after the matching of the recognition result and the data in the preset fault mode library is performed to determine whether to trigger a fault warning, if it is determined that the target substation equipment is abnormal and the operation and maintenance personnel cannot confirm it, a remote collaboration request can be initiated by the system to perform “remote collaborative operation and maintenance”.

[0064] Specifically, first, an end-to-end real-time communication connection can be established according to the Web Real-Time Communication (WebRTC) technology.

[0065] Next, the multi-modal interaction control data, the target substation equipment real-time data, and the operation and maintenance personnel operation screen can be synchronously transmitted to the dedicated client according to the real-time communication connection.

[0066] Then, the expert can view the information in the dedicated client and mark the fault position of the virtual substation scene by using a marking tool, and guide the on-site operation and maintenance personnel to handle the fault by voice, wherein the operation record and the guidance information are real-time returned and stored.

[0067] In the embodiment of the present application, the following video encoding algorithm can be used to ensure the stable transmission of VR scene data under different network environments:

[0068] wherein R is a target code rate, N is the number of video frames, Qi is a quantization parameter of the i-th frame, Si is the size of the i-th frame, is an adjustment coefficient.

[0069] In summary, the present application has the following advantages: (1) The defect recognition accuracy is improved: the deep combination of the multi-modal data fusion technology and the intelligent algorithm of the present application breaks through the limitations of single sensor or fixed threshold analysis, realizes the comprehensive perception and accurate judgment of the equipment appearance defect, operation parameter abnormality and other states, effectively reduces the missed detection and misjudgment, and significantly improves the accuracy and reliability of the equipment defect recognition.

[0070] (2) Reduce the risk of operation safety: the application replaces personnel on-site close contact with high-voltage equipment through a virtual scene, reduces the safety hazards caused by adverse environments, equipment failures, etc. in manual inspection, fundamentally reduces the risk of electric shock, high-altitude falling, etc. faced by operation and maintenance personnel, and greatly improves the operation safety.

[0071] (3) Optimize the operation and maintenance management mode: the application realizes intelligent evaluation of the device state through dynamic threshold early warning and trend prediction technology, combines digital twin and full life cycle data management, promotes the operation and maintenance strategy from periodic maintenance to predictive maintenance, effectively extends the device health operation cycle, reduces unnecessary maintenance frequency, and significantly improves the operation and maintenance resource utilization efficiency and management refinement level.

[0072] (4) Improve the training and collaboration efficiency: the application supports real-time collaborative inspection and fault disposal simulation through a virtual scene, breaks the geographical restrictions, and improves the cross-team collaboration efficiency; and also can highly restore the real operation scene through an immersive training system, so that new employees can quickly master the operation specification and fault handling process in a safe environment, significantly shorten the skill training cycle, and improve the training effect.

[0073] Based on the same inventive concept, the embodiment of the application provides a substation equipment detection and inspection device 40, as shown in the figure, which comprises: Figure 4 A multi-modal data acquisition unit 401 for acquiring multi-modal data of the target substation equipment and obtaining target multi-source detection data; An identification result obtaining unit 402 for identifying the target multi-source detection data by using a trained fault identification model to obtain an identification result; wherein the trained fault identification model comprises an LSTM layer, a Transformer layer, a feature fusion layer and an output layer, and the LSTM layer and the Transformer layer are parallel; the identification result comprises a device health score and a fault probability distribution; A fault determination unit 403 for matching the identification result with fault mode data in a preset fault mode library to determine whether the target substation equipment has a fault; An early warning and diagnosis unit 404 for triggering an early warning and generating a fault diagnosis report if it is determined that the target substation equipment has a fault. Optionally, the multi-modal data acquisition unit 401 is further configured to:

[0074] Acquire multi-modal data of the target substation equipment according to various sensors installed on the surface and inside of the target substation equipment to obtain initial multi-source detection data; ​The preset wavelet threshold denoising algorithm is used to perform denoising processing on the initial multi-source detection data, and target multi-source detection data is obtained.

[0075] Optionally, the recognition result obtaining unit 402 is further configured to: The time-space alignment algorithm is used to perform calibration preprocessing on the target multi-source detection data, and calibrated multi-source detection data is obtained. The trained fault recognition model is used to perform fault recognition on the calibrated multi-source detection data, and a recognition result is obtained.

[0076] Optionally, the recognition result obtaining unit 402 is further configured to: The LSTM layer in the trained fault recognition model is used to extract features from the calibrated multi-source detection data, and a local time sequence feature vector is obtained. The Transformer layer in the trained fault recognition model is used to extract features from the calibrated multi-source detection data, and a global dependency feature vector is obtained. The feature fusion layer in the trained fault recognition model is used to fuse the local time sequence features and the global dependency features, and a fusion feature vector is obtained. The output layer in the trained fault recognition model is used to process the fusion features, and a recognition result is obtained.

[0077] Optionally, the fault determining unit 403 is further configured to: According to the fault probability distribution in the recognition result, a target fault type is determined. According to the target fault type, a feature matching vector with the same fault type as the recognition result is matched from a preset fault mode library. A target similarity between the fusion feature vector and the feature matching vector is calculated. It is determined whether the target similarity exceeds a set similarity threshold. If it is determined that the target similarity exceeds the set similarity threshold, it is determined that the target substation equipment has a fault.

[0078] Optionally, the substation equipment detection and inspection device 40 further comprises a scene construction and interactive control unit 405, configured to: Unity 3D tools and geographic information systems (GIS) are used to construct a virtual substation scene. According to multi-modal data collected by the intelligent wearable device, multi-modal interactive control between the user and the virtual substation scene is completed; wherein the multi-modal data is obtained through gesture instructions or voice instructions.

[0079] Optionally, the substation equipment detection and inspection device 40 further comprises a remote collaborative operation and maintenance unit 406, configured to: Establish an end-to-end real-time communication connection based on WebRTC technology; Based on the real-time communication connection, multi-mode interactive control data, real-time data of target substation equipment, and operation screens of maintenance personnel are synchronously transmitted to a dedicated client. In the dedicated client, a labeling tool is used to mark the fault location in the virtual substation scene, and voice guidance is provided to on-site maintenance personnel to handle the fault.

[0080] The substation equipment detection and inspection device 40 can be used to perform... Figure 2 The method performed by the substation equipment detection and inspection device in the illustrated embodiment is described above. Therefore, the functions that each functional module of the substation equipment detection and inspection device 40 can achieve can be found by referring to [the relevant documentation / reference]. Figure 2 The embodiments shown are described in detail below.

[0081] In some possible implementations, various aspects of the methods provided in this application can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the methods according to the various exemplary embodiments of this application described above. For example, the computer device may perform actions such as... Figure 2 The method performed by the substation equipment detection and inspection device in the illustrated embodiment.

[0082] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Alternatively, if the integrated units of this application are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0083] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that such additions and modifications be included within the scope of the application. It is the following claims, including any amendments thereto, which define the scope of the application.

[0084] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method of substation equipment detection and patrol, characterized by, The method comprises: Multi-modal data acquisition is performed on the target substation equipment to obtain target multi-source detection data; A trained fault identification model is used to perform fault identification on the target multi-source detection data to obtain an identification result; wherein the trained fault identification model comprises an LSTM layer, a Transformer layer, a feature fusion layer and an output layer, and the LSTM layer and the Transformer layer are parallel; the identification result comprises an equipment health score and a fault probability distribution; The identification result is matched with fault mode data in a preset fault mode library to determine whether the target substation equipment has failed; If it is determined that the target substation equipment has failed, a warning is triggered and a fault diagnosis report is generated.

2. The method of claim 1, wherein, The step of performing multi-modal data acquisition on the target substation equipment to obtain target multi-source detection data comprises: Multi-modal data acquisition is performed on the target substation equipment according to various sensors installed on the surface and inside the target substation equipment to obtain initial multi-source detection data; A preset wavelet threshold denoising algorithm is used to perform denoising processing on the initial multi-source detection data to obtain the target multi-source detection data.

3. The method of claim 1, wherein, The step of using a trained fault identification model to perform fault identification on the target multi-source detection data to obtain an identification result comprises: A time-space alignment algorithm is used to perform calibration preprocessing on the target multi-source detection data to obtain calibrated multi-source detection data; The trained fault identification model is used to perform fault identification on the calibrated multi-source detection data to obtain the identification result.

4. The method of claim 3, wherein, The step of using a trained fault identification model to perform fault identification on the calibrated multi-source detection data to obtain an identification result comprises: An LSTM layer in the trained fault identification model is used to extract features from the calibrated multi-source detection data to obtain a local time sequence feature vector; A Transformer layer in the trained fault identification model is used to extract features from the calibrated multi-source detection data to obtain a global dependency feature vector; A feature fusion layer in the trained fault identification model is used to fuse the local time sequence features and the global dependency features to obtain a fusion feature vector; An output layer in the trained fault identification model is used to process the fusion features to obtain the identification result.

5. The method of claim 4, wherein, The step of matching the identification result with fault mode data in a preset fault mode library to determine whether the target substation equipment has failed comprises: A target fault type is determined according to the fault probability distribution in the identification result; A feature matching vector of the same fault type as the identification result is matched from the preset fault mode library according to the target fault type; A target similarity between the fusion feature vector and the feature matching vector is calculated; It is determined whether the target similarity exceeds a set similarity threshold; If it is determined that the target similarity exceeds the set similarity threshold, it is determined that the target substation equipment has failed.

6. The method of claim 1, wherein, After the target multi-source detection data is subjected to fault recognition by using the trained fault recognition model, and a recognition result is obtained, the method further comprises: A virtual substation scene is constructed by using a Unity 3D tool and a geographic information system (GIS). Multi-modal interaction control between a user and the virtual substation scene is completed according to multi-modal data collected by an intelligent wearable device, wherein the multi-modal data is obtained through a gesture instruction or a voice instruction.

7. The method of claim 1, wherein, After the recognition result is matched with data in a preset fault mode library to determine whether to trigger a fault warning, the method further comprises: An end-to-end real-time communication connection is established according to WebRTC technology. Multi-modal interaction control data, target substation equipment real-time data, and operation screen of an operation and maintenance personnel are synchronously transmitted to a special client according to the real-time communication connection. In the special client, a fault location of the virtual substation scene is marked by using a marking tool, and a field operation and maintenance personnel is guided to perform fault processing through voice.

8. A substation equipment detection and patrol apparatus, characterized by, The apparatus comprises: A multi-modal data acquisition unit configured to acquire multi-modal data of a target substation equipment to obtain target multi-source detection data. An identification result acquisition unit configured to perform fault recognition on the target multi-source detection data by using a trained fault recognition model to obtain an identification result, wherein the trained fault recognition model comprises an LSTM layer, a Transformer layer, a feature fusion layer, and an output layer, the LSTM layer and the Transformer layer are parallel, and the identification result comprises an equipment health score and a fault probability distribution. A fault determination unit configured to match the identification result with fault mode data in a preset fault mode library to determine whether the target substation equipment has a fault. A warning and diagnosis unit configured to trigger a warning and generate a fault diagnosis report if it is determined that the target substation equipment has a fault.

9. An electronic device, comprising: The device comprises: A memory configured to store program instructions. A processor configured to call the program instructions stored in the memory and execute the method according to any one of claims 1-7 according to the obtained program instructions.

10. A storage medium, characterized by The storage medium stores computer executable instructions, and the computer executable instructions are used to make a computer execute the method according to any one of claims 1-7.

Images