Transformer substation risk detection method and system based on multi-modal decision-making level fusion

Abstract

The invention discloses a transformer substation risk detection method and system based on multi-modal decision-level fusion, and relates to the technical field of data fusion, and the method comprises the steps: obtaining a real-time image data stream of a transformer substation dangerous area, synchronously obtaining time sequence operation parameters collected by a power equipment sensor, carrying out the target detection analysis of the image data stream, and obtaining a real-time image data stream; and identifying personnel targets in the dangerous area and generating a first early warning signal, performing abnormal mode analysis on the time sequence operation parameters, identifying equipment operation state abnormity and generating a second early warning signal, performing decision-level fusion on the first early warning signal and the second early warning signal, and outputting a comprehensive risk assessment result through a weighted voting mechanism. According to the invention, by combining the image data and the sensor data, the perception capability of potential risks in the transformer substation is enhanced, the monitoring accuracy and the early warning reliability are improved, and the safety and stability of the whole power grid are further improved.

Description

Technical Field

[0001] This invention relates to the field of data fusion technology, specifically to a substation risk detection method and system based on multimodal decision-level fusion. Background Technology

[0002] In the power industry, substations, as the core hubs of the power grid, play a crucial role. The high-voltage environment makes substations a concentration point of potential risks; any violation of regulations, unauthorized entry into dangerous areas, or improper behavior can lead to serious safety accidents. These accidents not only threaten personnel safety but may also damage power facilities or even cause a complete power grid outage. Therefore, ensuring the stability and safety of substations is essential for the stable operation of the power grid. With the rapid development and intelligentization of power systems, traditional substation management models have gradually revealed problems such as low efficiency, insufficient safety, and unintuitive information display. To effectively ensure the safety of personnel and equipment within substations, there is an urgent need for a monitoring system capable of timely and accurate identification of potential risks in complex environments.

[0003] Smart substations have thus become an important component of modern power systems. By integrating multiple data modalities such as images, sensors, and text, they achieve comprehensive monitoring of equipment operating status, environmental factors, and personnel safety. The technical characteristics of smart substations include intelligence, networking, and integration. They utilize advanced sensor, electronic, information, communication, control, and artificial intelligence technologies to achieve intelligent monitoring, control, and dispatching of substations. The application of these technologies enables real-time monitoring and remote control, improving the operational efficiency and safety of substations. The monitoring system of a smart substation can monitor various parameters of the substation in real time, including voltage, current, and temperature, and predict potential risks through data analysis, taking timely measures to prevent accidents. This comprehensive monitoring and intelligent analysis capability makes smart substations play a crucial role in improving the stability and safety of power systems, providing strong technical support for the modernization and intelligentization of power systems. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention provides a substation risk detection method and system based on multimodal decision-level fusion to solve the problems of low efficiency, insufficient safety and unintuitive information display in the prior art.

[0005] To address the aforementioned technical challenges, a substation risk detection method based on multimodal decision-level fusion is proposed, including:

[0006] The system acquires real-time image data streams of hazardous areas in substations and synchronously acquires time-series operating parameters collected by power equipment sensors; it performs target detection analysis on the image data streams to identify personnel targets within the hazardous areas and generate a first early warning signal; it performs abnormal mode analysis on the time-series operating parameters to identify abnormal equipment operating states and generate a second early warning signal; it then fuses the first and second early warning signals at the decision level and outputs a comprehensive risk assessment result through a weighted voting mechanism.

[0007] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the generation of the first early warning signal includes: processing image data through a target detection network, performing a spatial dimension global pooling operation after feature extraction to generate horizontal and vertical feature vectors, concatenating the bidirectional feature vectors and performing a convolution transformation to generate an intermediate feature vector, decomposing the intermediate feature vector into horizontal attention weights and vertical attention weights, weightedly fusing it with the original feature map to obtain an output feature map, and performing target detection on the dangerous area of ​​the substation to output an early warning signal;

[0008] The object detection network consists of a feature extraction layer, a multi-scale feature fusion layer, and a detection output layer.

[0009] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the spatial dimension global pooling operation includes performing horizontal global average pooling on the input feature map to generate a horizontal feature vector, and performing vertical global average pooling on the input feature map to generate a vertical feature vector.

[0010] The dimension of the bidirectional feature vector is related to the size of the feature map space; among them, the horizontal attention weights and vertical attention weights act on the output of the multi-scale feature fusion layer of the object detection network.

[0011] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the generation of the second early warning signal includes: compressing the feature dimensions of time-series monitoring data through an encoder, controlling the transmission of historical information using a gating mechanism, storing long-term time-dependent features through a memory unit, reconstructing time-series data and calculating the reconstruction error value through a decoder, and outputting an early warning signal when the error exceeds an abnormal threshold.

[0012] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the training process of the encoder and decoder includes: performing missing value filling and normalization preprocessing on the sensor data; using the mean square error between the reconstructed data and the original data as the loss function; and optimizing the network parameters through backpropagation until the error converges.

[0013] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the method of controlling the transmission of historical information using a gating mechanism includes controlling the retention ratio of new feature information through an input gate, controlling the discard ratio of historical features through a forget gate, and controlling the transmission ratio of features to the next time step through an output gate.

[0014] The storage of long-term time-dependent features through the memory unit includes storing feature information across time steps through the cell state, and the updating of the cell state is jointly regulated by the outputs of the input gate and the forget gate.

[0015] As a preferred embodiment of the substation risk detection method based on multimodal decision-level fusion described in this invention, the output comprehensive risk assessment result includes: dynamically allocating a first weight and a second weight according to the historical detection accuracy of the image analysis model and the sensor analysis model, calculating the comprehensive early warning signal value according to the weighted sum formula, and periodically updating the weight value with the model verification results;

[0016] The output comprehensive risk assessment results also include calculating precision and recall using a confusion matrix for the target detection network, plotting precision-recall curves and receiver operation characteristic curves for the time series analysis model, and enabling the detection model when the average precision index and the area under the curve reach the deployment threshold.

[0017] The comprehensive risk assessment formula is expressed as follows:

[0018]

[0019] Wherein, TP is the number of true positives (samples correctly classified as positive), FP is the number of false positives (samples incorrectly classified as positive), TN is the number of true negatives (samples correctly classified as negative), FN is the number of false negatives (samples incorrectly classified as negative), Recall is the recall rate, Precision is the precision rate, and Accuracy is the accuracy rate.

[0020] As a preferred embodiment of the substation risk detection system based on multimodal decision-level fusion described in this invention, it is characterized by including an image analysis module, a sensor analysis module, a decision fusion module, and a verification control module.

[0021] The image analysis module is used to perform real-time target detection on image data of dangerous areas in substations and identify personnel's unauthorized entry into risk areas.

[0022] The sensor analysis module is used to detect anomalies in the time-series data of power equipment sensors and identify equipment operation risks.

[0023] The decision fusion module is used to integrate the early warning signals from the image and sensor modules to generate a comprehensive risk assessment result.

[0024] The verification control module is used to monitor the module's performance indicators.

[0025] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for substation risk detection based on multimodal decision-level fusion.

[0026] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for substation risk detection based on multimodal decision-level fusion.

[0027] The beneficial effects of this invention are as follows: By integrating information from different data sources, this invention enhances our ability to perceive potential risks within substations, improves the accuracy of monitoring and the reliability of early warning, thereby enhancing the safety and stability of the entire power grid; by adopting a voting mechanism, it can make independent decisions among different data sources, and improve the accuracy and reliability of risk identification through fusion decision-making; it is particularly suitable for processing multimodal information with large differences in data frequency and structure, and can exhibit good robustness and flexible adaptability in the complex environment of smart substations. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 The above is a flowchart of a substation risk detection method based on multimodal decision-level fusion, provided as an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of the decision-level fusion process of a substation risk detection method based on multimodal decision-level fusion, provided as an embodiment of the present invention.

[0031] Figure 3 The flowchart shows a system scheme for a substation risk detection system based on multimodal decision-level fusion, as provided in one embodiment of the present invention. Detailed Implementation

[0032] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0033] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0034] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive, either alone or selectively, with other embodiments.

[0035] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0036] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0037] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0038] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a substation risk detection method based on multimodal decision-level fusion, including:

[0039] S1: Acquire real-time image data streams of hazardous areas in substations and synchronously acquire time-series operating parameters collected by power equipment sensors.

[0040] The collected timing parameters include voltage, current, and temperature data.

[0041] S2: Perform target detection analysis on the image data stream, identify personnel targets in the danger zone, and generate the first warning signal.

[0042] Furthermore, the target detection and analysis includes using deep learning technology to analyze the image data collected by the camera to achieve target detection in dangerous areas within the substation. The core objective is to quickly and accurately identify and locate personnel targets within the substation, thereby completing the risk detection task of the substation.

[0043] The identification of personnel targets within the danger zone includes using the YOLOv8 algorithm to identify personnel targets within the danger zone; the YOLOv8 network structure includes four parts: input layer, backbone network, neck network, and head network.

[0044] It should be noted that the input layer is used to receive the input image data; the backbone network is used to process features through the CBS and C2F modules, extracting deep features of the image, while gradually reducing the size of the feature map and increasing the number of channels; the neck network includes the aggregation of multi-scale features through the SPPF module and the fusion of feature maps of different scales using upsampling operations to adapt to detection targets of various sizes; the head network consists of multi-layer convolutions and the CBS module, extracting high-level semantic information and outputting bounding box loss and classification loss to complete target detection.

[0045] Considering the reduced detection accuracy of targets and occluded targets due to the varying distribution of cameras in substations, further optimizing the network's ability to capture specific targets in complex scenes remains crucial. To better adapt to the needs of complex scenes and multi-target detection, a CA attention mechanism is introduced into the YOLOv8 network to enhance its ability to capture key regional features and improve the overall performance of target detection. By combining coordinate information and channel attention, the CA mechanism not only improves the network's focus on target areas but also effectively preserves the spatial location information of targets, thereby further enhancing the model's performance in real-time target detection in hazardous areas of substations.

[0046] Furthermore, the generation of the first warning signal includes processing image data through a target detection network, performing a spatial dimension global pooling operation after the feature extraction layer to generate horizontal and vertical feature vectors, concatenating the horizontal and vertical feature vectors and performing a convolution transformation to generate an intermediate feature vector, decomposing the intermediate feature vector into horizontal attention weights and vertical attention weights, and then weighted and fused the attention weights with the original feature map to output a fused feature map, thereby detecting the target in the dangerous area and outputting a warning signal.

[0047] In this embodiment of the application, generating the first warning signal includes generating feature vectors by global average pooling in the horizontal and vertical directions, concatenating the vectors and then convolving them to generate intermediate features, decomposing them into horizontal and vertical attention weights, and then weighted and fused with the original feature map to generate a fused feature map.

[0048] In one optional implementation, generating the first warning signal includes performing global average pooling on the feature map, generating channel weights through MLP, performing max pooling along the channel dimension, performing average pooling along the channel dimension, and convolving the concatenated data to generate spatial weights.

[0049] In another optional implementation, the generation of the first warning signal includes global average pooling to compress features, learning channel relationships in a fully connected layer, and channel-weighted output.

[0050] The network comprises a feature extraction layer, a multi-scale feature fusion layer, and a detection output layer. The spatial dimension global pooling operation includes performing horizontal global average pooling on the input feature map to generate a horizontal feature vector, and performing vertical global average pooling on the input feature map to generate a vertical feature vector. The horizontal and vertical attention weights are applied to the output of the multi-scale feature fusion layer of the object detection network to enhance the feature response intensity of small-sized targets and occluded targets, as expressed by the formula:

[0051]

[0052] Where C is the number of channels, H is the height, W is the width, and x is the width. c (h,i) represents the value of the c-th channel in the h-th row and i-th column of the feature map. This represents the global pooling result for the c-th channel in the horizontal direction. This represents the global pooling result for the c-th channel in the vertical direction, where h, i, c, w, and j are variable indices, and [z′] is the value of the global pooling result for the c-th channel in the vertical direction. h ,z′ w To concatenate feature vectors in the horizontal and vertical directions, F1 is the convolution operation, δ is the activation function, and f is the new feature vector after convolution and activation. h For the horizontal part of the new feature vector f, f wF represents the vertical portion of the new feature vector f. h To perform convolution operations in the horizontal direction, F w To perform convolution in the vertical direction, σ is the Sigmoid function, and g h For the generated horizontal attention vector, g w This is the attention vector generated in the vertical direction.

[0053] The output feature map is represented as follows:

[0054]

[0055] Among them, y c (a, b) represents the value of the final output feature map in the a-th row and b-th column, x c (a, b) represents the value of the original feature map in row a and column b. For horizontal attention weights, The vertical attention weights are a and b, which are variable indices. Target detection is performed in the dangerous areas of the substation, and an early warning signal is issued when an abnormal target is detected.

[0056] S3: Perform abnormal mode analysis on timing operation parameters, identify abnormal equipment operating status, and generate a second early warning signal.

[0057] Furthermore, generating the second early warning signal includes compressing the feature dimensions of the time-series monitoring data through an encoder, controlling the transmission of historical information using a gating mechanism, storing long-term time-dependent features through a memory unit, reconstructing the time-series data through a decoder and calculating the reconstruction error value, and outputting the second early warning signal when the reconstruction error exceeds the abnormal threshold.

[0058] The mathematical expressions for the encoding and decoding processes are as follows:

[0059] A = Gθ1(X) = δ(W1X + d1)

[0060] X R =Gθ2(X)=δ(W2X+d2)

[0061] Where A is the output of the hidden layer, Gθ1 is the encoding function of the autoencoder, X is the input data, δ is the activation function, W1 is the weight matrix of the encoding layer, d1 is the bias vector of the encoding layer, and X... R For the reconstructed output data, Gθ2 is the decoding function of the autoencoder, W2 is the weight matrix of the decoding layer, and d2 is the bias vector of the decoding layer.

[0062] The training process for the encoder and decoder includes preprocessing the sensor data by filling in missing values ​​and normalizing it, using the mean square error between the reconstructed data and the original data as the loss function, and optimizing the network parameters through the backpropagation algorithm until the error converges.

[0063] To effectively identify and locate potential risks, mean squared error is chosen as the loss function for the autoencoder model, expressed as:

[0064]

[0065] Where MSE is the mean squared error, N is the number of samples, and l t For real labels, Let t be the predicted label and t be the variable index.

[0066] Furthermore, the gating mechanism includes controlling the retention ratio of new feature information through the input gate, controlling the discard ratio of historical features through the forget gate, and controlling the transfer ratio of features to the next time step through the output gate; the memory unit stores feature information across time steps through the cell state, and the update of the cell state is jointly regulated by the outputs of the input gate and the forget gate.

[0067] In this embodiment, the temporal anomaly detection includes: input gate and forget gate controlling the information flow; cell state storage of cross-time step features; decoder reconstructing temporal data; and calculating anomaly threshold based on MSE.

[0068] In one alternative implementation, the timing anomaly detection includes performing position encoding, the encoder processing a multi-head self-attention and feedforward network, and performing decoder reconstruction to determine whether an anomaly exists.

[0069] In another alternative implementation, temporal anomaly detection includes performing causal convolution to ensure temporal constraints, dilated convolution to expand the receptive field, residual connections to avoid gradient vanishing, and reconstruction error calculation.

[0070] S4: The first and second early warning signals are fused at the decision-level, and the comprehensive risk assessment results are output through a weighted voting mechanism.

[0071] like Figure 2 The decision-level fusion includes dynamically allocating a first weight based on the historical detection accuracy of the image analysis model, dynamically allocating a second weight based on the historical detection accuracy of the sensor analysis model, and calculating a comprehensive early warning signal value according to a weighted sum formula, i.e., comprehensive early warning signal value = first early warning signal × first weight + second early warning signal × second weight, with the weight value being updated periodically with the model validation results.

[0072] The output comprehensive risk assessment results include calculating precision and recall using a confusion matrix for the target detection network, plotting precision-recall curves and receiver operation characteristic curves for the time series analysis model, and activating the detection model when the average precision index and the area under the curve reach the deployment threshold.

[0073] It should be noted that the comprehensive risk assessment formula is expressed as follows:

[0074]

[0075] Wherein, TP is the number of true positives (samples correctly classified as positive), FP is the number of false positives (samples incorrectly classified as positive), TN is the number of true negatives (samples correctly classified as negative), FN is the number of false negatives (samples incorrectly classified as negative), Recall is the recall rate, Precision is the precision rate, and Accuracy is the accuracy rate.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0077] Example 2, refer to Figure 3 This is the second embodiment of the present invention, which provides a substation risk detection system based on multimodal decision-level fusion, including an image analysis module, a sensor analysis module, a decision fusion module, and a verification control module.

[0078] The image analysis module is used to perform real-time target detection on image data of dangerous areas in substations and identify personnel's unauthorized entry into risk areas.

[0079] The sensor analysis module is used to detect anomalies in the time-series data of power equipment sensors and identify equipment operation risks.

[0080] The decision fusion module is used to integrate the early warning signals from the image and sensor modules to generate a comprehensive risk assessment result.

[0081] The verification control module is used to monitor the module's performance indicators.

[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0083] Example 3, the third embodiment of the present invention, differs from the previous two embodiments in that:

[0084] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0086] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0087] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

Claims

1. A substation risk detection method based on multimodal decision-level fusion, characterized in that: include, Acquire real-time image data streams of hazardous areas in substations and synchronously acquire time-series operating parameters collected by power equipment sensors; Perform target detection analysis on the image data stream to identify personnel targets in dangerous areas and generate a first warning signal; Perform abnormal mode analysis on timing operation parameters to identify abnormal equipment operating status and generate a second early warning signal; The first and second early warning signals are merged at the decision-making level, and a comprehensive risk assessment result is output through a weighted voting mechanism.

2. The substation risk detection method based on multimodal decision-level fusion as described in claim 1, characterized in that: The generation of the first early warning signal includes processing image data through a target detection network, performing a spatial dimension global pooling operation after feature extraction to generate horizontal and vertical feature vectors, concatenating the bidirectional feature vectors and performing a convolution transformation to generate an intermediate feature vector, decomposing the intermediate feature vector into horizontal attention weights and vertical attention weights, weightedly fusing it with the original feature map to obtain an output feature map, and performing target detection on the dangerous area of ​​the substation to output an early warning signal. The object detection network consists of a feature extraction layer, a multi-scale feature fusion layer, and a detection output layer.

3. The substation risk detection method based on multimodal decision-level fusion as described in claim 2, characterized in that: The spatial dimension global pooling operation includes performing horizontal global average pooling on the input feature map to generate a horizontal feature vector, and performing vertical global average pooling on the input feature map to generate a vertical feature vector. The dimension of the bidirectional feature vector is related to the size of the feature map space; among them, the horizontal attention weights and vertical attention weights act on the output of the multi-scale feature fusion layer of the object detection network.

4. The substation risk detection method based on multimodal decision-level fusion as described in claim 3, characterized in that: The generation of the second early warning signal includes compressing the feature dimensions of the time-series monitoring data through an encoder, controlling the transmission of historical information using a gating mechanism, storing long-term time-dependent features through a memory unit, reconstructing the time-series data and calculating the reconstruction error value through a decoder, and outputting an early warning signal when the error exceeds the abnormal threshold.

5. The substation risk detection method based on multimodal decision-level fusion as described in claim 4, characterized in that: The training process for the encoder and decoder includes preprocessing the sensor data by filling in missing values ​​and normalizing it, using the mean square error between the reconstructed data and the original data as the loss function, and optimizing the network parameters through backpropagation until the error converges.

6. The substation risk detection method based on multimodal decision-level fusion as described in claim 5, characterized in that: The method of using a gating mechanism to control the transmission of historical information includes controlling the retention ratio of new feature information through an input gate, controlling the discard ratio of historical features through a forget gate, and controlling the transmission ratio of features to the next time step through an output gate. The storage of long-term time-dependent features through the memory unit includes storing feature information across time steps through the cell state, and the updating of the cell state is jointly regulated by the outputs of the input gate and the forget gate.

7. The substation risk detection method based on multimodal decision-level fusion as described in claim 6, characterized in that: The output comprehensive risk assessment result includes dynamically allocating the first weight and the second weight based on the historical detection accuracy of the image analysis model and the sensor analysis model, calculating the comprehensive early warning signal value according to the weighted sum formula, and updating the weight value periodically with the model verification results; The output comprehensive risk assessment results also include calculating precision and recall using a confusion matrix for the target detection network, plotting precision-recall curves and receiver operation characteristic curves for the time series analysis model, and enabling the detection model when the average precision index and the area under the curve reach the deployment threshold. The comprehensive risk assessment formula is expressed as follows: Wherein, TP is the number of true positives (samples correctly classified as positive), FP is the number of false positives (samples incorrectly classified as positive), TN is the number of true negatives (samples correctly classified as negative), FN is the number of false negatives (samples incorrectly classified as negative), Recall is the recall rate, Precision is the precision rate, and Accuracy is the accuracy rate.

8. A system employing the substation risk detection method based on multimodal decision-level fusion as described in any one of claims 1 to 7, characterized in that: It includes an image analysis module, a sensor analysis module, a decision fusion module, and a verification control module; The image analysis module is used to perform real-time target detection on image data of dangerous areas in substations and identify personnel's unauthorized entry into risk areas; The sensor analysis module is used to detect anomalies in the time-series data of power equipment sensors and identify equipment operation risks. The decision fusion module is used to integrate the early warning signals from the image and sensor modules to generate a comprehensive risk assessment result. The verification control module is used to monitor the module's performance indicators.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the substation risk detection method based on multimodal decision-level fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the substation risk detection method based on multimodal decision-level fusion as described in any one of claims 1 to 7.

Images