Unmanned aerial vehicle transformer substation anomaly detection method and system based on multi-modal fusion algorithm

Abstract

The invention relates to the technical field of substation inspection, and discloses an unmanned aerial vehicle substation anomaly detection method and system based on a multi-modal fusion algorithm. The method comprises the steps of collecting transformer substation data through a multi-modal sensor and inputting the transformer substation data into a multi-modal fusion algorithm; the convolutional neural network based on a multi-modal fusion algorithm performs feature extraction on the transformer substation data to output a plurality of data features; fusing the data features based on a multi-head attention mechanism of a multi-modal fusion algorithm to obtain fused features; and feature classification is carried out based on Gaussian kernel function local outlier factors of a multi-modal fusion algorithm according to fusion features so as to determine abnormal points. According to the method, the Gaussian kernel function local outlier factor, the convolutional neural network and the multi-head attention mechanism form a multi-modal fusion algorithm, substation anomaly detection is realized through the multi-modal fusion algorithm, the substation inspection intelligence level and the anomaly detection capability are improved, full-autonomous, efficient and safe intelligent inspection of the substation is realized, and the substation inspection efficiency is improved. And stable operation of the power grid is supported.

Description

Technical Field

[0001] The present invention relates to the technical field of substation inspection, and in particular to a method and system for detecting abnormalities in a substation using an unmanned aerial vehicle based on a multimodal fusion algorithm. Background Art

[0002] With the rapid expansion of the scale of power systems, traditional manual inspection methods face challenges. With the growth of the scale of power grids, the problems of low efficiency and high safety risks of manual inspections have become increasingly prominent. Drone technology provides new solutions with its flexibility and efficiency. It can cover areas that are difficult for humans to reach, reduce inspection blind spots, and improve inspection efficiency and safety. Drone technology integrates high-definition cameras, infrared detection, voiceprint monitoring and other means to achieve all-round monitoring of substation equipment, timely discover potential hidden dangers, reduce operation and maintenance costs, and improve power supply reliability. The remote operation capability of the drone intelligent inspection system reduces the risk of on-site operations of personnel, which is in line with the development trend of intelligence and automation in the power industry. However, there are still defects in drone route planning, autonomous inspection, and intelligent fault diagnosis. Summary of the invention

[0003] The main purpose of the present invention is to provide a method and system for detecting anomalies in a substation using a drone based on a multimodal fusion algorithm, aiming to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above object, the present invention provides a method for detecting abnormality of a substation using a drone based on a multimodal fusion algorithm, comprising:

[0005] Collecting substation data through a multimodal sensor and inputting the substation data into a multimodal fusion algorithm;

[0006] A convolutional neural network based on the multimodal fusion algorithm performs feature extraction on the substation data and outputs a plurality of data features;

[0007] The data features are fused based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features;

[0008] The Gaussian kernel function local outlier factor based on the multimodal fusion algorithm performs feature classification according to the fusion features to determine the abnormal points.

[0009] In some embodiments, the convolutional neural network based on the multimodal fusion algorithm extracts features from the substation data and outputs multiple data features, including:

[0010] Inputting the substation data into the convolutional neural network of the multimodal fusion algorithm to automatically extract features of the substation data through the convolutional layer of the convolutional neural network;

[0011] Applying a non-linear activation function after the convolutional layer;

[0012] Reducing the dimension of features through the pooling layer of the convolutional neural network;

[0013] The features are flattened by repeating multiple layers of the convolutional layer and the pooling layer, and input into the fully connected layer;

[0014] The features are integrated through the fully connected layer and the classification results are output to obtain multiple data features.

[0015] In some embodiments, the convolutional neural network includes a convolutional layer, an activation function, a pooling layer, and a fully connected layer.

[0016] In some embodiments, the multi-head attention mechanism based on the multimodal fusion algorithm fuses the data features to obtain fused features, including:

[0017] Generate a feature matrix according to the multiple data features extracted by the convolutional neural network;

[0018] Calculate a query matrix, a key matrix and a value matrix according to the feature matrix;

[0019] Calculate similarity scores between the query matrix, the key matrix, and the value matrix based on a multi-head attention mechanism of the multimodal fusion algorithm;

[0020] The multi-head attention mechanism feature fusion vector is solved according to the similarity score to obtain the fusion feature. In some embodiments, the query matrix Q, key matrix K and value matrix V are respectively:

[0021] Q=W q X, K = W k X, V = W v X

[0022] Wherein, X is the feature matrix extracted by the convolutional neural network; W q , W k , W v are the weight matrices of Q, K, and V respectively.

[0023] In some embodiments, the Gaussian kernel function local outlier factor based on the multimodal fusion algorithm performs feature classification according to the fusion feature to determine the abnormal point, including:

[0024] The local outlier factor algorithm is improved by using Gaussian kernel density function to obtain the Gaussian kernel function local outlier factor;

[0025] Based on the local outlier factor of the Gaussian kernel function, feature classification is performed according to the fusion feature to determine the abnormal point.

[0026] In some embodiments, the performing feature classification based on the fusion feature based on the local outlier factor of the Gaussian kernel function to determine the abnormal point includes:

[0027] Determine the Gaussian kernel local density of the data point p of the fusion feature based on the Gaussian kernel function local outlier factor;

[0028] Calculate the Gaussian kernel distance between data point p and data point o;

[0029] The Gaussian kernel local outlier factor of data point p is calculated based on the Gaussian kernel local density of data point p and the Gaussian kernel local density of data point o;

[0030] Analyze the Gaussian kernel local density, Gaussian kernel distance and Gaussian kernel local outlier factor to obtain a rule for distinguishing normal points and outliers;

[0031] The abnormal points are determined based on the normal point and outlier point distinction rule.

[0032] In some embodiments, the multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device, and a voiceprint.

[0033] In addition, to achieve the above purpose, the present invention also proposes a UAV substation anomaly detection system based on a multimodal fusion algorithm, comprising:

[0034] A data acquisition module, used to collect substation data through a multimodal sensor and input the substation data into a multimodal fusion algorithm;

[0035] A feature extraction module, used for extracting features from the substation data based on the convolutional neural network of the multimodal fusion algorithm, and outputting a plurality of data features;

[0036] A feature fusion module, used to fuse the data features based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features;

[0037] The feature classification module is used to perform feature classification according to the fusion features based on the Gaussian kernel function local outlier factor of the multimodal fusion algorithm to determine the abnormal points.

[0038] In some embodiments, the multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device, and a voiceprint.

[0039] The present invention provides a method for detecting abnormalities of substations using drones based on a multimodal fusion algorithm, comprising: collecting substation data through a multimodal sensor, and inputting the substation data into a multimodal fusion algorithm; extracting features from the substation data using a convolutional neural network based on the multimodal fusion algorithm, and outputting multiple data features; fusing the data features based on a multi-head attention mechanism based on the multimodal fusion algorithm to obtain fusion features; and performing feature classification based on the fusion features using a Gaussian kernel function local outlier factor based on the multimodal fusion algorithm to determine abnormal points. In the present invention, the feature classification method based on the Gaussian kernel function local outlier factor, the convolutional neural network, and the multi-head attention mechanism form a multimodal fusion algorithm, and drone substation abnormality detection is realized through the multimodal fusion algorithm, thereby improving the intelligence level and abnormality detection capability of the substation inspection system, and realizing fully autonomous, efficient, and safe intelligent inspections of substations, thereby providing strong support for the stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;

[0041] Figure 2 It is a flow chart of an embodiment of a method for detecting abnormality of a substation using a drone based on a multimodal fusion algorithm according to the present invention;

[0042] Figure 3 This is a schematic diagram of a convolutional neural network (CNN) feature extraction method according to an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of the effect of local data distribution disturbance on the LOF and GKLOF values ​​of a data point p involved in an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a low-altitude route inspection of a UAV main transformer involved in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of a multimodal fusion algorithm framework involved in an embodiment of the present invention;

[0046] Figure 7 This is a structural block diagram of an embodiment of a UAV substation anomaly detection system based on a multimodal fusion algorithm of the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0050] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] To achieve the above-mentioned objectives, the present invention proposes an electronic device, comprising: a memory, a processor, and a UAV substation anomaly detection program based on a multimodal fusion algorithm stored in the memory and executable on the processor, wherein the UAV substation anomaly detection program based on a multimodal fusion algorithm is configured to implement the UAV substation anomaly detection method based on a multimodal fusion algorithm as described above.

[0052] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of an electronic device of a hardware operating environment involved in an embodiment of the present invention.

[0053] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM memory) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0055] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a drone substation anomaly detection program based on a multimodal fusion algorithm.

[0056] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls the drone substation anomaly detection program based on the multimodal fusion algorithm stored in the memory 1005 through the processor 1001, and executes the drone substation anomaly detection method based on the multimodal fusion algorithm provided in an embodiment of the present invention.

[0057] The present invention proposes a method and system for detecting abnormality of a substation using an unmanned aerial vehicle based on a multimodal fusion algorithm.

[0058] The embodiment of the present invention provides a method for detecting abnormality of a substation by using a drone based on a multimodal fusion algorithm. Figure 2 , Figure 2 The present invention is a flowchart of an embodiment of a method for detecting abnormality of a substation using a drone based on a multimodal fusion algorithm.

[0059] like Figure 2 As shown, the UAV substation anomaly detection method based on multimodal fusion algorithm includes:

[0060] Step S100: collecting substation data through a multimodal sensor, and inputting the substation data into a multimodal fusion algorithm;

[0061] Step S200: extracting features from the substation data based on the convolutional neural network of the multimodal fusion algorithm, and outputting a plurality of data features;

[0062] Step S300: fusing the data features based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features;

[0063] Step S400: performing feature classification based on the fusion features according to the Gaussian kernel function local outlier factor of the multimodal fusion algorithm to determine abnormal points.

[0064] It should be noted that the execution subject in this embodiment may be an electronic device, which may be a computer device with data processing functions, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

[0065] It is understandable that this embodiment is explained by taking drone inspection of substation as an example. The method described in this embodiment includes feature extraction, feature fusion and expression, and feature classification, data extraction based on convolutional neural networks (CNN); feature vector fusion based on multi-head attention mechanism; feature classification based on Gaussian kernel densitybased local outlier factor (GKLOF). In this embodiment, the substation data collected by the sensor is analyzed under the framework of the multimodal fusion algorithm to accurately and efficiently determine the abnormal points, improve the ability of drone inspection, and promote the construction of smart grids and smart substations.

[0066] In one embodiment, a convolutional neural network based on the multimodal fusion algorithm performs feature extraction on the substation data and outputs multiple data features, including: inputting the substation data into the convolutional neural network of the multimodal fusion algorithm to automatically extract features of the substation data through a convolutional layer of the convolutional neural network; applying a nonlinear activation function after the convolutional layer; reducing the dimension of the features through a pooling layer of the convolutional neural network; flattening the features by repeating multiple layers of the convolutional layer and the pooling layer, and inputting them into a fully connected layer; integrating features through the fully connected layer and outputting classification results to obtain multiple data features.

[0067] In one embodiment, the convolutional neural network includes a convolutional layer, an activation function, a pooling layer, and a fully connected layer.

[0068] For example, data feature extraction based on convolutional neural network CNN: automatically extract the features of input data through convolutional layers; reduce the dimension of feature maps through pooling layers to reduce the amount of calculation and enhance model robustness; apply activation functions after convolutional layers to introduce nonlinearity; multi-layer convolution and pooling extract higher-level features; finally, integrate features through fully connected layers and output prediction results.

[0069] Specifically, the convolutional neural network CNN is mainly composed of convolutional layers and pooling layers, in which the convolutional layer contains filters. The filters slide on the input data to perform convolution operations, thereby extracting data features. The feature extraction principle of the convolutional neural network CNN is as follows Figure 3As shown, the following steps are included: Convolution operation: The convolution operation is the core of CNN. It extracts features from the input signal through a sliding window (convolution kernel). In one-dimensional signal processing, the convolution kernel moves in one dimension, usually the time dimension, to capture the local patterns and features of the signal; Feature extraction: CNN processes the images (substation data) collected by the drone through the convolution layer and automatically extracts local features such as edges and textures. This process is achieved through convolution operations, in which each convolution kernel is responsible for extracting specific features in the image; Data dimensionality reduction: The pooling layer is used in CNN to reduce the dimension of the feature map and the number of parameters, thereby reducing the computational complexity and improving the robustness of the model. For example, pooling operations include maximum pooling and average pooling; Activation function: An activation function is usually added after the convolution layer, such as ReLU (Rectified LinearUnit) to introduce nonlinearity and enhance the expressiveness of the model; multi-layer convolution and pooling: higher-level features can be further extracted by repeated convolution and pooling operations; fully connected layer: in the last few layers of CNN, a fully connected layer is used to integrate the previously extracted features and output the final prediction results.

[0070] It should be noted that in this embodiment, a suitable convolutional neural network CNN architecture can be selected according to the task requirements, including but not limited to LeNet, AlexNet, VGG, ResNet, etc. Use convolutional layers to extract local features in the image through convolution operations and activation functions (such as ReLU). Use pooling layers (such as maximum pooling) to reduce the spatial dimension of the features, reduce the amount of calculation, and maintain important features. At the end of the convolutional neural network, a fully connected layer is used to map the extracted features to the final output.

[0071] In one example, an image is obtained based on the substation data collected by the drone and input into the convolutional neural network CNN; the image passes through the first convolution layer; the ReLU activation function is applied; the pooling layer (such as maximum pooling) is applied; the convolution layer, activation function and pooling layer steps are repeated, and multiple convolution-activation-pooling layers are stacked; after a series of convolution layers, the features are flattened and input into the fully connected layer; after the fully connected layer, the activation function (such as softmax) is applied to produce the final classification result. In this embodiment, in this way, the convolutional neural network CNN can gradually extract complex features from the input substation data and use them for subsequent anomaly detection tasks.

[0072] In one embodiment, the data features are fused based on a multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features, including: generating a feature matrix based on multiple data features extracted by the convolutional neural network; calculating a query matrix, a key matrix, and a value matrix based on the feature matrix; calculating similarity scores between the query matrix, the key matrix, and the value matrix based on the multi-head attention mechanism of the multimodal fusion algorithm; solving a multi-head attention mechanism feature fusion vector based on the similarity score to obtain a fused feature.

[0073] In one embodiment, the query matrix Q, key matrix K and value matrix V are respectively:

[0074] Q=W q X, K = W k X, V = W v X

[0075] Wherein, X is the feature matrix extracted by the convolutional neural network; W q , W k , W v are the weight matrices of Q, K, and V respectively.

[0076] Exemplarily, feature vector fusion based on multi-head attention mechanism: the input data is linearly transformed to generate query (Q), key (K) and value (V) matrices, and then the attention weights of multiple heads are calculated in parallel, each head captures different features or aspects of the input data, and finally the outputs of all heads are concatenated and integrated through a linear layer to fuse multi-head information and form a comprehensive feature representation, thereby enhancing the model's ability to understand complex data relationships.

[0077] Specifically, the multi-head attention mechanism is composed of multiple groups of self-attention units, which fuse different feature vectors extracted by the convolutional neural network CNN according to their own characteristics to form new features (fused features) that are more expressive than the original features. The feature vector fusion based on the multi-head attention mechanism mainly includes the following steps: First, calculate the query / key / value matrix (Q / K / V) and its formula is as follows:

[0078] Q=W q X, K = W k X, V = W v X

[0079] Among them, X is the feature matrix extracted by CNN; W q , W k , W v They are Q, K, and V weight matrices respectively.

[0080] Secondly, calculate the similarity score between Q and K. The specific formula is as follows:

[0081]

[0082] Where M is the self-attention network; σ is the Softmax activation function; is the scaling factor.

[0083] Finally, solve the feature fusion vector of the multi-head attention mechanism. The specific formula is as follows:

[0084]

[0085] Among them, H i is the i-th head of multi-head attention; W i Q , W i K , W i V is the weight of the i-th attention head; M c is the matrix concatenation function; is the feature fusion matrix.

[0086] In one embodiment, the Gaussian kernel function local outlier factor based on the multimodal fusion algorithm performs feature classification according to the fusion feature to determine the abnormal point, including: improving the local outlier factor algorithm using the Gaussian kernel density function to obtain the Gaussian kernel function local outlier factor; based on the Gaussian kernel function local outlier factor, performing feature classification according to the fusion feature to determine the abnormal point.

[0087] In one embodiment, feature classification is performed according to the fused feature based on the local outlier factor of the Gaussian kernel function to determine the abnormal point, including: determining the Gaussian kernel local density of the data point p of the fused feature based on the local outlier factor of the Gaussian kernel function; calculating the Gaussian kernel distance between the data point p and the data point o; calculating the Gaussian kernel local outlier factor of the data point p based on the Gaussian kernel local density of the data point p and the Gaussian kernel local density of the data point o; analyzing the Gaussian kernel local density, the Gaussian kernel distance and the Gaussian kernel local outlier factor to obtain a rule for distinguishing normal points and outliers; judging the abnormal point based on the rule for distinguishing normal points and outliers.

[0088] Exemplarily, feature classification based on Gaussian kernel density based local outlier factor (GKLOF): the local density of each data point is estimated using the Gaussian kernel function, the local outlier factor score of each point relative to its neighbors is calculated, the degree of outlierness of the data point is evaluated, and the data is classified or outliers are identified accordingly.

[0089] Specifically, the feature classification based on Gaussian kernel densitybased local outlier factor (GKLOF) mainly includes the following steps: First, the original local outlier factor (Local OutlierFac, LOF) algorithm is analyzed. The specific steps of the LOF algorithm are as follows:

[0090] (1) Find the distance between each node and its nearest node, that is, K distance K dist (p).

[0091] (2) Calculate the K neighborhood N of each node k (p):

[0092] N k (p)={q∈N / {p|dist(p,q)≤K dist (p)}

[0093] Among them, dist(p,q) represents the spatial distance between the p-th object and the q-th object in the data.

[0094] (3) Determine the local reachable distance D of each node reach (p,q):

[0095] D reach(p,q) =max{K dist (q),dist(p,q)}

[0096] (4) Calculate the local reachability density ρ of each node Irdk (p):

[0097]

[0098] Where, ο represents the K neighborhood N k(p) Any object in .

[0099] (5) Solve the local anomaly factor f of each object LOFk (p):

[0100]

[0101] In this embodiment, the Gaussian kernel density function is used to improve the local outlier factor LOF algorithm to reduce the impact of local data distribution on the detection results (see Figure 4 The effect of local data distribution disturbance on the LOF and GKLOF values ​​of data point p is shown in the figure. The specific improvements are as follows:

[0102] D k (p): The local density of the Gaussian kernel at data point p can be expressed as:

[0103]

[0104] Where G is the Gaussian kernel density function; ||x0-x p || is the Euclidean distance between data points p and o; N k (p) is the number of data points in the k-order domain of data point p; h is the standard deviation of the distances between all sample data points in this embodiment, and the standard deviation is quoted to reflect the degree of change in the distances between all sample points.

[0105]

[0106] The above formula is the Gaussian kernel distance between data points p and o. By calculating this formula, the k-order distance range can be determined. At the same time, the Gaussian kernel distance also characterizes the attenuation degree of the distance between data point p and its adjacent points, reflecting the size of the local density of the Gaussian kernel. For example, for outliers, the attenuation degree is usually higher than that of normal points, and the local density of the Gaussian kernel is small. On the contrary, the attenuation degree is small, and the local density of the Gaussian kernel is approximately equal to 1.

[0107] F k (p): The Gaussian kernel local outlier factor of data point p is expressed as follows:

[0108]

[0109] By analyzing the above three formulas (Gaussian kernel local density of data point p, Gaussian kernel distance between data point p and o, and Gaussian kernel local outlier factor of data point p), we can further obtain the rules for distinguishing normal points and outliers as shown in the following formula:

[0110]

[0111] By comparing F k By comparing the size of (p) and 1, normal points and outliers can be distinguished, and abnormal devices can be identified.

[0112] It is understandable that the Local Outlier Factor (LOF) algorithm is an unsupervised method for detecting outliers in a data set. The LOF algorithm calculates the local density between each data point and its neighbors, and those data points with lower local density are regarded as outliers. However, the LOF algorithm is very sensitive to the local distribution of data points, and may mistakenly regard some normal data points located near dense areas as outliers. In the present embodiment, in order to reduce the influence of the local data distribution on the LOF detection result, the LOF algorithm is improved by using the Gaussian kernel density function to obtain the Gaussian kernel function local outlier factor GKLOF. Since the Gaussian kernel density estimation can smooth the local density estimation, making it less sensitive to the local distribution of data points, by converting the distance into the density value of the Gaussian kernel, the influence on the LOF value due to the uneven distribution of local data can be reduced. Therefore, in the present embodiment, the local density around each data point can be estimated more accurately by using the Gaussian kernel density estimation, so that the LOF algorithm is insensitive to the local distribution of data points.

[0113] The improved Gaussian kernel function local outlier factor GKLOF in this embodiment is used to detect outliers in a data set, combining Gaussian kernel density estimation and local outlier factor to more accurately identify outliers in a data set. Gaussian kernel density estimation is used to estimate the local density of each data point, and then the outlier factor of each point is calculated based on these local densities. Compared with the traditional LOF algorithm, the GKLOF in this embodiment uses Gaussian kernel density estimation to better handle data points in different density areas, thereby reducing false positives and false negatives. In practical applications, the parameters of the kernel function and the number of nearest neighbors can be adjusted according to the specific data set and requirements.

[0114] In one embodiment, the multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device, and a voiceprint.

[0115] It should be noted that, refer to Figure 5 The drone low-altitude patrol of the main transformer shown can collect substation data through drone inspection. Here, the multimodal sensor includes but is not limited to drones, inspection robots, optical cameras, infrared cameras, visual micro-meteorological monitoring devices, and voiceprints, which are not limited in this embodiment.

[0116] Specifically, Figure 6As shown, the substation data collected by the multimodal sensor is input into the multimodal fusion algorithm. The multimodal fusion algorithm framework includes a convolutional neural network, a multi-head attention mechanism, and a Gaussian kernel function local outlier factor. After input into the multimodal feature neural network, feature fusion and expression, and feature classification are performed to obtain multimodal AI structured data, such as optical camera image structured data, voiceprint structured data, drone image structured data, and infrared image structured data. Data analysis is performed based on the data output by the multimodal fusion algorithm framework to determine abnormal points, which can be displayed on the large screen for centralized equipment control.

[0117] This embodiment provides a method for detecting abnormalities of substations using drones based on a multimodal fusion algorithm, including: collecting substation data through a multimodal sensor, and inputting the substation data into a multimodal fusion algorithm; extracting features from the substation data based on a convolutional neural network based on the multimodal fusion algorithm, and outputting multiple data features; fusing the data features based on a multi-head attention mechanism based on the multimodal fusion algorithm to obtain fusion features; and classifying features based on the fusion features based on the Gaussian kernel function local outlier factor of the multimodal fusion algorithm to determine abnormal points. In this embodiment, a feature classification method based on the Gaussian kernel function local outlier factor, a convolutional neural network, and a multi-head attention mechanism form a multimodal fusion algorithm, and drone substation abnormality detection is realized through a multimodal fusion algorithm, thereby improving the intelligence level and abnormality detection capability of the substation inspection system, and realizing fully autonomous, efficient, and safe intelligent inspections of substations, thereby providing strong support for the stable operation of the power grid.

[0118] In addition, an embodiment of the present invention also proposes a storage medium, on which is stored a drone substation anomaly detection program based on a multimodal fusion algorithm. When the drone substation anomaly detection program based on a multimodal fusion algorithm is executed by a processor, the steps of the drone substation anomaly detection method based on a multimodal fusion algorithm as described above are implemented.

[0119] Reference Figure 7 , Figure 7 This is a structural block diagram of an embodiment of the UAV substation anomaly detection system based on the multimodal fusion algorithm of the present invention.

[0120] like Figure 7 As shown, the UAV substation anomaly detection system based on the multimodal fusion algorithm includes:

[0121] A data acquisition module 10, for collecting substation data through a multimodal sensor and inputting the substation data into a multimodal fusion algorithm;

[0122] A feature extraction module 20, configured to extract features from the substation data based on a convolutional neural network of the multimodal fusion algorithm and output a plurality of data features;

[0123] A feature fusion module 30, used to fuse the data features based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features;

[0124] The feature classification module 40 is used to perform feature classification according to the fusion features based on the Gaussian kernel function local outlier factor of the multimodal fusion algorithm to determine the abnormal points.

[0125] In one embodiment, the multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device, and a voiceprint.

[0126] This embodiment proposes a UAV substation anomaly detection system based on a multimodal fusion algorithm. In this embodiment, a feature classification method based on the local outlier factor of the Gaussian kernel function is used to form a multimodal fusion algorithm with a convolutional neural network and a multi-head attention mechanism. The UAV substation anomaly detection is realized through the multimodal fusion algorithm, the intelligence level and anomaly detection capability of the substation inspection system are improved, and the substation is fully autonomous, efficient and safe. Intelligent inspection, thereby providing strong support for the stable operation of the power grid.

[0127] It should be noted that the technical details that are not described in detail in the embodiment of the drone substation anomaly detection system based on the multimodal fusion algorithm can be referred to the drone substation anomaly detection method based on the multimodal fusion algorithm as described above provided in any embodiment of the present invention, and will not be repeated here.

[0128] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0129] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0130] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0131] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0133] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A UAV substation anomaly detection method based on multimodal fusion algorithm, characterized in that: include: Collecting substation data through a multimodal sensor and inputting the substation data into a multimodal fusion algorithm; A convolutional neural network based on the multimodal fusion algorithm performs feature extraction on the substation data and outputs a plurality of data features; The data features are fused based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features; The Gaussian kernel function local outlier factor based on the multimodal fusion algorithm performs feature classification according to the fusion features to determine the abnormal points.

2. The method according to claim 1, characterized in that The convolutional neural network based on the multimodal fusion algorithm extracts features from the substation data and outputs multiple data features, including: Inputting the substation data into the convolutional neural network of the multimodal fusion algorithm to automatically extract features of the substation data through the convolutional layer of the convolutional neural network; Applying a non-linear activation function after the convolutional layer; Reducing the dimension of features through the pooling layer of the convolutional neural network; The features are flattened by repeating multiple layers of the convolutional layer and the pooling layer, and input into the fully connected layer; The features are integrated through the fully connected layer and the classification results are output to obtain multiple data features.

3. The method according to claim 2, characterized in that The convolutional neural network includes a convolutional layer, an activation function, a pooling layer and a fully connected layer.

4. The method according to claim 1, characterized in that The multi-head attention mechanism based on the multimodal fusion algorithm fuses the data features to obtain fused features, including: Generate a feature matrix according to the multiple data features extracted by the convolutional neural network; Calculate a query matrix, a key matrix and a value matrix according to the feature matrix; Calculate similarity scores between the query matrix, the key matrix, and the value matrix based on a multi-head attention mechanism of the multimodal fusion algorithm; The multi-head attention mechanism feature fusion vector is solved according to the similarity score to obtain the fusion feature.

5. The method according to claim 4, characterized in that The query matrix Q, key matrix K and value matrix V are respectively: Q=W q X,K=W k X,V=W v X Wherein, X is the feature matrix extracted by the convolutional neural network; W q , W k , W v are the weight matrices of Q, K, and V respectively.

6. The method according to any one of claims 1 to 5, characterized in that The Gaussian kernel function local outlier factor based on the multimodal fusion algorithm performs feature classification according to the fusion feature to determine the abnormal point, including: The local outlier factor algorithm is improved by using Gaussian kernel density function to obtain the Gaussian kernel function local outlier factor; Based on the local outlier factor of the Gaussian kernel function, feature classification is performed according to the fusion feature to determine the abnormal point.

7. The method according to claim 6, characterized in that The performing feature classification based on the Gaussian kernel function local outlier factor according to the fusion feature to determine the abnormal point includes: Determine the Gaussian kernel local density of the data point p of the fusion feature based on the Gaussian kernel function local outlier factor; Calculate the Gaussian kernel distance between data point p and data point o; The Gaussian kernel local outlier factor of data point p is calculated based on the Gaussian kernel local density of data point p and the Gaussian kernel local density of data point o; Analyze the Gaussian kernel local density, Gaussian kernel distance and Gaussian kernel local outlier factor to obtain a rule for distinguishing normal points and outliers; The abnormal points are determined based on the normal point and outlier point distinction rule.

8. The method according to claim 1, characterized in that The multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device and a voiceprint.

9. A UAV substation anomaly detection system based on multimodal fusion algorithm, characterized in that: include: A data acquisition module, used to collect substation data through a multimodal sensor and input the substation data into a multimodal fusion algorithm; A feature extraction module, used for extracting features from the substation data based on the convolutional neural network of the multimodal fusion algorithm, and outputting a plurality of data features; A feature fusion module, used to fuse the data features based on the multi-head attention mechanism of the multimodal fusion algorithm to obtain fused features; The feature classification module is used to perform feature classification according to the fusion features based on the Gaussian kernel function local outlier factor of the multimodal fusion algorithm to determine the abnormal points.

10. The system according to claim 9, characterized in that The multimodal sensor includes a drone, an inspection robot, an optical camera, an infrared camera, a visual micro-meteorological monitoring device and a voiceprint.

Images