Transformer anomaly detection method and device, computer device, storage medium and computer program product

Abstract

The application relates to a transformer anomaly detection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring current multi-modal data of a transformer to be detected; the current multi-modal data at least comprises audio data, infrared video data and temperature data; performing anomaly detection processing on the audio data to obtain an audio anomaly detection result corresponding to the audio data; in the case that the audio anomaly detection result is abnormal, inputting the current multi-modal data into a pre-trained multi-modal attention anomaly detection model to obtain current target fusion data corresponding to the current multi-modal data; acquiring historical target fusion data corresponding to historical multi-modal data of the transformer to be detected; and obtaining an anomaly detection result corresponding to the transformer to be detected according to the similarity between the current target fusion data and the historical target fusion data. The method can improve the anomaly detection accuracy of the transformer.

Description

Technical Field

[0001] This application relates to the field of power grid technology, and in particular to a transformer anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] In the power grid sector, abnormal detection of transformers is crucial for ensuring the safe and stable operation of the power system.

[0003] In traditional technology, transformer anomaly detection is generally performed by taking intermittent photos of the transformer. However, this method yields relatively limited data, which can only reflect the local condition of the transformer, resulting in a low accuracy rate for transformer anomaly detection. Summary of the Invention

[0004] Therefore, it is necessary to provide a transformer anomaly detection method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of transformer anomaly detection in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for detecting transformer anomalies, including:

[0006] Acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data, and temperature data;

[0007] Anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data;

[0008] If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data;

[0009] Historical target fusion data corresponding to the historical multimodal data of the transformer under test is obtained, and the anomaly detection result corresponding to the transformer under test is obtained based on the similarity between the current target fusion data and the historical target fusion data.

[0010] In one embodiment, the step of inputting the current multimodal data into a pre-trained multimodal attention anomaly detection model to obtain current target fusion data corresponding to the current multimodal data includes:

[0011] The pre-trained multimodal attention anomaly detection model is used to extract features from the audio data, infrared video data, and temperature data to obtain audio data features, infrared video data features, and temperature data features.

[0012] Based on the infrared video data features and the temperature data features, the audio data features are updated to obtain the first updated features and the second updated features. Based on the audio data features and the temperature data features, the infrared video data features are updated to obtain the third updated features and the fourth updated features. Based on the audio data features and the infrared video data features, the temperature data features are updated to obtain the fifth updated features and the sixth updated features.

[0013] Based on the first update feature, the second update feature, the third update feature, the fourth update feature, the fifth update feature, and the sixth update feature, the current target fusion data corresponding to the current multimodal data is obtained.

[0014] In one embodiment, obtaining the current target fusion data corresponding to the current multimodal data based on the first update feature, the second update feature, the third update feature, the fourth update feature, the fifth update feature, and the sixth update feature includes:

[0015] The first update feature and the second update feature are concatenated to obtain the first concatenated feature; the third update feature and the fourth update feature are concatenated to obtain the second concatenated feature; and the fifth update feature and the sixth update feature are concatenated to obtain the third concatenated feature.

[0016] The first splicing feature, the second splicing feature, and the third splicing feature are spliced ​​again to obtain the target splicing feature, which serves as the current target fusion data corresponding to the current multimodal data.

[0017] In one embodiment, the step of performing anomaly detection processing on the audio data to obtain an audio anomaly detection result corresponding to the audio data includes:

[0018] Extract the target audio features from the audio data and use them as the current target audio features;

[0019] Based on the current target audio features and historical target audio features, the audio anomaly detection result corresponding to the audio data is determined; the historical target audio features are the target audio features of the audio data in the historical multimodal data.

[0020] In one embodiment, the pre-trained multimodal attention anomaly detection model is trained in the following manner:

[0021] Acquire sample multimodal data of the sample transformer; the sample multimodal data includes at least audio data, infrared video data, and temperature data;

[0022] The sample multimodal data is input into the multimodal attention anomaly detection model to be trained to obtain sample target fusion data corresponding to the sample multimodal data;

[0023] Based on the sample target fusion data, the multimodal attention anomaly detection model to be trained is iteratively trained to obtain the pre-trained multimodal attention anomaly detection model.

[0024] In one embodiment, after acquiring historical target fusion data corresponding to the historical multimodal data of the transformer under test, and obtaining the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data, the method further includes:

[0025] If the abnormal detection result corresponding to the transformer under test is abnormal, an early warning message is generated for the transformer under test based on the current multimodal data.

[0026] The warning information is sent to the terminal associated with the transformer to be tested.

[0027] Secondly, this application also provides a transformer anomaly detection device, comprising:

[0028] The data acquisition module is used to acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data and temperature data;

[0029] The audio detection module is used to perform anomaly detection processing on the audio data and obtain the audio anomaly detection result corresponding to the audio data;

[0030] The data fusion module is used to input the current multimodal data into a pre-trained multimodal attention anomaly detection model when the audio anomaly detection result is abnormal, so as to obtain the current target fusion data corresponding to the current multimodal data;

[0031] The result determination module is used to acquire historical target fusion data corresponding to the historical multimodal data of the transformer under test, and to obtain the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data.

[0032] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0033] Acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data, and temperature data;

[0034] Anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data;

[0035] If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data;

[0036] Historical target fusion data corresponding to the historical multimodal data of the transformer under test is obtained, and the anomaly detection result corresponding to the transformer under test is obtained based on the similarity between the current target fusion data and the historical target fusion data.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0038] Acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data, and temperature data;

[0039] Anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data;

[0040] If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data;

[0041] Historical target fusion data corresponding to the historical multimodal data of the transformer under test is obtained, and the anomaly detection result corresponding to the transformer under test is obtained based on the similarity between the current target fusion data and the historical target fusion data.

[0042] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0043] Acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data, and temperature data;

[0044] Anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data;

[0045] If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data;

[0046] Historical target fusion data corresponding to the historical multimodal data of the transformer under test is obtained, and the anomaly detection result corresponding to the transformer under test is obtained based on the similarity between the current target fusion data and the historical target fusion data.

[0047] The aforementioned transformer anomaly detection method, apparatus, computer equipment, storage medium, and computer program product first acquire audio data, infrared video data, and temperature data of the transformer under test as the current multimodal data of the transformer under test. Then, anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data. If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data. Finally, historical target fusion data corresponding to the historical multimodal data of the transformer under test is acquired. Based on the similarity between the current target fusion data and the historical target fusion data, the anomaly detection result corresponding to the transformer under test is obtained. In this way, when performing anomaly detection on transformers, anomaly detection is first performed on the audio data in the current multimodal data. If the audio anomaly detection result is abnormal, the multimodal attention anomaly detection model is then used to further detect anomalies in the current multimodal data. This allows for a more accurate identification of potential problems and helps reduce unnecessary computational resource consumption. Moreover, by comprehensively utilizing multimodal information such as audio data, infrared video data, and temperature data, the operating status of the transformer can be more comprehensively reflected. This avoids the shortcomings of relying on a single data source, which can only reflect a local situation of the transformer and leads to a low accuracy rate in anomaly detection, thus improving the accuracy rate of transformer anomaly detection. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a transformer anomaly detection method in one embodiment;

[0050] Figure 2 Here is a structural diagram of a multimodal attention anomaly detection model in one embodiment;

[0051] Figure 3 This is a flowchart illustrating the steps for obtaining the current target fused data corresponding to the current multimodal data in one embodiment;

[0052] Figure 4 Here is a cross-modal Transformer structure diagram in one embodiment;

[0053] Figure 5 This is a diagram illustrating the cross-modal attention structure between different modal sequences in one embodiment.

[0054] Figure 6 This is a flowchart illustrating a transformer anomaly detection method in another embodiment;

[0055] Figure 7 This is a flowchart of a transformer edge-end anomaly detection system based on multimodal attention in one embodiment;

[0056] Figure 8 This is a structural block diagram of a transformer anomaly detection device in one embodiment;

[0057] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0060] In one exemplary embodiment, such as Figure 1 As shown, a transformer anomaly detection method is provided. This embodiment illustrates the application of this method to a server; it is understood that this method can also be applied to a terminal, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets; the server can be a standalone server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0061] Step S101: Obtain the current multimodal data of the transformer to be tested; the current multimodal data includes at least audio data, infrared video data, and temperature data.

[0062] Among them, the transformer to be tested refers to the transformer that needs to be tested for abnormalities.

[0063] Among them, the current multimodal data refers to the multimodal data corresponding to the current time of the transformer under test, including audio data, infrared video data, and temperature data.

[0064] Among them, audio data refers to audio-related data of the transformer under test, including short-time energy, short-time zero-crossing rate, short-time autocorrelation coefficient, short-time average amplitude difference function, linear prediction cepstral coefficient spectral characteristics, and Mel cepstral coefficients.

[0065] Infrared video data refers to video data of the operating status of the transformer to be tested, obtained through infrared thermal imaging technology.

[0066] Temperature data refers to data related to the temperature of the transformer under test, such as temperature magnitude and temperature change.

[0067] For example, in response to an anomaly detection command for the transformer under test, the server obtains the current time audio data of the transformer under test from an audio sensor deployed on the transformer under test, the current time infrared video data of the transformer under test from an infrared camera deployed on the transformer under test, and the current time temperature data of the transformer under test from a temperature sensor deployed on the transformer under test; then, the server uses the current time audio data, infrared video data, and temperature data of the transformer under test as the current multimodal data of the transformer under test.

[0068] Step S102: Perform anomaly detection processing on the audio data to obtain the audio anomaly detection result corresponding to the audio data.

[0069] Anomaly detection processing refers to the process of identifying whether there is abnormal audio data in the audio data.

[0070] The audio anomaly detection result refers to the anomaly detection result corresponding to the audio data. In real-world scenarios, the audio anomaly detection result is either normal or abnormal.

[0071] For example, the server obtains the historical audio data corresponding to the audio data; then, the server performs anomaly detection processing on the audio data based on the audio data and the historical audio data to obtain the audio anomaly detection result corresponding to the audio data.

[0072] Step S103: If the audio anomaly detection result is abnormal, input the current multimodal data into the pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data.

[0073] The multimodal attention anomaly detection model refers to a network model that can utilize the current multimodal data to obtain the fused data corresponding to the current target data. It should be noted that the structure diagram of the multimodal attention anomaly detection model is as follows: Figure 2 As shown.

[0074] Here, the current target fused data refers to the fusion embedding code corresponding to the current multimodal data, which can be represented by emb′.

[0075] For example, when the audio anomaly detection result is abnormal, the server inputs the audio data, infrared video data, and temperature data into a pre-trained multimodal attention anomaly detection model. The pre-trained multimodal attention anomaly detection model performs feature extraction processing on the audio data, infrared video data, and temperature data to obtain audio data features, infrared video data features, and temperature data features. Then, based on the audio data features, infrared video data features, and temperature data features, the server obtains the target fusion data corresponding to the audio data features, infrared video data features, and temperature data features, which is used as the current target fusion data corresponding to the current multimodal data.

[0076] Step S104: Obtain historical target fusion data corresponding to the historical multimodal data of the transformer under test, and obtain the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data.

[0077] Historical multimodal data refers to the multimodal data corresponding to historical time periods of the transformer under test. It should be noted that historical multimodal data is used to characterize the multimodal data corresponding to normal operation of the transformer under test in relation to abnormal detection results.

[0078] Among them, historical target fusion data refers to the fusion embedding code corresponding to historical multimodal data.

[0079] Here, similarity is used to characterize the degree of similarity between the current target fused data and the historical target fused data. In real-world scenarios, the similarity is cosine similarity.

[0080] The abnormal detection result for the transformer under test refers to the comprehensive abnormal detection result for that transformer. In real-world scenarios, the abnormal detection result for the transformer under test is either normal or abnormal.

[0081] For example, the server acquires historical multimodal data corresponding to the multimodal data as historical multimodal data of the transformer to be detected; then, the server acquires target fusion data corresponding to the historical multimodal data of the transformer to be detected as historical target fusion data; next, the server determines the cosine similarity between the current target fusion data and the historical target fusion data as the similarity between the current target fusion data and the historical target fusion data; finally, the server obtains the anomaly detection result corresponding to the transformer to be detected based on the similarity between the current target fusion data and the historical target fusion data.

[0082] For example, acquiring multiple target fusion data (emb) corresponding to the historical multimodal data of the transformer under test. n The vector average of multiple historical target fusion data is taken to obtain historical target fusion data c. The cosine similarity cos(c,emb′) between the current target fusion data and the historical target fusion data is calculated. After activation by the sigmoid function σ(x), it is compared with the threshold score. If the result is greater than or equal to the threshold score, the abnormal detection result corresponding to the transformer to be detected is judged to be normal. If the result is less than the threshold score, the abnormal detection result corresponding to the transformer to be detected is judged to be abnormal.

[0083] For example, the server can calculate historical target fusion data using the following formula:

[0084]

[0085] in, This refers to the Expectation-Maximization algorithm, where n represents the number of target fusion data.

[0086] For example, the server can calculate historical target fusion data using the following formula:

[0087]

[0088] Where y refers to the abnormal detection result corresponding to the transformer under test.

[0089] In the above-mentioned transformer anomaly detection method, audio data, infrared video data, and temperature data of the transformer to be detected are first acquired as the current multimodal data of the transformer to be detected. Then, anomaly detection processing is performed on the audio data to obtain the audio anomaly detection result corresponding to the audio data. If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data. Finally, historical target fusion data corresponding to the historical multimodal data of the transformer to be detected is acquired. Based on the similarity between the current target fusion data and the historical target fusion data, the anomaly detection result corresponding to the transformer to be detected is obtained. In this way, when performing anomaly detection on transformers, anomaly detection is first performed on the audio data in the current multimodal data. If the audio anomaly detection result is abnormal, the multimodal attention anomaly detection model is then used to further detect anomalies in the current multimodal data. This allows for a more accurate identification of potential problems and helps reduce unnecessary computational resource consumption. Moreover, by comprehensively utilizing multimodal information such as audio data, infrared video data, and temperature data, the operating status of the transformer can be more comprehensively reflected. This avoids the shortcomings of relying on a single data source, which can only reflect a local situation of the transformer and leads to a low accuracy rate in anomaly detection, thus improving the accuracy rate of transformer anomaly detection.

[0090] In one exemplary embodiment, such as Figure 3 As shown, step S103 above, which inputs the current multimodal data into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data, specifically includes the following steps:

[0091] Step S301: Using a pre-trained multimodal attention anomaly detection model, feature extraction processing is performed on audio data, infrared video data, and temperature data to obtain audio data features, infrared video data features, and temperature data features.

[0092] Step S302: Based on the infrared video data features and temperature data features, update the audio data features to obtain the first updated features and the second updated features. Based on the audio data features and temperature data features, update the infrared video data features to obtain the third updated features and the fourth updated features. Based on the audio data features and infrared video data features, update the temperature data features to obtain the fifth updated features and the sixth updated features.

[0093] Step S303: Based on the first update feature, the second update feature, the third update feature, the fourth update feature, the fifth update feature, and the sixth update feature, obtain the current target fusion data corresponding to the current multimodal data.

[0094] Feature extraction processing refers to the process of extracting data features.

[0095] Among them, audio data features refer to the feature vectors corresponding to audio data.

[0096] Among them, infrared video data features refer to the feature vectors corresponding to infrared video data.

[0097] Among them, temperature data features refer to the feature vectors corresponding to temperature data.

[0098] Among them, update processing refers to the process of adjusting the features.

[0099] The first updated feature refers to the feature vector obtained by updating the audio data features based on the infrared video data features.

[0100] The second update feature refers to the feature vector obtained by updating the audio data features based on the temperature data features.

[0101] The third update feature refers to the feature vector obtained by updating the infrared video data features based on the audio data features.

[0102] The fourth update feature refers to the feature vector obtained by updating the infrared video data features based on the temperature data features.

[0103] The fifth update feature refers to the feature vector obtained by updating the temperature data features based on the audio data features.

[0104] The sixth update feature refers to the feature vector obtained by updating the temperature data features based on the infrared video data features.

[0105] For example, the server inputs audio data, infrared video data, and temperature data into a pre-trained multimodal attention anomaly detection model. The model performs feature extraction on the audio, infrared video, and temperature data, obtaining feature vectors corresponding to the audio, infrared video, and temperature data, which are used as the audio, infrared video, and temperature data features, respectively. Then, the server updates the audio data features based on the infrared video data features to obtain the first updated features, and then updates the audio data features based on the temperature data features to obtain the second updated features. The second update feature; then, the server updates the infrared video data features based on the audio data features to obtain the third update feature, and updates the infrared video data features based on the temperature data features to obtain the fourth update feature; next, the server updates the temperature data features based on the audio data features to obtain the fifth update feature, and updates the temperature data features based on the infrared video data features to obtain the sixth update feature; finally, the server obtains the current target fusion data corresponding to the current multimodal data based on the first, second, third, fourth, fifth, and sixth update features.

[0106] For example, the server performs feature extraction on audio data 'a', infrared video data 'v', and temperature data 'tpt', and then performs further feature extraction through a one-dimensional temporal convolutional layer to obtain the processed features. And by embedding position encoding, low-level position-aware features are obtained. like Figure 4 , Figure 5 As shown, the server uses a Transformer to update the audio data features multiple times based on the infrared video data features and temperature data features, to obtain the first and second updated features. Based on the audio data features and temperature data features, the server updates the infrared video data features multiple times, to obtain the third and fourth updated features. Based on the audio data features and infrared video data features, the server updates the temperature data features multiple times, to obtain the fifth and sixth updated features.

[0107] For example, the server can calculate the result of further feature extraction through a one-dimensional temporal convolutional layer using the following formula:

[0108]

[0109] Where, k {a，v，tpt} The size of the corresponding modality convolution kernel is d, where d is the common dimension.

[0110] For example, the server can calculate low-level location-aware features using the following formula:

[0111]

[0112] in, This indicates the embedding of each position index.

[0113] For example, the server can calculate an update for a feature using the following formula:

[0114]

[0115] Among them, Y a It refers to the features after an update, CM v→a (X a X v ) indicates that the transit mode is X a The target mode is X v A single-head cross-modal Transformer attention mechanism, in It is a learnable weight matrix.

[0116] For example, the server can calculate the number of times a feature is updated using the following formula:

[0117]

[0118] in, This refers to the feature after multiple updates, specifically the first updated feature. It should be noted that the second, third, fourth, fifth, and sixth updated features can also be obtained using the same method.

[0119] In this embodiment, by fusing and interactively updating multimodal data, abnormal information can be captured more comprehensively and holistically, thereby improving the accuracy of anomaly detection. This is beneficial for a deeper understanding of the correlation and mutual influence between audio data, infrared video data, and temperature data. Moreover, by combining the processing of multiple types of data features, the limitations and biases that may exist in a single data type are reduced, making the anomaly detection results of the transformer more robust.

[0120] In an exemplary embodiment, step S303 above, which obtains the current target fusion data corresponding to the current multimodal data based on the first updated feature, the second updated feature, the third updated feature, the fourth updated feature, the fifth updated feature, and the sixth updated feature, further includes: concatenating the first updated feature and the second updated feature to obtain a first concatenated feature; concatenating the third updated feature and the fourth updated feature to obtain a second concatenated feature; and concatenating the fifth updated feature and the sixth updated feature to obtain a third concatenated feature; and concatenating the first concatenated feature, the second concatenated feature, and the third concatenated feature again to obtain a target concatenated feature, which serves as the current target fusion data corresponding to the current multimodal data.

[0121] Among them, splicing processing refers to the process of combining features.

[0122] For example, the server can calculate the first, second, and third splicing features using the following formula:

[0123]

[0124] Here, concat refers to concatenation processing, Z a This refers to the first splicing feature. This refers to the first update feature. This refers to the second update feature, Z. v This refers to the second splicing feature. This refers to the third update feature. This refers to the fourth update feature, Z. tpt This refers to the third splicing feature. This refers to the fifth update feature. This refers to the sixth update feature.

[0125] The first concatenated feature refers to the feature vector obtained by concatenating the first updated feature and the second updated feature.

[0126] The second concatenated feature refers to the feature vector obtained by concatenating the third and fourth updated features.

[0127] The third concatenated feature refers to the feature vector obtained by concatenating the fifth and sixth updated features.

[0128] The target splicing feature refers to the feature vector obtained by splicing the first splicing feature, the second splicing feature, and the third splicing feature again.

[0129] For example, the server can calculate the target stitching features using the following formula:

[0130] emb = concat[Za Z v Z tpt Equation (8)

[0131] Here, emb refers to the target splicing feature.

[0132] For example, the server concatenates the first and second updated features to obtain a concatenated feature corresponding to the first and second updated features, which is used as the first concatenated feature. Next, the server concatenates the third and fourth updated features to obtain a concatenated feature corresponding to the third and fourth updated features, which is used as the second concatenated feature. Then, the server concatenates the fifth and sixth updated features to obtain a concatenated feature corresponding to the fifth and sixth updated features, which is used as the third concatenated feature. Finally, the server concatenates the first, second, and third concatenated features again to obtain the target concatenated feature, which is used as the current target fusion data corresponding to the current multimodal data.

[0133] In this embodiment, by splicing different update features, it is possible to integrate data from multiple aspects and provide a more comprehensive and richer feature representation. Moreover, multiple splicing processes help to better integrate data from multiple modalities, making the final target spliced ​​features more representative and accurate.

[0134] In an exemplary embodiment, step S102, which involves performing anomaly detection processing on the audio data to obtain an audio anomaly detection result corresponding to the audio data, further includes: extracting the target audio features of the audio data as the current target audio features; determining the audio anomaly detection result corresponding to the audio data based on the current target audio features and historical target audio features; the historical target audio features are the target audio features of the audio data in historical multimodal data.

[0135] Among them, target audio features refer to the feature data in the audio data that are related to the anomaly detection results, including short-time energy, short-time zero-crossing rate, short-time autocorrelation coefficient, short-time average amplitude difference function, linear prediction cepstral coefficient spectral features, and Mel cepstral coefficients, etc.

[0136] Among them, the current target audio feature refers to the target audio feature of the audio data.

[0137] Among them, historical target audio features refer to the target audio features of audio data in historical multimodal data.

[0138] For example, the server determines the importance of each sub-audio data in the audio data; then, the server filters out the sub-audio data whose importance meets a preset importance level as the target audio feature of the audio data; then, the server uses the target audio feature of the audio data as the current target audio feature; next, the server averages the feature vectors corresponding to each sub-audio data in the current target audio feature to obtain the first feature vector corresponding to the current target audio feature, and averages the feature vectors corresponding to each sub-audio data in the historical target audio features to obtain the second feature vector corresponding to the historical target audio feature; then, the server determines the audio anomaly detection result corresponding to the audio data based on the cosine similarity between the first feature vector and the second feature vector; for example, if the cosine similarity is within a certain threshold, the audio anomaly detection result is determined to be normal, the detection ends, and no further action is taken; if the cosine similarity exceeds a certain threshold, the audio anomaly detection result is determined to be abnormal, and subsequent steps are performed.

[0139] In this embodiment, by comparing the current target audio features with the historical target audio features, it is possible to more accurately discover possible anomalies in the audio data; moreover, by making full use of the historical target audio features, the detection results are more referential and reliable, thereby enabling continuous improvement of detection capabilities based on the accumulation of historical data.

[0140] In an exemplary embodiment, the transformer anomaly detection method provided in this application further includes a training step of a pre-trained multimodal attention anomaly detection model, which further includes: acquiring sample multimodal data of a sample transformer; the sample multimodal data includes at least audio data, infrared video data, and temperature data; inputting the sample multimodal data into the multimodal attention anomaly detection model to be trained to obtain sample target fusion data corresponding to the sample multimodal data; and iteratively training the multimodal attention anomaly detection model to be trained based on the sample target fusion data to obtain the pre-trained multimodal attention anomaly detection model.

[0141] Among them, the sample transformer refers to the transformer used to train the multimodal attention anomaly detection model.

[0142] Among them, sample multimodal data refers to the multimodal data of the sample transformer, including audio data, infrared video data, and temperature data.

[0143] Here, the sample audio anomaly detection result refers to the anomaly detection result corresponding to the audio data of the sample multimodal data. In real-world scenarios, the sample audio anomaly detection result is either normal or abnormal.

[0144] Among them, the sample target fusion data refers to the fusion embedding code corresponding to the sample multimodal data.

[0145] For example, in response to a model training instruction for a multimodal attention anomaly detection model, the server retrieves audio data, infrared video data, and temperature data of a sample transformer from a database, all of which are used as sample multimodal data of the sample transformer. Then, the server inputs the sample multimodal data into the multimodal attention anomaly detection model to be trained, obtaining target fusion data corresponding to the sample multimodal data, which is used as sample target fusion data. Next, the server obtains a loss value based on the differences between the sub-sample target fusion data in the sample target fusion data. Then, the server adjusts the model parameters of the multimodal attention anomaly detection model to be trained based on the loss value. Then, the server retrains the multimodal attention anomaly detection model with adjusted model parameters until the loss value obtained by the trained multimodal attention anomaly detection model is less than a loss value threshold, at which point training stops, and this trained multimodal attention anomaly detection model is used as the pre-trained multimodal attention anomaly detection model.

[0146] For example, the server can calculate the loss value using the following formula:

[0147]

[0148] Here, Loss refers to the loss value, N is the number of embeddings (vectors), ∥·∥ represents the matrix norm, margin is a hyperparameter representing the similarity threshold, and sim refers to the similarity calculation. It should be noted that the loss function can encourage cosine similarity between similar embedding pairs to be greater than the margin.

[0149] In this embodiment, by pre-training a multimodal attention anomaly detection model, it is convenient to predict the anomaly detection result corresponding to the transformer to be detected after obtaining the current multimodal data of the transformer to be detected in practical applications. Moreover, the multimodal attention anomaly detection model receives new data in each iteration, and performs internal model improvement and optimization, which makes it easier to make predictions more effectively and improves the prediction accuracy of the multimodal attention anomaly detection model.

[0150] In an exemplary embodiment, step S104 above, after obtaining historical target fusion data corresponding to the historical multimodal data of the transformer under test, and obtaining the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data, further includes: if the anomaly detection result corresponding to the transformer under test is abnormal, generating early warning information for the transformer under test based on the current multimodal data; and sending the early warning information to the terminal associated with the transformer under test.

[0151] Among them, early warning information refers to warning information related to abnormalities in the transformer under test.

[0152] For example, when the abnormal detection result corresponding to the transformer under test is abnormal, the server generates an early warning message corresponding to the current multimodal data based on the current multimodal data, as an early warning message for the transformer under test; then, the server determines the terminal associated with the transformer under test and sends the early warning message to the terminal associated with the transformer under test.

[0153] In this embodiment, by rapidly generating and sending early warning information when an abnormality occurs in the transformer under test, relevant personnel are informed of the situation in a timely manner, which is conducive to taking timely measures to avoid potential dangers and failures, ensuring the safe operation of the transformer under test and related systems, and thus improving the safety of the transformer under test.

[0154] In one exemplary embodiment, such as Figure 6 As shown, another method for transformer anomaly detection is provided. Taking the application of this method to a server as an example, it includes the following steps:

[0155] Step S601: Obtain the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data and temperature data.

[0156] Step S602: Extract the target audio features from the audio data as the current target audio features; determine the audio anomaly detection result corresponding to the audio data based on the current target audio features and the historical target audio features; the historical target audio features are the target audio features of the audio data in the historical multimodal data.

[0157] Step S603: Using a pre-trained multimodal attention anomaly detection model, feature extraction processing is performed on audio data, infrared video data, and temperature data to obtain audio data features, infrared video data features, and temperature data features.

[0158] Step S604: Based on the infrared video data features and temperature data features, update the audio data features to obtain the first updated features and the second updated features. Based on the audio data features and temperature data features, update the infrared video data features to obtain the third updated features and the fourth updated features. Based on the audio data features and infrared video data features, update the temperature data features to obtain the fifth updated features and the sixth updated features.

[0159] Step S605: The first and second update features are concatenated to obtain the first concatenated feature; the third and fourth update features are concatenated to obtain the second concatenated feature; and the fifth and sixth update features are concatenated to obtain the third concatenated feature.

[0160] Step S606: Perform a second splicing process on the first splicing feature, the second splicing feature, and the third splicing feature to obtain the target splicing feature, which serves as the current target fusion data corresponding to the current multimodal data.

[0161] Step S607: Obtain historical target fusion data corresponding to the historical multimodal data of the transformer to be tested, and obtain the anomaly detection result corresponding to the transformer to be tested based on the similarity between the current target fusion data and the historical target fusion data.

[0162] In the aforementioned transformer anomaly detection method, when detecting transformer anomalies, anomaly detection is first performed on the audio data in the current multimodal data. If the audio anomaly detection result is abnormal, a multimodal attention anomaly detection model is further used to detect anomalies in the current multimodal data. This allows for a more accurate identification of potential problems and helps reduce unnecessary computational resource consumption. Moreover, by comprehensively utilizing multimodal information such as audio data, infrared video data, and temperature data, the operating status of the transformer can be more comprehensively reflected. This avoids the shortcomings of relying solely on single data points, which can only reflect a local condition of the transformer and result in low anomaly detection accuracy. Thus, the accuracy of transformer anomaly detection is improved.

[0163] In an exemplary embodiment, to more clearly illustrate the transformer anomaly detection method provided in this application, the following detailed description uses a specific embodiment. In one embodiment, as... Figure 7 As shown, this application also provides a transformer edge-end anomaly detection system based on multimodal attention. When detecting anomalies in a transformer, the system first acquires the audio data, infrared video data, and temperature data of the transformer to be detected as the current multimodal data. Then, anomaly detection processing is performed on the audio data to obtain the corresponding audio anomaly detection result. If the audio anomaly detection result is abnormal, the current multimodal data is input into a pre-trained multimodal attention anomaly detection model to obtain the current target fusion data corresponding to the current multimodal data. Finally, historical target fusion data corresponding to the historical multimodal data of the transformer to be detected is obtained. Based on the similarity between the current target fusion data and the historical target fusion data, the anomaly detection result corresponding to the transformer to be detected is obtained. Specifically, it includes the following:

[0164] 1. A transformer edge-end anomaly detection system based on multimodal attention, characterized in that: Figure 7 As shown, the steps include: For ease of description, the basic mathematical symbols of this patent are explained here first: x∈R t×d Let x represent a real matrix of type t×d; Conv1D(·) represents a one-dimensional convolution operation; ||·|| represents the matrix norm; The expression represents the expectation-maximization algorithm; max(x1, x2, ...) represents finding the maximum value; cos(x1, x2) represents the cosine similarity between vectors x1 and x2; softmax(·) represents the softmax activation function; σ(·) represents the sigmoid activation function; CM β→α (X β ;X α ) represents a single-head cross-modal Transformer attention mechanism with the transit modality being β and the target modality being α.

[0165] S1, Data Acquisition.

[0166] Data from sensors or cameras is collected through edge-side monitoring devices deployed on the transformer. This application utilizes three modalities of data: audio, infrared video, and temperature.

[0167] S2, Audio Anomaly Detection.

[0168] Because audio anomaly detection is highly accurate and sensitive in assessing equipment operating status, and because it is a non-invasive detection method that requires no disassembly or modification of the equipment, it is the preferred method for determining the operating condition of transformers.

[0169] For each segment of transformer operation audio, extract and calculate the short-time energy, short-time zero-crossing rate, short-time autocorrelation coefficient, short-time average amplitude difference function, linear prediction cepstral coefficient spectral characteristics, and Mel cepstral coefficient of the current audio frame. Calculate the cosine similarity between this and the average of the features from historical audio frames. If the similarity is within a certain threshold, the audio is considered normal, the detection ends, and no further action is taken. If the similarity exceeds a certain threshold, the audio detection is considered abnormal, and subsequent steps are performed.

[0170] S3, train the multimodal attention anomaly detection network.

[0171] like Figure 2 As shown, the multimodal attention anomaly detection network first constructs an N+1 tuple, which contains N segments of historical multimodal data in normal states, and a segment of multimodal data to be tested.

[0172] For each segment of audio modal data a, infrared video modal data v, and temperature modal data tpt acquired by the edge computing gateway, features are first extracted to obtain...

[0173] To ensure that each element can access its neighboring elements and thus obtain more global contextual information, the input sequence is passed through a one-dimensional temporal convolutional layer:

[0174]

[0175] Where, k {a，v，tpt} The size of the corresponding modality convolution kernel is d, where d is the common dimension.

[0176] In order to enable the feature sequence to carry time information, Positional embedding (PE) is used to obtain low-level position-aware features of different modalities.

[0177]

[0178] in, This indicates the embedding of each position index.

[0179] Then, using a cross-modal Transformer, information from one modality can potentially be passed to another.

[0180] Cross-modal attention structures between different modal sequences, such as Figure 3 As shown. First, the infrared video modal information is transferred to the audio modal, defining... in It is a learnable weight matrix. Therefore, the potential adaptive performance of transferring infrared video modal information to audio modal information can be used. express:

[0181]

[0182] At this time, Y a With Q a (i.e. T) a They have the same length, and also indicate that in V v In the feature space. Specifically, the equation computes the attention weight matrix using the scaling softmax(·) function. The (i, j)th term in the matrix represents the attention weight of the i-th time step of the audio modality to the j-th time step of the infrared video modality, which can be understood as the degree of importance or attention.

[0183] Therefore, Y a The i-th time step is V v The weighted sum is determined by the i-th row of the weight matrix.

[0184] After a CMβ→α (X β ;X α This operation is called a single-head cross-modal attention.

[0185] Cross-modal Transformer structure such as Figure 4 As shown, it consists of a stack of D layers of crossmodal attention blocks. When the number of layers D and the number of heads in the multi-head attention are divisible, each modality receives associated external modal information from the crossmodal attention blocks to continuously update its sequence. When using infrared video modal information to update the audio modal sequence, the crossmodal attention update process for each layer is as follows:

[0186]

[0187] Similarly, the remaining temperature mode information is also gradually transferred to the audio mode to obtain...

[0188] Similarly, cross-modal information update operations are also performed for infrared video mode and temperature mode.

[0189] Finally, the outputs of cross-modal Transformers sharing the same target modality are concatenated to generate Right now:

[0190]

[0191] The three modality-updated embedding vectors are concatenated to form the embedding code for the historical multimodal data:

[0192] emb = concat[Z a Z v Z tpt Equation (8)

[0193] To ensure that the multimodal attention anomaly detection network extracts the most common features from the samples, a loss function is needed to constrain the similarity between the obtained embeddings; that is, the smaller the distance between normal samples, the better.

[0194]

[0195] Where N is the number of embeddings, ||·|| represents the matrix norm, and margin is a hyperparameter representing the similarity threshold. This loss function encourages cosine similarity between similar embedding pairs to be greater than the margin.

[0196] S4: Multimodal signal detection.

[0197] During the detection phase, the vector average of the embedding codes of all historical multimodal data is taken to obtain the aggregated embedding code:

[0198]

[0199] The cosine similarity cos(c, emb′) between the aggregated embedding codes of the verification multimodal data and historical multimodal data is calculated, activated by the sigmoid function σ(x), and compared with the threshold score. If the result is greater than or equal to the threshold, it is judged as normal; otherwise, it is judged as abnormal.

[0200]

[0201] S5: Alarm response.

[0202] If the multimodal attention anomaly detection network detects an anomaly, the edge computing gateway immediately issues an alarm signal and uploads relevant monitoring data. This helps relevant personnel take timely measures to deal with the anomaly, such as urgently shutting down equipment.

[0203] 2. A transformer edge-end anomaly detection system based on multimodal attention, characterized in that: anomaly detection is performed at the edge end to save network transmission bandwidth.

[0204] 3. A transformer edge-end anomaly detection system based on multimodal attention, characterized in that: multimodal data is used for anomaly detection in steps 3 and 4.

[0205] In the above embodiments, when performing anomaly detection on transformers, anomaly detection is first performed on the audio data in the current multimodal data. If the audio anomaly detection result is abnormal, a multimodal attention anomaly detection model is further used to perform anomaly detection on the current multimodal data. This allows for a more accurate identification of potential problems and helps reduce unnecessary computational resource consumption. Moreover, by comprehensively utilizing multimodal information such as audio data, infrared video data, and temperature data, the operating status of the transformer can be more comprehensively reflected, avoiding the shortcomings of relying solely on single data points, which can only reflect a local situation of the transformer and result in low anomaly detection accuracy. This improves the anomaly detection accuracy of the transformer. Simultaneously, the method of combining multiple data sources makes maintenance management more data-driven, improving the accuracy and efficiency of decision-making. By integrating information from multimodal data sources and the computing power on edge devices, the system can perform anomaly detection in real time at the edge, avoiding data transmission delays and reducing the pressure on data storage and analysis in upper-layer systems. Overall, this application has the advantages of high accuracy, low latency, and privacy protection, enabling timely detection of anomalies and the implementation of corresponding measures, providing strong support for the operation and maintenance of transformer equipment.

[0206] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0207] Based on the same inventive concept, this application also provides a transformer anomaly detection device for implementing the transformer anomaly detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the transformer anomaly detection device provided below can be found in the limitations of the transformer anomaly detection method described above, and will not be repeated here.

[0208] In one exemplary embodiment, such as Figure 8 As shown, a transformer anomaly detection device is provided, comprising: a data acquisition module 801, an audio detection module 802, a data fusion module 803, and a result determination module 804, wherein:

[0209] The data acquisition module 801 is used to acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data and temperature data.

[0210] The audio detection module 802 is used to perform anomaly detection processing on audio data and obtain the audio anomaly detection results corresponding to the audio data.

[0211] The data fusion module 803 is used to input the current multimodal data into a pre-trained multimodal attention anomaly detection model when the audio anomaly detection result is abnormal, so as to obtain the current target fused data corresponding to the current multimodal data.

[0212] The result determination module 804 is used to acquire historical target fusion data corresponding to the historical multimodal data of the transformer under test, and to obtain the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data.

[0213] In an exemplary embodiment, the data fusion module 803 is further configured to perform feature extraction processing on audio data, infrared video data, and temperature data using a pre-trained multimodal attention anomaly detection model to obtain audio data features, infrared video data features, and temperature data features; update the audio data features according to the infrared video data features and temperature data features to obtain a first updated feature and a second updated feature; update the infrared video data features according to the audio data features and temperature data features to obtain a third updated feature and a fourth updated feature; update the temperature data features according to the audio data features and infrared video data features to obtain a fifth updated feature and a sixth updated feature; and obtain the current target fusion data corresponding to the current multimodal data based on the first updated feature, second updated feature, third updated feature, fourth updated feature, fifth updated feature, and sixth updated feature.

[0214] In an exemplary embodiment, the data fusion module 803 is further configured to perform concatenation processing on the first updated feature and the second updated feature to obtain a first concatenated feature, concatenate the third updated feature and the fourth updated feature to obtain a second concatenated feature, and concatenate the fifth updated feature and the sixth updated feature to obtain a third concatenated feature; and perform concatenation processing on the first concatenated feature, the second concatenated feature and the third concatenated feature again to obtain a target concatenated feature, which serves as the current target fusion data corresponding to the current multimodal data.

[0215] In an exemplary embodiment, the audio detection module 802 is further configured to extract target audio features from the audio data as current target audio features; and determine the audio anomaly detection result corresponding to the audio data based on the current target audio features and historical target audio features; the historical target audio features are the target audio features of audio data in historical multimodal data.

[0216] In an exemplary embodiment, the transformer anomaly detection device further includes a model training module for acquiring sample multimodal data of a sample transformer; the sample multimodal data includes at least audio data, infrared video data, and temperature data; the sample multimodal data is input into the multimodal attention anomaly detection model to be trained to obtain sample target fusion data corresponding to the sample multimodal data; based on the sample target fusion data, the multimodal attention anomaly detection model to be trained is iteratively trained to obtain a pre-trained multimodal attention anomaly detection model.

[0217] In an exemplary embodiment, the transformer anomaly detection device further includes an information sending module, which is used to generate early warning information for the transformer under test based on the current multimodal data when the anomaly detection result corresponding to the transformer under test is abnormal; and send the early warning information to the terminal associated with the transformer under test.

[0218] Each module in the aforementioned transformer anomaly detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0219] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores current multimodal data, historical multimodal data, etc. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a transformer anomaly detection method.

[0220] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0221] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0222] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0223] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0224] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0225] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0226] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting transformer anomalies, characterized in that, The method includes: Acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data, and temperature data; Extract the target audio features from the audio data as the current target audio features; determine the audio anomaly detection result corresponding to the audio data based on the current target audio features and historical target audio features; the historical target audio features are the target audio features of audio data in historical multimodal data; When the audio anomaly detection result is abnormal, a pre-trained multimodal attention anomaly detection model is used to extract features from the audio data, infrared video data, and temperature data to obtain audio data features, infrared video data features, and temperature data features. Based on the infrared video data features and the temperature data features, the audio data features are updated to obtain a first updated feature and a second updated feature. Similarly, based on the audio data features and the temperature data features, the infrared video data features are updated to obtain a third updated feature and a fourth updated feature. Finally, based on the audio data features and the infrared video data features, the temperature data features are updated to obtain a fifth updated feature and a sixth updated feature. Based on the first updated feature, the second updated feature, the third updated feature, the fourth updated feature, the fifth updated feature, and the sixth updated feature, the current target fusion data corresponding to the current multimodal data is obtained. Obtain historical target fusion data corresponding to the historical multimodal data of the transformer under test, and obtain the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data; The pre-trained multimodal attention anomaly detection model is trained in the following manner: Acquire sample multimodal data of the sample transformer; the sample multimodal data includes at least audio data, infrared video data, and temperature data; The sample multimodal data is input into the multimodal attention anomaly detection model to be trained to obtain sample target fusion data corresponding to the sample multimodal data; Based on the sample target fusion data, the multimodal attention anomaly detection model to be trained is iteratively trained to obtain the pre-trained multimodal attention anomaly detection model.

2. The method according to claim 1, characterized in that, The step of obtaining the current target fusion data corresponding to the current multimodal data based on the first update feature, the second update feature, the third update feature, the fourth update feature, the fifth update feature, and the sixth update feature includes: The first update feature and the second update feature are concatenated to obtain the first concatenated feature; the third update feature and the fourth update feature are concatenated to obtain the second concatenated feature; and the fifth update feature and the sixth update feature are concatenated to obtain the third concatenated feature. The first splicing feature, the second splicing feature, and the third splicing feature are spliced ​​again to obtain the target splicing feature, which serves as the current target fusion data corresponding to the current multimodal data.

3. The method according to any one of claims 1 to 2, characterized in that, After acquiring historical target fusion data corresponding to the historical multimodal data of the transformer under test, and obtaining the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data, the method further includes: If the abnormal detection result corresponding to the transformer under test is abnormal, an early warning message is generated for the transformer under test based on the current multimodal data. The warning information is sent to the terminal associated with the transformer to be tested.

4. The method according to claim 1, characterized in that, The audio data includes short-time energy, short-time zero-crossing rate, short-time autocorrelation coefficient, short-time average amplitude difference function, linear prediction cepstral coefficient spectral characteristics, and Mel cepstral coefficients.

5. A transformer anomaly detection device, characterized in that, The device includes: The data acquisition module is used to acquire the current multimodal data of the transformer under test; the current multimodal data includes at least audio data, infrared video data and temperature data; An audio detection module is used to extract target audio features from the audio data as current target audio features; and to determine the audio anomaly detection result corresponding to the audio data based on the current target audio features and historical target audio features; the historical target audio features are the target audio features of audio data in historical multimodal data. The data fusion module is used to, when the audio anomaly detection result is abnormal, perform feature extraction processing on the audio data, the infrared video data, and the temperature data using a pre-trained multimodal attention anomaly detection model to obtain audio data features, infrared video data features, and temperature data features; update the audio data features according to the infrared video data features and the temperature data features to obtain a first updated feature and a second updated feature; update the infrared video data features according to the audio data features and the temperature data features to obtain a third updated feature and a fourth updated feature; update the temperature data features according to the audio data features and the infrared video data features to obtain a fifth updated feature and a sixth updated feature; and obtain the current target fused data corresponding to the current multimodal data based on the first updated feature, the second updated feature, the third updated feature, the fourth updated feature, the fifth updated feature, and the sixth updated feature. The result determination module is used to acquire historical target fusion data corresponding to the historical multimodal data of the transformer under test, and to obtain the anomaly detection result corresponding to the transformer under test based on the similarity between the current target fusion data and the historical target fusion data. The device further includes a model training module for acquiring sample multimodal data of the sample transformer; the sample multimodal data includes at least audio data, infrared video data, and temperature data; the sample multimodal data is input into the multimodal attention anomaly detection model to be trained to obtain sample target fusion data corresponding to the sample multimodal data; the multimodal attention anomaly detection model to be trained is iteratively trained based on the sample target fusion data to obtain the pre-trained multimodal attention anomaly detection model.

6. The apparatus according to claim 5, characterized in that, The data fusion module is further configured to concatenate the first updated feature and the second updated feature to obtain a first concatenated feature, concatenate the third updated feature and the fourth updated feature to obtain a second concatenated feature, and concatenate the fifth updated feature and the sixth updated feature to obtain a third concatenated feature; and concatenate the first concatenated feature, the second concatenated feature, and the third concatenated feature again to obtain a target concatenated feature, which serves as the current target fused data corresponding to the current multimodal data.

7. The apparatus according to any one of claims 5 to 6, characterized in that, The device further includes an information sending module, which is used to generate a warning message for the transformer under test based on the current multimodal data when the abnormal detection result corresponding to the transformer under test is abnormal; and to send the warning message to the terminal associated with the transformer under test.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images