Hydroelectric generator abnormal sample set construction method and device, equipment and medium

Abstract

This disclosure relates to a method, apparatus, equipment, and medium for constructing an anomaly sample set for hydropower units. The method includes: constructing a multimodal sample set based on acquired multimodal raw data of the hydropower unit; constructing a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set; and then constructing corresponding multimodal feature extractors and multimodal anomaly event discriminators. This achieves fusion analysis and anomaly identification of multimodal data, avoiding misjudgments and omissions caused by insufficient information in single-modal data. Through comparative analysis of healthy samples and samples to be evaluated, abnormal periods can be automatically and efficiently identified from massive amounts of multimodal data, and an anomaly sample set can be constructed, providing high-quality data support for subsequent fault diagnosis expert systems.

Description

Technical Field

[0001] This disclosure relates to the field of hydropower unit fault prediction and health management technology, and in particular to a method, apparatus, equipment and medium for constructing an abnormal sample set of hydropower units. Background Technology

[0002] With the continuous improvement of the intelligence level of hydropower stations, the computer monitoring system, condition monitoring system, video monitoring system, voiceprint monitoring system, and operation and maintenance management system configured in the station have accumulated massive amounts of multimodal data, including time series data, image data, text data, and voiceprint data. This data contains important information reflecting the operating status of the units, especially samples of abnormal operating conditions, which are of great value for building expert systems for fault diagnosis and condition assessment.

[0003] However, the current situation regarding the lack of anomalous samples in the hydropower field remains severe. In existing technologies, anomalous samples mainly originate from single-modal data (such as time-series data), and the sample source relies entirely on manual annotation, leading to the following problems: The severe lack of anomalous samples makes it difficult to support the development and training of expert systems such as fault diagnosis modules; anomalous samples generated using limited anomalous samples, combined with adversarial generative networks and data augmentation methods, often follow a specific distribution and are prone to mode collapse, resulting in excessively high sample similarity and making it difficult to construct high-quality anomalous sample sets; screening anomalous samples from single-modal data often results in low detection rates and high false positive rates, and the constructed anomalous sample sets lack sufficient information abundance to meet practical application needs.

[0004] Therefore, how to make full use of the multimodal data accumulated by hydropower stations to efficiently and accurately construct anomaly sample sets is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the aforementioned technical problems, this disclosure provides a method, apparatus, equipment, and medium for constructing anomaly sample sets for hydropower units.

[0006] Firstly, this disclosure provides a method for constructing an anomaly sample set for hydropower units, including: A multimodal sample set is constructed based on the acquired multimodal raw data of the hydropower unit, and a healthy multimodal sample set and a multimodal sample set to be evaluated are constructed based on the multimodal sample set; For each modality of data in the multimodal sample set, feature similarity is calculated, and a corresponding multimodal feature extractor is constructed. The multimodal feature extractor performs first-modal similarity calculation on the healthy multimodal sample set, and constructs a corresponding multimodal abnormal event discriminator based on the obtained first-modal similarity. The multimodal feature extractor calculates the intermodal similarity of the multimodal sample set to be evaluated, and the multimodal anomaly discriminator judges the anomaly of the obtained intermodal similarity to obtain an abnormal sample set.

[0007] Secondly, this disclosure provides an apparatus for constructing anomaly sample sets for hydropower units, comprising: The sample construction module is used to construct a multimodal sample set based on the acquired multimodal raw data of the hydropower unit, and to construct a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set; The first calculation module is used to calculate the feature similarity of each modality data in the multimodal sample set and construct the corresponding multimodal feature extractor. The second calculation module is used to calculate the first intermodal similarity of the healthy multimodal sample set through the multimodal feature extractor, and to construct a corresponding multimodal abnormal event discriminator based on the obtained first intermodal similarity. The third calculation module is used to calculate the inter-modal similarity of the multimodal sample set to be evaluated through the multimodal feature extractor, and to judge the anomaly of the obtained inter-modal similarity by combining the multimodal anomaly event discriminator, so as to obtain an abnormal sample set.

[0008] Thirdly, this disclosure provides a device for constructing anomaly sample sets for hydropower units, comprising: processor; Memory, used to store executable instructions; The processor is used to read executable instructions from memory and execute the executable instructions to implement the hydropower unit anomaly sample set construction method of the first aspect.

[0009] Fourthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the hydropower unit anomaly sample set construction method of the first aspect.

[0010] The technical solution provided in this disclosure has the following advantages compared with the prior art: The method for constructing an abnormal sample set for hydropower units according to this embodiment can construct a multimodal sample set based on the acquired multimodal raw data of the hydropower units, and construct a healthy multimodal sample set and an evaluation multimodal sample set based on the multimodal sample set. Then, feature similarity calculation is performed on the modal data of each modality in the multimodal sample set to construct a corresponding multimodal feature extractor. Then, the first intermodal similarity calculation is performed on the healthy multimodal sample set by the multimodal feature extractor, and a corresponding multimodal abnormal event discriminator is constructed based on the obtained first intermodal similarity. Finally, the second intermodal similarity calculation is performed on the evaluation multimodal sample set by the multimodal feature extractor, and the second intermodal similarity is judged by the multimodal abnormal event discriminator to obtain an abnormal sample set. Therefore, by constructing a multimodal feature extractor and a multimodal abnormal event discriminator, the fusion analysis and anomaly detection of multimodal data are realized, avoiding the problems of misjudgment and missed judgment caused by insufficient information in single-modal data. Through comparative analysis of healthy samples and samples to be evaluated, abnormal time periods can be automatically and efficiently identified from massive multimodal data and an abnormal sample set can be constructed, providing high-quality data support for subsequent fault diagnosis expert systems. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 A flowchart illustrating a method for constructing an anomaly sample set for a hydropower unit, as provided in this embodiment of the disclosure; Figure 2 A flowchart illustrating another method for constructing anomaly sample sets for hydropower units provided in this embodiment of the disclosure; Figure 3 This is a schematic diagram of a device for constructing anomaly sample sets for hydropower units, provided in an embodiment of this disclosure. Figure 4 This is a schematic diagram of a device for constructing anomaly sample sets for hydropower units, provided in an embodiment of this disclosure. Detailed Implementation

[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0014] It should be understood that the various steps described in the method implementation of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method implementation may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0015] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] To address the aforementioned problems, this disclosure provides a method, apparatus, equipment, and medium for constructing anomaly sample sets for hydropower units. The following is a detailed explanation... Figure 1-2 The method for constructing anomaly sample sets for hydropower units provided in this disclosure will be described in detail.

[0020] Figure 1 A flowchart illustrating a method for constructing an anomaly sample set for a hydropower unit, as provided in an embodiment of this disclosure, is shown.

[0021] In this embodiment of the disclosure, the method for constructing the abnormal sample set of the hydropower unit can be executed by an electronic device. This electronic device may include, but is not limited to, devices such as computer equipment, cloud servers, or cloud server clusters.

[0022] like Figure 1 As shown, the method for constructing the abnormal sample set of the hydropower unit may include the following steps.

[0023] S110. Construct a multimodal sample set based on the acquired multimodal raw data of the hydropower unit, and construct a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set.

[0024] In this embodiment of the disclosure, the electronic device can construct a multimodal sample set based on the acquired multimodal raw data of the hydropower unit, and construct a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set.

[0025] Specifically, electronic equipment can acquire multimodal raw data of hydropower units, such as time-series monitoring data of vibration, sway, temperature, and pressure pulsation corresponding to key parts through a condition monitoring system; image monitoring data from video surveillance and industrial television systems; text data from maintenance records, repair logs, and defect work orders; and voiceprint data from microphones and voiceprint inspection systems. Then, a multimodal sample set is constructed based on this multimodal raw data. A healthy multimodal sample set and a multimodal sample set to be evaluated are then constructed based on the multimodal sample set. For example, based on maintenance records and defect logs, the period of good operation, such as the initial operation of key parts, is considered the stable operation stage. Samples from the good operation period in the multimodal sample set are extracted to form the healthy multimodal sample set for that part. The multimodal sample set of the period to be evaluated is used as the multimodal sample set to be evaluated.

[0026] S120. Calculate the feature similarity of each modality data in the multimodal sample set and construct the corresponding multimodal feature extractor.

[0027] In this embodiment of the disclosure, the electronic device can perform feature similarity calculations on the modal data of each modality in the multimodal sample set to construct a corresponding multimodal feature extractor.

[0028] Specifically, after obtaining the multimodal sample set, the electronic device can perform feature similarity calculation on the data of each modality in the multimodal sample set, and then construct a corresponding multimodal feature extractor based on the calculation results to quantify the degree of correlation between modalities.

[0029] S130. The first modal similarity is calculated on the healthy multimodal sample set by the multimodal feature extractor, and a corresponding multimodal abnormal event discriminator is constructed based on the obtained first modal similarity.

[0030] In this embodiment of the disclosure, the electronic device can use the multimodal feature extractor to calculate the first intermodal similarity of the healthy multimodal sample set, and construct a corresponding multimodal abnormal event discriminator based on the obtained first intermodal similarity to describe the normal distribution range of intermodal similarity under healthy conditions.

[0031] Specifically, the electronic device can use the multimodal feature extractor to calculate the first modal similarity of the healthy multimodal sample set to obtain the first modal similarity during the period of good operation, and then construct the corresponding multimodal abnormal event discriminator based on the first modal similarity.

[0032] S140. The multimodal feature extractor calculates the intermodal similarity of the multimodal sample set to be evaluated, and the multimodal abnormal event discriminator judges the obtained intermodal similarity to determine abnormality, thereby obtaining an abnormal sample set.

[0033] In this embodiment of the disclosure, the electronic device can use the multimodal feature extractor to calculate the second modal similarity of the multimodal sample set to be evaluated, and combine the multimodal abnormal event discriminator to judge the abnormality of the obtained second modal similarity to obtain an abnormal sample set.

[0034] Specifically, the electronic device can use the multimodal feature extractor to calculate the second modal similarity of the multimodal sample set to be evaluated, obtain the second modal similarity under the time period to be evaluated, and then make anomaly judgment based on the second modal similarity, that is, judge whether it deviates from the normal range, thereby identifying the abnormal time period and constructing an abnormal sample set.

[0035] Therefore, a multimodal sample set can be constructed based on the acquired multimodal raw data of the hydropower unit, and a healthy multimodal sample set and an evaluation multimodal sample set can be constructed based on the multimodal sample set. Then, feature similarity calculation is performed on the data of each modality in the multimodal sample set to construct a corresponding multimodal feature extractor. Then, the first intermodal similarity calculation is performed on the healthy multimodal sample set through the multimodal feature extractor, and a corresponding multimodal abnormal event discriminator is constructed based on the obtained first intermodal similarity. Finally, the second intermodal similarity calculation is performed on the evaluation multimodal sample set through the multimodal feature extractor, and the abnormality judgment is performed on the obtained second intermodal similarity in combination with the multimodal abnormal event discriminator to obtain an abnormal sample set. Therefore, by constructing a multimodal feature extractor and a multimodal abnormal event discriminator, the fusion analysis and anomaly detection of multimodal data are realized, avoiding the problems of misjudgment and missed judgment caused by insufficient information in single-modal data. Through comparative analysis of healthy samples and samples to be evaluated, abnormal time periods can be automatically and efficiently identified from massive multimodal data and an abnormal sample set can be constructed, providing high-quality data support for subsequent fault diagnosis expert systems.

[0036] Optionally, S110 may specifically include: acquiring multimodal raw data of key parts of the hydropower unit, preprocessing the multimodal raw data to obtain timestamp information of the multimodal raw data; and aggregating data based on a preset basic time unit and the timestamp information to obtain the multimodal sample set.

[0037] In this embodiment of the disclosure, the electronic device can acquire multimodal raw data of key parts of the hydropower unit and preprocess the multimodal raw data to obtain the timestamp information of the multimodal raw data.

[0038] Specifically, the electronic equipment first identifies key components of the hydropower unit to construct the abnormal sample set, such as the generator's upper and lower frames, stator, rotor, upper and lower guide bearings, turbine top cover, guide vanes, bottom ring, water guide bearings, and main shaft. Next, it acquires multimodal raw data of these key components and preprocesses the data. For example, for text data, it retains timestamps, removes abnormal tags and stop words, and performs word segmentation; for time-series data, it retains timestamps and performs noise smoothing, outlier removal, and linear interpolation of missing values; for image data, it removes redundant samples, filters damaged images, standardizes dimensions, and performs format conversion and normalization; for voiceprint data, it performs sampling rate unification, silence segmentation, and environmental background noise reduction. After preprocessing, the timestamp information of the multimodal raw data is retained.

[0039] Furthermore, the electronic device can perform data aggregation based on a preset basic time unit and the timestamp information to obtain the multimodal sample set.

[0040] Specifically, electronic devices can sort the modal sample data in chronological order according to timestamps, and select minutes, hours, or days as the basic time unit based on the timestamp interval and data density of each modal sample. Time series monitoring data, image monitoring data, voiceprint monitoring data, and text data of key parts within a specific time unit are aggregated into multimodal samples of that key part, accumulating to form a multimodal sample set.

[0041] Therefore, by preprocessing the multimodal raw data and aggregating the data based on timestamps, the alignment problem of multi-source heterogeneous data on the time scale is solved, enabling subsequent feature extraction and similarity calculation to be performed on a unified time unit, thereby improving the accuracy and comparability of data analysis.

[0042] Optionally, S120 may specifically include: using a pre-trained single-modal feature extractor to extract features from each modality of the multimodal sample set to obtain features of each modality; converting each modality feature into a feature vector in a unified format; calculating the similarity between any two modality data based on the feature vector, and constructing the multimodal feature extractor.

[0043] In this embodiment of the disclosure, the electronic device can use a pre-trained single-modal feature extractor to extract features from each modality of the multimodal sample set to obtain the features of each modality.

[0044] Optionally, the pre-trained unimodal feature extractor includes: a first pre-trained network for time series data, a second pre-trained network for image data, a third pre-trained network for text data, and a fourth pre-trained network for voiceprint data.

[0045] Specifically, electronic devices can utilize pre-trained unimodal feature extractors to extract features from each modality of the multimodal sample set. For example, time series data can use a pre-trained Informer model (first pre-trained network), image data can use a pre-trained Visual Transformer (ViT) model (second pre-trained network), text data can use a pre-trained lightweight BERT (ALBERT) model (third pre-trained network), and voiceprint data can use a pre-trained lightweight audio classification network (YAMNet) model (fourth pre-trained network). Electronic devices can then extract features from each modality of the data using the corresponding unimodal feature extractors to obtain the features of each modality.

[0046] Furthermore, electronic devices can convert the features of various modal data into feature vectors in a unified format.

[0047] Specifically, after obtaining the features of each modality, the electronic device can use a flattening layer to convert each modality feature into a one-dimensional feature vector, thereby obtaining one-dimensional feature vectors corresponding to time series, image, text, and voiceprint data respectively, i.e., feature vectors in a unified format.

[0048] Furthermore, the electronic device can calculate the similarity between any two modal data based on the feature vector and construct the multimodal feature extractor.

[0049] Specifically, the electronic device can calculate the similarity between any two modal data based on the feature vector. For example, it can use the DTW algorithm to calculate the similarity between time series and image, time series and text, time series and voiceprint, image and text, image and voiceprint, and text and voiceprint, respectively, to obtain the similarity value for time period t, thereby constructing the multimodal feature extractor. The corresponding similarity value formula is: .

[0050] Therefore, by employing a pre-trained single-modal feature extractor, the feature extraction capabilities of existing large-scale pre-trained models can be fully utilized to efficiently extract deep features from data of various modalities. By converting features of different modalities into a unified format and calculating similarity, effective alignment and fusion of cross-modal data are achieved, laying a solid foundation for subsequent anomaly detection.

[0051] Optionally, S130 may specifically include: calculating the first intermodal similarity of the healthy multimodal sample set through the multimodal feature extractor to obtain the first intermodal similarity under the first runtime; generating a probability density function of any intermodal similarity based on the kernel density estimation algorithm for the first intermodal similarity; and calculating the health similarity distribution between each modality according to the probability density function, which serves as the multimodal abnormal event discriminator.

[0052] In this embodiment of the disclosure, the electronic device can use the multimodal feature extractor to perform a first modal similarity calculation on the healthy multimodal sample set to obtain the first modal similarity under the first runtime.

[0053] Specifically, the electronic device can input a healthy multimodal sample set into a multimodal feature extractor to calculate the intermodal similarity of the first mode, thereby obtaining the intermodal similarity of the first mode under the first running period, which is the period of good operation. Similarity between first modes at different time periods .

[0054] Furthermore, the electronic device can generate a probability density function for any intermodal similarity based on a kernel density estimation algorithm, targeting the first intermodal similarity. Then, based on the probability density function, the health similarity distribution between each modality is calculated, serving as the multimodal abnormal event discriminator. For example, for the time series-image modal combination, 720 similarity values ​​are obtained: s1,2 = [0.32, 0.35, 0.33, …, 0.41]. Using a kernel density estimation algorithm, a Gaussian kernel is selected, and the optimal bandwidth is calculated to be 0.028, generating a probability density function f1,2(h). Based on f1,2(h), the cumulative distribution function F1,2(h) is calculated to obtain the health similarity distribution of this modal combination. Similarly, the health similarity distributions of the other 5 modal combinations are calculated sequentially to obtain F1,2(h). This constitutes a multimodal abnormal event discriminator.

[0055] Therefore, based on multimodal samples under healthy operating conditions of the unit, the health similarity distribution between modes is constructed through kernel density estimation algorithm. Without assuming the data distribution form, it can accurately describe the statistical characteristics of the similarity between modes under healthy conditions, and provide a reliable benchmark for anomaly detection.

[0056] Optionally, S140 may specifically include: calculating the second intermodal similarity of the multimodal sample set to be evaluated through the multimodal feature extractor to obtain the second intermodal similarity under the second runtime; and making anomaly judgment by comparing the second intermodal similarity with the health similarity distribution of each modality in the multimodal abnormal event discriminator to obtain the abnormal sample set.

[0057] In this embodiment of the disclosure, the electronic device can use the multimodal feature extractor to calculate the second intermodal similarity of the multimodal sample set to be evaluated, and obtain the second intermodal similarity under the second runtime.

[0058] Specifically, the electronic device can input the multimodal sample set to be evaluated into the multimodal feature extractor to calculate the intermodal similarity, thereby obtaining the data similarity of each modality under the time period w. That is, the similarity between the second modes under the second runtime.

[0059] Furthermore, the electronic device can make anomaly judgments by comparing the second intermodal similarity with the health similarity distribution among the modalities in the multimodal anomaly event discriminator, thereby obtaining the abnormal sample set.

[0060] Specifically, after obtaining the second intermodal similarity, the electronic device can perform anomaly judgment by comparing the second intermodal similarity with the intermodal health similarity distribution in the multimodal anomaly event discriminator, such as determining whether the similarity falls within the corresponding modal health similarity distribution F. of quantiles 0.9987 quantile Within the range, if si, j(w) If [Q0.0013(Fi,j), Q0.9987(Fi,j)] is used, then the similarity between modalities i and j is determined to be abnormal, and an abnormal sample set is obtained.

[0061] Therefore, by comparing the intermodal similarity of the time period to be evaluated with the similarity distribution under healthy conditions, abnormal time periods that deviate from the normal range can be effectively identified, thus achieving automated screening of abnormal samples.

[0062] Optionally, the abnormal sample set is obtained by performing anomaly judgment on the second intermodal similarity and the health similarity distribution between each modality in the multimodal abnormal event discriminator: performing anomaly judgment on the second intermodal similarity and the health similarity distribution between each modality in the multimodal abnormal event discriminator, and counting the number of intermodal combinations with abnormal similarity in the second running segment; and judging the multimodal sample set of the second running segment as an abnormal sample set according to the relationship between the number and a preset anomaly judgment threshold.

[0063] In this embodiment of the disclosure, the electronic device can make anomaly judgments by comparing the second intermodal similarity with the health similarity distribution among the modalities in the multimodal anomaly event discriminator, and count the number of intermodal combinations with abnormal similarity during the second runtime period. Based on the relationship between the number and a preset anomaly judgment threshold, the multimodal sample set of the second runtime period is determined to be an abnormal sample set.

[0064] Specifically, after the judgment, time periods with 2 to 4 similarity anomalies are identified as suspected abnormal time periods, and the multimodal sample set of these time periods is added to the suspected abnormal sample set; time periods with more than 4 similarity anomalies are identified as abnormal time periods, and the multimodal sample set of these time periods is added to the abnormal sample set.

[0065] Therefore, by counting the number of combinations between abnormal modes and combining them with a preset threshold for anomaly determination, the misjudgment that may be caused by abnormal similarity between single modes is avoided, thereby improving the accuracy and robustness of abnormal sample identification.

[0066] Figure 2 A flowchart illustrating another method for constructing anomaly sample sets for hydropower units provided in this embodiment is shown.

[0067] like Figure 2 As shown, electronic equipment can identify key components of the hydropower unit in the constructed abnormal sample set, such as the generator upper frame, lower frame, stator, rotor, upper guide bearing, lower guide bearing, turbine top cover, guide vanes, bottom ring, water guide bearing, and main shaft. Time-series monitoring data reflecting the operating status, such as vibration, sway, temperature, and pressure pulsation, corresponding to each key component are selected from the condition monitoring system; image monitoring data from video surveillance and industrial television; text data from maintenance records, repair logs, and defect work orders; and voiceprint data accumulated from microphone and voiceprint inspection systems. Based on the above data characteristics, a sample cleaning method is selected to clean each modal sample. The modal data are sorted chronologically to determine the basic time unit for constructing the abnormal sample set, thus constructing a multimodal sample set MS covering the corresponding time-series monitoring data, image monitoring data, voiceprint monitoring data, and text data for key components.

[0068] For example, multimodal sample cleaning methods are as follows: For text data, text cleaning is carried out while retaining the timestamps of the text data. This mainly includes removing abnormal tags (special characters, HTML tags, etc.), removing stop words, and word segmentation. For time series data, time series data cleaning is completed based on noise smoothing, outlier removal, and linear interpolation of missing values, while retaining the timestamps of the time series data. Image data cleaning is completed using strategies such as redundant sample removal, damaged image screening, size unification, format conversion and normalization, while retaining the timestamps of the image data. For voiceprint data, preprocessing methods such as unified sampling rate, silence segmentation, and environmental background noise reduction are selected, while retaining the timestamps of the voiceprint data.

[0069] The sample data for each modality are sorted chronologically according to their timestamps. Based on the timestamp intervals and data density of each modality, time scales such as minutes, hours, and days can be selected as the basic time units. After selecting a basic time unit, the time-series monitoring data, image monitoring data, voiceprint monitoring data, and text data of key parts within that specific time unit are recorded as multimodal samples of that key part. Accumulate multimodal samples over a specific time period to form a multimodal sample set. .

[0070] Furthermore, electronic equipment can be oriented towards a multimodal sample set. Based on maintenance records, defect ledgers, etc., the period of good operation, such as the initial stage of operation of key parts, can be taken as the stable operation stage. Samples from the period of good operation in the multimodal sample set can be extracted to form a healthy multimodal sample set for that part.

[0071] Furthermore, electronic devices can construct single-modal feature extractors based on pre-trained Informer, pre-trained Visual Transformer (ViT), pre-trained lightweight BERT (ALBERT), and pre-trained lightweight audio classification network (YAMNet) models for time-series monitoring data, image monitoring data, text data, and voiceprint data, respectively. Then, a flattening layer is used to convert the features extracted by each modality feature extractor into one-dimensional features. Dynamic Time Warping (DTW) is used to measure the similarity between the features of each modality to obtain the time period. Similarity of modal data under the following conditions Based on this, a multimodal feature extractor is constructed.

[0072] For example, lightweight pre-trained models for each modality extract single-modal features. After obtaining the single-modal features, a flattening layer is used to convert the features extracted by each modality feature extractor into one-dimensional features. (These correspond to one-dimensional features of time-series monitoring data, image monitoring data, text data, and voiceprint data, respectively), and then the DTW algorithm is used to calculate the pairwise similarity between the features of each modality. calculate: For the t-th time period, the length is The Modal features With length The Modal features ,calculate and The cumulative distance matrix, in which the first... element express and The Euclidean distance between them is calculated by... arrive The minimum cumulative sum as and similarity : , .

[0073] In the formula The calculation is as follows: .

[0074] Furthermore, the electronic device feeds the health multimodal sample set data into the multimodal feature extractor, resulting in a well-functioning operating period. Modal data similarity at different time periods Construct a modal health similarity distribution F based on kernel density estimation. It serves as a multimodal abnormal event discriminator.

[0075] For example, for a well-running segment Modalities at different time periods Data similarity between The kernel function is a Gaussian kernel, and the optimal bandwidth for kernel density estimation is determined using the following expression. : .

[0076] In the formula express The standard deviation.

[0077] according to generate exist probability density function at time : .

[0078] according to Calculate the health similarity distribution ,according to 0.0013 quantile 0.9987 quantile .

[0079] Furthermore, construct the period to be evaluated. The multimodal sample set is fed into the multimodal feature extractor to obtain the time period. Similarity of modal data under the following conditions Combined with a multimodal abnormal event discriminator, it is determined whether the similarity falls within the corresponding modality health similarity distribution F. of quantiles 0.9987 quantile Within the range, if Then the mode is identified. For periods with 2 to 4 anomalous similarities, define them as suspected anomalous periods and add the multimodal sample sets of these periods to the suspected anomalous sample set. For periods with more than 4 anomalous similarities, define them as anomalous periods and add the multimodal sample sets of these periods to the anomalous sample set. Repeat the above process in S5 to complete the discrimination of multimodal sample sets for all periods to be evaluated.

[0080] Figure 3 A schematic diagram of a device for constructing anomaly sample sets for hydropower units provided in an embodiment of this disclosure is shown.

[0081] like Figure 3 As shown, the hydropower unit abnormal sample set construction device 300 may include a sample construction module 310, a first calculation module 320, a second calculation module 330 and a third calculation module 340.

[0082] The sample construction module 310 can be used to construct a multimodal sample set based on the acquired multimodal raw data of the hydropower unit, and to construct a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set.

[0083] The first calculation module 320 can be used to calculate the feature similarity of each modality data in the multimodal sample set and construct the corresponding multimodal feature extractor.

[0084] The second calculation module 330 can be used to calculate the first intermodal similarity of the healthy multimodal sample set through the multimodal feature extractor, and construct a corresponding multimodal abnormal event discriminator based on the obtained first intermodal similarity.

[0085] The third calculation module 340 can be used to calculate the inter-modal similarity of the multimodal sample set to be evaluated through the multimodal feature extractor, and to judge the anomaly of the obtained inter-modal similarity by combining the multimodal anomaly event discriminator, so as to obtain an abnormal sample set.

[0086] Therefore, in this embodiment of the disclosure, a multimodal sample set can be constructed based on the acquired multimodal raw data of the hydropower unit, and a healthy multimodal sample set and a multimodal sample set to be evaluated can be constructed based on the multimodal sample set. Then, feature similarity calculation is performed on the modal data of each modality in the multimodal sample set to construct a corresponding multimodal feature extractor. Then, the first intermodal similarity calculation is performed on the healthy multimodal sample set through the multimodal feature extractor, and a corresponding multimodal abnormal event discriminator is constructed based on the obtained first intermodal similarity. Finally, the second intermodal similarity calculation is performed on the multimodal sample set to be evaluated through the multimodal feature extractor, and the abnormality judgment is performed on the obtained second intermodal similarity in combination with the multimodal abnormal event discriminator to obtain an abnormal sample set. Therefore, by constructing a multimodal feature extractor and a multimodal abnormal event discriminator, the fusion analysis and anomaly detection of multimodal data are realized, avoiding the problems of misjudgment and missed judgment caused by insufficient information in single-modal data. Through comparative analysis of healthy samples and samples to be evaluated, abnormal time periods can be automatically and efficiently identified from massive multimodal data and an abnormal sample set can be constructed, providing high-quality data support for subsequent fault diagnosis expert systems.

[0087] In some embodiments of this disclosure, the sample construction module 310 includes: The first processing unit is used to acquire multimodal raw data of key parts of the hydropower unit and preprocess the multimodal raw data to obtain the timestamp information of the multimodal raw data. The second processing unit is used to aggregate data based on a preset basic time unit and the timestamp information to obtain the multimodal sample set.

[0088] In some embodiments of this disclosure, the first computing module 320 includes: The feature extraction unit is used to extract features from each modality data in the multimodal sample set using a pre-trained single-modal feature extractor to obtain the features of each modality data. The format conversion unit is used to convert the features of various modal data into feature vectors in a unified format. The third processing unit is used to calculate the similarity between any two modal data based on the feature vector, and to construct the multimodal feature extractor.

[0089] In some embodiments of this disclosure, the pre-trained unimodal feature extractor includes: a first pre-trained network for time series data, a second pre-trained network for image data, a third pre-trained network for text data, and a fourth pre-trained network for voiceprint data.

[0090] In some embodiments of this disclosure, the second computing module 330 includes: The fourth processing unit is used to perform first modal similarity calculation on the healthy multimodal sample set through the multimodal feature extractor to obtain the first modal similarity under the first running period; The fifth processing unit is used to generate a probability density function for the inter-modal similarity based on the kernel density estimation algorithm for the first inter-modal similarity. The sixth processing unit is used to calculate the health similarity distribution among modalities based on the probability density function, and to serve as the multimodal abnormal event discriminator.

[0091] In some embodiments of this disclosure, the third computing module 340 includes: The seventh processing unit is used to calculate the inter-modal similarity of the multimodal sample set to be evaluated through the multimodal feature extractor, so as to obtain the inter-modal similarity of the second running segment. The eighth processing unit is used to perform anomaly judgment on the second intermodal similarity and the health similarity distribution between each modality in the multimodal anomaly event discriminator, and obtain the abnormal sample set.

[0092] In some embodiments of this disclosure, the eighth processing unit includes: An anomaly detection subunit is used to perform anomaly detection by comparing the second intermodal similarity with the health similarity distribution of each modality in the multimodal anomaly event discriminator, and to count the number of intermodal combinations with abnormal similarity during the second runtime period. The relationship judgment subunit is used to determine the multimodal sample set of the second runtime segment as an abnormal sample set based on the relationship between the quantity and the preset abnormal judgment threshold.

[0093] It should be noted that, Figure 3 The hydropower unit anomaly sample set construction device 300 shown can perform... Figure 1-2 The various steps in the method embodiment shown are implemented. Figure 1-2 The processes and effects in the method embodiments shown are not described in detail here.

[0094] Figure 4 A schematic diagram of a device for constructing anomaly sample sets for hydropower units, provided in an embodiment of this disclosure, is shown.

[0095] In some embodiments of this disclosure, Figure 4 The device for constructing the abnormal sample set of hydropower units shown can be an electronic device. Specifically, the electronic device can include, but is not limited to, devices such as computer equipment, cloud servers, or cloud server clusters.

[0096] like Figure 4 As shown, the device for constructing the abnormal sample set of the hydropower unit may include a processor 401 and a memory 402 storing computer program instructions.

[0097] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0098] Memory 402 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway device. In a particular embodiment, memory 402 is a non-volatile solid-state memory. In a particular embodiment, memory 402 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0099] The processor 401 reads and executes the computer program instructions stored in the memory 402 to perform the steps of the hydropower unit anomaly sample set construction method provided in this embodiment of the disclosure.

[0100] In one example, the device for constructing anomaly sample sets for hydroelectric generators may further include a transceiver 403 and a bus 404. Wherein, as... Figure 4As shown, the processor 401, memory 402 and transceiver 403 are connected via bus 404 and communicate with each other.

[0101] Bus 404 includes hardware, software, or both. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0102] This disclosure also provides a computer-readable storage medium that can store a computer program. When the computer program is executed by a processor, the processor enables the processor to implement the hydropower unit anomaly sample set construction method provided in this disclosure.

[0103] The aforementioned storage medium may, for example, include a memory 402 containing computer program instructions, which can be executed by the processor 401 of the hydropower unit anomaly sample set construction device to complete the hydropower unit anomaly sample set construction method provided in this embodiment. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage device.

[0104] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0105] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing an abnormal sample set of a hydroelectric generating unit, characterized in that, include: A multimodal sample set is constructed based on the acquired multimodal raw data of the hydropower unit, and a healthy multimodal sample set and a multimodal sample set to be evaluated are constructed based on the multimodal sample set; For each modality of data in the multimodal sample set, feature similarity is calculated, and a corresponding multimodal feature extractor is constructed. The multimodal feature extractor performs first-modal similarity calculation on the healthy multimodal sample set, and constructs a corresponding multimodal abnormal event discriminator based on the obtained first-modal similarity. The multimodal feature extractor calculates the intermodal similarity of the multimodal sample set to be evaluated, and the multimodal anomaly discriminator judges the anomaly of the obtained intermodal similarity to obtain an abnormal sample set.

2. The method of claim 1, wherein, The construction of a multimodal sample set based on the acquired multimodal raw data of the hydropower unit includes: Acquire multimodal raw data of key components of the hydropower unit, and preprocess the multimodal raw data to obtain the timestamp information of the multimodal raw data; The multimodal sample set is obtained by aggregating data based on the preset basic time unit and the timestamp information.

3. The method of claim 1, wherein, The step of calculating feature similarity for each modality data in the multimodal sample set and constructing a corresponding multimodal feature extractor includes: The features of each modality are extracted from the multimodal sample set using a pre-trained single-modal feature extractor. Convert the features of each modality into feature vectors in a unified format; The similarity between any two modal data is calculated based on the feature vector, and the multimodal feature extractor is constructed.

4. The method of claim 3, wherein, The pre-trained unimodal feature extractor includes: a first pre-trained network for time series data, a second pre-trained network for image data, a third pre-trained network for text data, and a fourth pre-trained network for voiceprint data.

5. The method of claim 1, wherein, The step of calculating the first-modal similarity of the healthy multimodal sample set using the multimodal feature extractor, and constructing a corresponding multimodal abnormal event discriminator based on the obtained first-modal similarity, includes: The first modality similarity is calculated by using the multimodal feature extractor on the healthy multimodal sample set to obtain the first modality similarity under the first runtime. For the first inter-modal similarity, a probability density function for the inter-modal similarity is generated based on the kernel density estimation algorithm; Based on the probability density function, the health similarity distribution among each modality is calculated and used as the multimodal abnormal event discriminator.

6. The method of claim 5, wherein, The process involves calculating the inter-modal similarity of the multimodal sample set to be evaluated using the multimodal feature extractor, and then using the multimodal anomaly discriminator to determine anomalies in the obtained inter-modal similarity, resulting in an abnormal sample set, including: The second modal similarity is calculated by the multimodal feature extractor on the multimodal sample set to be evaluated, and the second modal similarity under the second runtime is obtained. The abnormal sample set is obtained by comparing the second intermodal similarity with the health similarity distribution among the modalities in the multimodal abnormal event discriminator.

7. The method of claim 6, wherein, The step of performing anomaly judgment by comparing the second inter-modal similarity with the health similarity distribution among modalities in the multimodal anomaly event discriminator to obtain the anomaly sample set includes: The second intermodal similarity is compared with the health similarity distribution of each modality in the multimodal abnormal event discriminator to determine anomalies, and the number of intermodal combinations with abnormal similarity during the second running period is counted. Based on the relationship between the quantity and the preset anomaly determination threshold, the multimodal sample set of the second runtime segment is determined to be an anomaly sample set.

8. An apparatus for constructing an abnormal sample set of a hydroelectric generating unit, characterized in that, include: The sample construction module is used to construct a multimodal sample set based on the acquired multimodal raw data of the hydropower unit, and to construct a healthy multimodal sample set and a multimodal sample set to be evaluated based on the multimodal sample set; The first calculation module is used to calculate the feature similarity of each modality data in the multimodal sample set and construct the corresponding multimodal feature extractor. The second calculation module is used to calculate the first intermodal similarity of the healthy multimodal sample set through the multimodal feature extractor, and to construct a corresponding multimodal abnormal event discriminator based on the obtained first intermodal similarity. The third calculation module is used to calculate the inter-modal similarity of the multimodal sample set to be evaluated through the multimodal feature extractor, and to judge the anomaly of the obtained inter-modal similarity by combining the multimodal anomaly event discriminator, so as to obtain an abnormal sample set.

9. An apparatus for constructing an abnormal sample set of a hydroelectric generating unit, characterized by, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the hydropower unit abnormal sample set construction method according to any one of claims 1-7.

10. A non-transitory computer readable storage medium, comprising: The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method for constructing anomaly sample sets for hydropower units as described in any one of claims 1-7.

Images