Measuring point time sequence anomaly analysis method and system based on multi-modal large model

Abstract

The invention relates to the technical field of thermal power production, artificial intelligence and time sequence analysis, in particular to a measuring point time sequence anomaly analysis method and system based on a multi-modal large model. Calculating a cross-modal feature similarity weight based on a self-attention mechanism to obtain a fusion feature vector; comparing the fusion feature vector with the normal mode feature representation through a self-supervised learning algorithm to realize anomaly detection, wherein a result comprises anomaly time, a fault image area and a text keyword; and a result is input into a multi-modal large model, cross-modal attention mechanism association data is utilized, fault reasoning and causal relationship analysis are carried out, a visual diagnosis report is generated, and then the accuracy of measuring point screening and time sequence anomaly analysis is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of thermal power production, artificial intelligence, and time series analysis, and particularly to a method and system for analyzing the time series anomalies of measuring points based on a multimodal large model. Background Art

[0002] In the current era of highly developed digitalization and intelligentization, time series anomaly detection plays a crucial role in many fields, such as industrial production, financial transactions, medical health, etc. Accurately and timely detecting anomalies in time series can help enterprises discover equipment failures in advance, prevent financial risks, ensure patient safety, etc., thereby avoiding huge economic losses and potential safety hazards.

[0003] Existing time series anomaly detection methods mainly rely on statistical analysis, machine learning, or deep learning models. Statistical analysis methods quantify and analyze the distribution characteristics, autocorrelation, etc. of time series data, and set thresholds to identify anomaly points, such as traditional statistical models based on moving average, exponential smoothing, etc. Machine learning methods use algorithms such as support vector machines (SVM), decision trees, and random forests to learn the characteristics of normal and abnormal patterns from labeled historical data, and then classify and judge new data. Deep learning methods such as recurrent neural networks (RNN) and their variants (LSTM, GRU), etc., can automatically extract complex features in time series and have achieved good results in anomaly detection tasks.

[0004] However, these methods have obvious limitations. On the one hand, they usually only focus on single-modal time series data and ignore the key information that may be contained in other data sources. In actual application scenarios, in addition to time series data, there are often rich multimodal data such as text, images, and audio. For example, in industrial production, equipment operation logs (text), monitoring videos (images), sounds generated by equipment operation (audio), etc. may all contain important clues related to the equipment operation status. Relying solely on single-modal time series data for anomaly detection may miss this key information, resulting in insufficient accuracy and comprehensiveness of the detection results.

[0005] On the other hand, traditional deep learning methods also expose some problems in anomaly detection. Their generalization ability for abnormal patterns is limited, and it is often difficult to effectively adapt when facing multi-measurement point fault patterns in complex industrial environments. The fault patterns in complex industrial environments are diverse and uncertain, and the data of different measurement points may present different abnormal characteristics. Traditional deep learning methods may not be able to learn these complex and variable abnormal patterns, thus affecting the accuracy and robustness of anomaly detection.

[0006] In recent years, multi-modal large models have made breakthroughs in the fields of natural language processing, computer vision, etc. Multi-modal large models can integrate data of different modalities, fully exploit the correlations and complementary information between modalities, thereby enhancing the performance and performance of the models. For example, in the task of image description generation, multi-modal large models can simultaneously process image and text data to generate text that accurately describes the content of the image. However, in the field of time series anomaly detection, how to effectively integrate multi-modal data, give full play to the advantages of multi-modal large models, and improve the accuracy and robustness of anomaly detection remains an urgent challenge to be solved. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method and system for analyzing the time series anomalies of measurement points based on a multi-modal large model in view of the deficiencies in the above-mentioned prior art, so as to solve the technical problem that the time series model of a single modality affects the accuracy and robustness of anomaly detection in thermal power plants.

[0008] The object of the present invention is achieved by the following technical solutions: In a first aspect, the present invention provides a method for analyzing the time series anomalies of measurement points based on a multi-modal large model, including: Obtain multiple modalities of data corresponding to multiple measurement points related to faulty equipment in a thermal power plant, and the modalities of data include at least text data, time series data, and image data; Extract the modal features in each modality of data respectively, calculate the cross-modal feature similarity weights based on the self-attention mechanism, and obtain the fused feature vector; Based on the self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result; the anomaly detection result includes the occurrence time of the anomaly event, the detected image fault area, and the text fault keyword; Input the anomaly detection result into the multi-modal large model, associate the modal data according to the cross-modal attention mechanism, perform fault reasoning, analyze the anomaly causal relationship, and obtain a visual diagnostic report.

[0009] As a further improvement of the present invention, after obtaining multiple modalities of data corresponding to multiple measurement points related to faulty equipment in a thermal power plant, it further includes a preliminary screening of the measurement points, specifically including: Construct a measurement point correlation matrix based on mutual information; Calculate the Pearson correlation coefficient between the data of each measurement point in the measurement point correlation matrix and the historical fault label, and retain the fault correlation measurement points with a fault correlation higher than the correlation threshold according to the Pearson correlation coefficient; Use the mutual information method to evaluate the redundancy between measurement points and eliminate redundant measurement points; Dynamically allocate weights to the remaining measurement points with high fault correlation, and adjust the weights of the measurement points according to the data fluctuation degree and abnormal trigger frequency.

[0010] As a further improvement of the present invention, it also includes preprocessing the multi-modal data corresponding to the initially screened measurement points; specifically including: Use the set sliding window interpolation method to fill in the missing values for the time series data, remove high-frequency noise through wavelet denoising, and then unify the dimension through Z-score standardization; For the image data, use adaptive histogram equalization to enhance the contrast, use the object detection model to locate the area of the key fault components, and crop and resize them to a unified size; For the text data, use regular expressions to extract the fault-related text information, and use TF-IDF or BERT word embedding to convert the fault-related text information into numerical vectors.

[0011] As a further improvement of the present invention, respectively extract the modal features in each modal data, specifically including: Extract time domain features, frequency domain features, and wavelet entropy features from the time series data to obtain time series features, use a deep convolutional neural network to extract the local graph features related to faults for the image data, and capture semantic features for the text data through a bidirectional recurrent neural network; Based on the self-attention mechanism, calculate the similarity weights between the time series features, local graph features, and semantic features to obtain a cross-modal aligned fusion feature vector.

[0012] As a further improvement of the present invention, based on the self-supervised learning algorithm, perform contrastive learning on the fusion feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result, specifically including: Construct a multi-branch self-supervised network model, and use the data set to train the multi-branch self-supervised network model; Use the trained multi-branch self-supervised network model to process the fusion feature vector corresponding to each measurement point respectively; Learn the feature representations of each modality in the normal mode through the reconstruction loss function, and compare the fusion feature vector with the feature representations of each modality in the normal mode; Calculate the reconstruction error for the corresponding time series features in the fusion feature vector to obtain the occurrence time of the abnormal event; for the corresponding local graph features in the fusion feature vector, obtain the visual fault area through the attention mechanism; count the occurrence frequency of the high-frequency fault keywords for the text data to obtain the text fault keywords.

[0013] As a further improvement of the present invention, the abnormal detection results are input into a multi-modal large model, and the modal data are associated according to the cross-modal attention mechanism for fault reasoning and analysis of abnormal causal relationships, specifically including: Construct a knowledge graph database for thermal power equipment, which includes the structure of thermal power equipment, historical fault cases, and corresponding solutions, and is stored in the form of triples; When the abnormal detection results are input into the multi-modal large model, relevant cases in the knowledge graph database of thermal power equipment are retrieved for fault reasoning to obtain a visual diagnostic report including fault causes, evolution paths, and maintenance suggestions.

[0014] As a further improvement of the present invention, after generating the visual diagnostic report, it also includes feedback optimization of the multi-modal large model, specifically including: Regularly calculate the confidence of the multi-modal large model, perform active learning on multi-modal data samples with confidence lower than the preset threshold, and introduce an expert annotation mechanism to annotate the multi-modal data samples.

[0015] In a second aspect, the present invention provides a multi-modal large model-based measuring point time series abnormal analysis system for implementing the above-mentioned multi-modal large model-based measuring point time series abnormal analysis method, including: A measuring point data acquisition module that acquires various modal data corresponding to multiple measuring points related to faulty equipment in a thermal power plant, and the modal data at least includes text data, time series data, and image data; A multi-modal feature fusion module that respectively extracts modal features from each modal data, calculates cross-modal feature similarity weights based on the self-attention mechanism, and obtains a fused feature vector; A self-supervised fault detection module that performs contrastive learning on the fused feature vector and the feature representation in the normal mode based on the self-supervised learning algorithm to obtain abnormal detection results; the abnormal detection results include the occurrence time of abnormal events, the fault area of the detected image, and text fault keywords; A multi-modal analysis and report generation module that inputs the abnormal detection results into the multi-modal large model, associates the modal data according to the cross-modal attention mechanism, performs fault reasoning, analyzes abnormal causal relationships, and obtains a visual diagnostic report.

[0016] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, where the one or more programs include instructions that, when executed by a computing device, cause the computing device to execute the above-mentioned multi-modal large model-based measuring point time series abnormal analysis method.

[0017] In a fourth aspect, the present invention provides a computing device, including: One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for performing the above-mentioned method for analyzing abnormal time series of measurement points based on a multi-modal large model.

[0018] The beneficial effects of the present invention are as follows: The method for analyzing abnormal time series of measurement points based on a multi-modal large model provided by the present invention can comprehensively cover various information of the operation of thermal power plant equipment by combining text data, time series data, and image data. Data of different modalities can complement each other, providing richer fault clues and improving the robustness of anomaly detection. Even if there is noise or missing data in a certain modality, data of other modalities can still provide effective information, thus ensuring the accuracy of detection. By calculating the cross-modal feature similarity weights based on the self-attention mechanism, features of different modalities can be effectively fused. Through contrastive learning, the differences between the normal mode and the abnormal mode can be learned, thereby improving the accuracy of anomaly detection. By inputting the anomaly detection results into the multi-modal large model and using the cross-modal attention mechanism, data of different modalities can be further associated for fault reasoning. This method can deeply analyze the causal relationship of abnormal events and provide a more comprehensive fault diagnosis.

[0019] Furthermore, by screening measurement points with high correlation and non-redundancy, the amount of data to be processed can be significantly reduced. By calculating the Pearson correlation coefficient between the measurement point data and historical fault labels, measurement points highly correlated with faults can be identified. These measurement points have higher indicative properties for fault detection and can more accurately reflect the state changes of the equipment.

[0020] Furthermore, key information related to faults can be effectively extracted through regular expressions, filtering out irrelevant text data and improving the quality and relevance of the text data. The preprocessing method can effectively remove noise, fill in missing values, enhance image quality, extract key information, etc., improving the data quality. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of the method for analyzing abnormal time series of measurement points based on a multi-modal large model provided by the embodiments of the present invention; Figure 2 It is a diagram of the anomaly detection results provided by the embodiments of the present invention; Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0023] In order to make the objectives and technical solutions of the present invention clearer and easier to understand, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. Among them, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0025] Embodiment 1 As Figure 1 , Figure 2 shown, this embodiment provides a method for analyzing the time-series anomalies of measurement points based on a multimodal large model. It can make full use of multimodal data of different types of measurement points, improve the accuracy of anomaly detection, and perform precise analysis of faults. The specific implementation manners are as follows.

[0026] First, obtain various modal data corresponding to multiple measurement points related to the faulty equipment in the thermal power plant. The modal data at least includes text data, time-series data, and image data.

[0027] The process of obtaining multimodal data specifically includes: real-time collecting the time-series data of the operation of the thermal power plant equipment through sensors, and using visual devices such as industrial cameras to capture the state images of the key components of the equipment.

[0028] After obtaining various modal data corresponding to multiple measurement points related to the faulty equipment in the thermal power plant, it further includes a preliminary screening of the measurement points, specifically including: Construct a measurement point correlation matrix based on mutual information; Calculate the Pearson correlation coefficient between the data of each measurement point in the measurement point correlation matrix and the historical fault label, and retain the fault-correlated measurement points with a fault correlation higher than the correlation threshold according to the Pearson correlation coefficient; Use the mutual information method to evaluate the redundancy between measurement points and eliminate redundant measurement points; Dynamically assign weights to the retained measurement points with higher fault correlations, and adjust the weights of the measurement points according to the data fluctuation degree and anomaly trigger frequency.

[0029] Among them, in this embodiment, the cross-correlation method is used to screen the measurement points strongly correlated with the target fault mode and eliminate redundant measurement points, and the correlation threshold is dynamically set according to the operating state of the thermal power plant equipment and the measurement point position. For example, measurement points with a sensor noise ratio > 30% can be eliminated.

[0030] This embodiment also includes preprocessing the multi-modal data corresponding to the initially screened measurement points. Specifically, it includes: Using a set sliding window interpolation method to fill in the missing values of the time series data, removing high-frequency noise through wavelet denoising, and then unifying the dimension through Z-score standardization; For image data, adaptive histogram equalization is used to enhance the contrast, a target detection model is used to locate the area of the key fault components, and after cropping, it is resized to a unified size; the target detection model in this embodiment uses algorithms such as YOLO or FasterR-CNN.

[0031] For text data, regular expressions are used to extract the text information related to faults, and TF-IDF or BERT word embedding is used to convert the text information related to faults into numerical vectors. The dynamic measurement point screening mechanism avoids the limitations of fixed measurement points, can optimize the monitoring focus in real time with the change of equipment status, and improve the data utilization efficiency.

[0032] Extract the modal features in each modal data respectively, calculate the cross-modal feature similarity weights based on the self-attention mechanism, and obtain the fused feature vector.

[0033] Specifically, for time series data, time domain features, frequency domain features, and wavelet entropy features are extracted to obtain time series features. For image data, a deep convolutional neural network is used to extract the local graph features related to faults. For text data, a bidirectional recurrent neural network is used to capture semantic features; Based on the self-attention mechanism, calculate the similarity weights between the time series features, local graph features, and semantic features to obtain the cross-modal aligned fused feature vector. By reducing the dimension of each modal feature respectively, and then inputting through splicing into the shared encoder; at the same time, retain the independent features of each modality, and integrate the final result through attention weighting to obtain the cross-modal aligned fused feature vector.

[0034] Specifically, the time domain features in this embodiment include mean and variance, and the frequency domain feature is the FFT amplitude.

[0035] The local graph features are extracted in the way of ResNet; while the text data captures the context semantic features through BiLSTM or Transformer.

[0036] Based on the self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result; the anomaly detection result includes the occurrence time of the anomaly event, the detected image fault area, and the text fault keywords.

[0037] Based on the self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result, specifically including: Specifically, a multi-branch self-supervised network model is constructed and trained using a dataset; in this embodiment, each branch adopts an autoencoder (AE) or a Transformer structure.

[0038] The trained multi-branch self-supervised network model is used to process the fused feature vectors corresponding to each measurement point respectively. The feature representations of each modality in the normal mode are learned through a reconstruction loss function, and the fused feature vectors are compared with the feature representations of each modality in the normal mode. By calculating the reconstruction error of the corresponding time series features in the fused feature vectors, the occurrence time of the abnormal event is obtained; for the corresponding local graph features in the fused feature vectors, the visual fault area is obtained through an attention mechanism; for the text data, the occurrence frequency of high-frequency fault keywords is counted to obtain text fault keywords. Specifically, for time series, by calculating the error between the real-time data and the autoencoder reconstructed data and dynamically adjusting the threshold, the time point of the fault occurrence is accurately located; for image data, the model generates a visual heat map through an attention mechanism to intuitively display the fault area of the device, such as pipeline cracks or component wear; text analysis identifies potential fault causes by counting the occurrence frequency of high-frequency fault keywords (such as overload, trip).

[0039] In this embodiment, the multi-branch self-supervised network model introduces a contrastive learning mechanism. In the feature space, contrastive learning actively reduces the distance between similar samples (such as different modality data under normal working conditions), and at the same time pushes away abnormal samples (such as fault data), significantly improving the model's discriminative ability for fault modes. This model can capture subtle anomalies in multi-modal data and avoid the limitations of a single modality.

[0040] The abnormal detection results are input into a multi-modal large model, and the modality data are associated according to the cross-modal attention mechanism for fault reasoning, analyzing the abnormal causal relationship, and obtaining a visual diagnostic report.

[0041] Specifically, a knowledge graph database of thermal power equipment is constructed. The knowledge graph database of thermal power equipment includes the structure of thermal power equipment, historical fault cases, and corresponding solutions, and is stored in the form of triples. When the abnormal detection results are input into the multi-modal large model, relevant cases in the knowledge graph database of thermal power equipment are retrieved for fault reasoning, and a visual diagnostic report including the fault cause, evolution path, and maintenance suggestions is obtained.

[0042] After generating the visual diagnostic report, it also includes feedback optimization of the multi-modal large model, specifically including: Regularly calculate the confidence of the multimodal large model, perform active learning on multimodal data samples with confidence lower than the preset threshold, and introduce an expert annotation mechanism to annotate the multimodal data samples.

[0043] Embodiment 2 This embodiment provides a measuring point time series anomaly analysis system based on a multimodal large model. This system can mainly implement the measuring point time series anomaly analysis method based on the multimodal large model in the embodiment. Specifically, this system includes: A measuring point data acquisition module, which acquires various modal data corresponding to multiple measuring points related to the faulty equipment in a thermal power plant. The modal data at least includes text data, time series data, and image data; A multimodal feature fusion module, which respectively extracts the modal features in each modal data, calculates the cross-modal feature similarity weights based on the self-attention mechanism, and obtains the fused feature vector; A self-supervised fault detection module, which based on the self-supervised learning algorithm, performs contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtains the anomaly detection result; the anomaly detection result includes the occurrence time of the anomaly event, the detected image fault area, and the text fault keywords; A multimodal analysis and report generation module, which inputs the anomaly detection result into the multimodal large model, associates the modal data according to the cross-modal attention mechanism, performs fault reasoning, analyzes the anomaly causal relationship, and obtains a visual diagnostic report.

[0044] Embodiment 3 In an embodiment of the present invention, a computer-readable storage medium is provided. This medium belongs to the memory device of the terminal device and is mainly used to store programs and data. The computer-readable storage medium includes both the storage medium built into the terminal and the extended storage medium supported by the terminal. Specifically, any tangible medium that can store a program and be used by an instruction execution system, device, or component belongs to this category. This storage medium provides storage space for storing the terminal operating system and instructions (including one or more computer programs and their codes) that can be loaded and executed by the processor. Examples include electrically connected, portable disks, hard disks, RAM, ROM, EPROM / flash memory, optical fibers, CD-ROMs, optical storage devices, magnetic storage devices, etc., and combinations thereof.

[0045] In addition, the computer-readable storage medium also relates to data signals propagated in the baseband or as a carrier wave. These signals carry readable program codes and can be manifested in the form of electromagnetic signals, optical signals, etc. The readable storage medium is not limited to the above types and also includes other media that can send, propagate, or transmit programs for use by an instruction execution system, device, or component. The program code can be transmitted wirelessly, wired, via optical cable, RF, etc.

[0046] The program code can be written in various programming languages, such as object-oriented languages (Python, Java, C++, etc.) and procedural languages (C language, etc.). The code can be executed completely or partially on the user device, or can be used as an independent software package, or executed partially / fully on a remote device. The remote device connects to the user device through LAN, WAN or an Internet service provider.

[0047] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the method for analyzing the time series anomalies of measurement points based on a multi-modal large model in the above embodiments; Obtain various modal data corresponding to multiple measurement points related to a faulty device in a thermal power plant, and the modal data includes at least text data, time series data, and image data; Extract the modal features in each modal data respectively, calculate the cross-modal feature similarity weights based on the self-attention mechanism, and obtain a fused feature vector; Based on a self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain an anomaly detection result; the anomaly detection result includes the occurrence time of an abnormal event, the faulty area of the detected image, and the text fault keywords; Input the anomaly detection result into a multi-modal large model, associate the modal data according to the cross-modal attention mechanism, perform fault reasoning, analyze the anomaly causal relationship, and obtain a visual diagnosis report.

[0048] Embodiment 4 Figure 3 It is a block diagram of an electronic device provided by the present invention according to an embodiment.

[0049] Please refer to Figure 3 , the terminal device 600 is an electronic device, and the electronic device is presented in the form of a general computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0050] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above method part of this specification. For example, the processing unit 610 can execute the steps as shown in Figure 1 shown.

[0051] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0052] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0053] The bus 630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0054] The electronic device 600 may also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or may communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

Claims

1. A method for analyzing the time-series anomaly of measurement points based on a multi-modal large model, characterized in that, Including: Obtain various modal data corresponding to multiple measurement points related to the faulty equipment in the thermal power plant. The modal data at least includes text data, time series data, and image data; Extract the modal features in each modal data respectively, calculate the cross-modal feature similarity weights based on the self-attention mechanism, and obtain the fused feature vector; Based on the self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result; the anomaly detection result includes the occurrence time of the abnormal event, the faulty area of the detected image, and the text fault keywords; Input the anomaly detection result into the multi-modal large model, associate the modal data according to the cross-modal attention mechanism, perform fault reasoning, analyze the abnormal causal relationship, and obtain the visual diagnostic report.

2. The method for analyzing abnormal time series of measurement points based on a multimodal large model according to claim 1, wherein After obtaining various modal data corresponding to multiple measurement points related to the faulty equipment in the thermal power plant, it further includes a preliminary screening of the measurement points, specifically including: Construct a measurement point correlation matrix based on mutual information; Calculate the Pearson correlation coefficient between the data of each measurement point in the measurement point correlation matrix and the historical fault label, and retain the fault correlation measurement points with a fault correlation higher than the correlation threshold according to the Pearson correlation coefficient; Use the mutual information method to evaluate the redundancy between measurement points and eliminate redundant measurement points; Dynamically assign weights to the retained measurement points with higher fault correlations, and adjust the weights of the measurement points according to the data fluctuation degree and the abnormal trigger frequency.

3. The method for analyzing the time series anomaly of measurement points based on the multi-modal large model according to claim 2, wherein, It further includes preprocessing the multi-modal data corresponding to the measurement points after preliminary screening; specifically including: Use the set sliding window interpolation method to fill in the missing values for the time series data, remove the high-frequency noise through wavelet denoising, and then unify the dimension through Z-score standardization; Adopt adaptive histogram equalization for the image data to enhance the contrast, use the object detection model to locate the key component areas of the fault, and crop and resize them to a unified size; Adopt regular expressions to extract the fault-related text information for the text data, and use TF-IDF or BERT word embedding to convert the fault-related text information into numerical vectors.

4. The method for analyzing the time series anomaly of measurement points based on a multi-modal large model according to claim 3, wherein, Extract the modal features in each modal data respectively, specifically including: Extract the time domain features, frequency domain features, and wavelet entropy features for the time series data to obtain the time series features, use the deep convolutional neural network for the image data to extract the local graph features related to the fault, and capture the semantic features for the text data through the bidirectional recurrent neural network; Calculate the similarity weights between the time series features, local graph features, and semantic features based on the self-attention mechanism, and obtain the cross-modal aligned fused feature vector.

5. The method for analyzing abnormal time series of measuring points based on a multi-modal large model according to claim 1, wherein Based on the self-supervised learning algorithm, perform contrastive learning on the fused feature vector and the feature representation in the normal mode, and then obtain the anomaly detection result, specifically including: Construct a multi-branch self-supervised network model, and use the data set to train the multi-branch self-supervised network model; Use the trained multi-branch self-supervised network model to process the fused feature vector corresponding to each measurement point respectively; Learn the feature representations of each modality in the normal mode through the reconstruction loss function, and compare the fused feature vector with the feature representations of each modality in the normal mode; By calculating the reconstruction error of the corresponding time series features in the fused feature vector, the occurrence time of the abnormal event is obtained; for the corresponding local graph features in the fused feature vector, the visual fault area is obtained through the attention mechanism; for the text data, the occurrence frequency of high-frequency fault keywords is counted to obtain the text fault keywords.

6. The method for analyzing the time series anomaly of measurement points based on a multi-modal large model according to claim 1, wherein, The anomaly detection results are input into the multi-modal large model, and the modal data are associated according to the cross-modal attention mechanism for fault reasoning and analyzing the abnormal causal relationship, specifically including: Construct a knowledge graph database for thermal power equipment, which includes the structure of thermal power equipment, historical fault cases, and corresponding solutions, and is stored in the form of triples; When the anomaly detection results are input into the multi-modal large model, relevant cases in the knowledge graph database of thermal power equipment are retrieved for fault reasoning to obtain a visual diagnostic report including fault causes, evolution paths, and maintenance suggestions.

7. The method for analyzing the time series anomaly of measurement points based on the multi-modal large model according to claim 6, characterized in that, After generating the visual diagnostic report, it also includes feedback optimization of the multi-modal large model, specifically including: Regularly calculate the confidence of the multi-modal large model, perform active learning on the multi-modal data samples with confidence lower than the preset threshold, and introduce an expert annotation mechanism to annotate the multi-modal data samples.

8. A measuring point time series anomaly analysis system based on a multimodal large model, which is used to implement the measuring point time series anomaly analysis method based on a multimodal large model according to any one of claims 1 to 7, characterized in that Including: A measuring point data acquisition module that acquires multiple types of modal data corresponding to multiple measuring points related to the faulty equipment in the thermal power plant. The modal data at least includes text data, time series data, and image data; A multi-modal feature fusion module that respectively extracts the modal features in each type of modal data, calculates the cross-modal feature similarity weights based on the self-attention mechanism, and obtains a fused feature vector; A self-supervised fault detection module that performs contrastive learning on the fused feature vector and the feature representation in the normal mode based on the self-supervised learning algorithm to obtain the anomaly detection results. The anomaly detection results include the occurrence time of the abnormal event, the detected image fault area, and the text fault keywords; A multi-modal analysis and report generation module that inputs the anomaly detection results into the multi-modal large model, associates the modal data according to the cross-modal attention mechanism, performs fault reasoning, analyzes the abnormal causal relationship, and obtains a visual diagnostic report.

9. A computer-readable storage medium storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to execute the method for analyzing the abnormal time series of measuring points based on a multi-modal large model according to any one of claims 1 to 7.

10. A computing device, characterized in that, Including: One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors. The one or more programs include steps for executing the method for analyzing the abnormal time series of measuring points based on a multi-modal large model according to any one of claims 1 to 7.

Images