Offshore wind turbine fault early warning and operation and maintenance system based on multimodal large model
By integrating SCADA, visual language models, and edge-cloud collaborative technologies with multimodal large models, the problem of multi-source heterogeneous data fusion and decision optimization in offshore wind turbine fault early warning and operation and maintenance has been solved. This has enabled efficient and accurate fault identification and operation and maintenance suggestion generation, thereby improving the operation and maintenance efficiency and early warning capabilities of offshore wind turbines.
Patent Information
- Application Number
- CN202611600476.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-16
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for fault early warning and operation and maintenance of offshore wind turbines suffer from several problems, including insufficient ability to fuse multi-source heterogeneous data, difficulty in intelligent identification of time-series fault curves, inadequate generation of knowledge-enhanced operation and maintenance suggestions, and lack of sea state-constrained hierarchical decision-making. These issues result in high false alarm rates, low operation and maintenance efficiency, and insufficient timeliness and accuracy of early warnings.
A fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model is adopted. By integrating SCADA time series data, visual language models, retrieval enhancement generation and edge-cloud collaborative early warning, combined with dynamic attention weighting mechanism and closed-loop learning module, intelligent monitoring and operation and maintenance optimization of offshore wind turbines throughout their entire life cycle can be achieved.
It improves the detection rate of early and minor faults, reduces the false alarm rate, achieves an efficient fault response cycle, supports proactive intervention decision-making, ensures the continuity of early warning and the long-term effectiveness of the system, and improves the timeliness and accuracy of operation and maintenance.
Smart Images

Figure CN123118076A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent operation and maintenance technology of new energy equipment, specifically involving a fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model. Background Technology
[0002] Offshore wind turbines operate in harsh marine environments characterized by high salt spray, high humidity, and strong waves. The resulting power loss and maintenance costs due to turbine failures far exceed those of onshore wind power. Furthermore, sea state windows limit the opportunities for maintenance vessels to operate at sea, placing higher demands on the timeliness of fault warnings and the accuracy of maintenance decisions. Traditional SCADA alarm schemes based on fixed thresholds rely on manually set parameters, resulting in high false alarm rates when facing dynamic conditions such as turbine aging and changing sea states, failing to effectively identify early, subtle faults.
[0003] At the fault diagnosis level, existing methods are mainly divided into two categories: signal processing-based methods and deep learning-based methods. The former extracts vibration spectrum features through Fourier transform, wavelet decomposition, and other means, but suffers from severe feature aliasing when multiple faults occur concurrently. The latter, while capable of automatically extracting high-dimensional features, is limited by single-modal input, hindering its ability to comprehensively utilize multi-source information, and exhibits weak generalization ability when training data is insufficient. Existing knowledge base retrieval schemes mostly employ keyword matching, lacking the ability to jointly semantically reason with real-time fault state vectors and operational knowledge, resulting in insufficient matching degree between retrieval results and the current fault scenario.
[0004] At the operational decision-making level, existing systems generally lack mechanisms for collaboratively optimizing fault severity, equipment accessibility, and resource constraints. Functions such as dynamic assessment of sea state accessibility probability, automatic generation of tiered response work orders, and edge-cloud collaborative early warning have not yet been organically integrated on a single platform. Furthermore, with the continuous increase in the number of generating units, the manual review of operational recommendations is unsustainable, necessitating a closed-loop learning mechanism with continuous self-updating capabilities to ensure the long-term effective operation of the system.
[0005] In summary, existing technologies have significant shortcomings in areas such as multi-source heterogeneous data fusion, intelligent identification of time-series fault curves, knowledge-enhanced operation and maintenance suggestion generation, and sea state constraint-based hierarchical decision-making. There is an urgent need for a comprehensive intelligent operation and maintenance decision-making system that can integrate the above capabilities. Summary of the Invention
[0006] The purpose of this invention is to address the problems existing in the background technology and provide a fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model. By integrating core modules such as SCADA time series data analysis, visual language large model curve recognition, retrieval enhancement to generate operation and maintenance suggestions, multimodal feature fusion decision-making, and edge-cloud collaborative early warning, it realizes intelligent monitoring and operation and maintenance optimization of the entire life cycle of offshore wind turbines.
[0007] To achieve the above objectives, the technical solution of the present invention is: a fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model, comprising:
[0008] The data acquisition and processing module is configured to acquire SCADA data, CMS vibration data, sea ice and meteorological data, historical operation and maintenance text data and fault curve data, and perform data cleaning, alignment and standardization processing.
[0009] The curve recognition module is configured to use a visual language model (Qwen-VL visual language model) fine-tuned by fault curve samples to recognize features in the time-series fault prediction curve, and output the fault type, recognition confidence and mechanism explanation after verification.
[0010] The knowledge retrieval enhancement generation module is configured to retrieve evidence from the operation and maintenance knowledge base based on the retrieval enhancement generation (RAG) mechanism, and after combining the knowledge graph constraint ranking, introduce causal inference to calculate the rate of return, driving the large language model (Qwen-3.5B model) to generate intervention decision inference and executable operation and maintenance suggestions;
[0011] The fusion decision module is configured to perform joint scoring on multi-source heterogeneous fault characteristics, sea ice accessibility, causal inference and risk constraints based on a dynamic attention weighting mechanism, and output graded handling strategies and structured operation and maintenance work orders.
[0012] The edge-cloud collaboration module is configured to perform edge prediction and cloud verification collaboration under communication delay or interruption conditions, and adopts a link adaptive fusion strategy to ensure the continuity of early warning;
[0013] The closed-loop learning module is configured to form an evidence chain based on the feedback from work order execution and to update the sample pool and model parameters using an incremental learning strategy.
[0014] Furthermore, the curve recognition module includes a data preprocessing unit, which is configured as follows:
[0015] A dual-table fusion error correction mechanism is used to read SCADA high-frequency time series data, and data cleaning and initial screening are performed based on dead value removal strategy and effective grid-connected power threshold.
[0016] Establish a two-dimensional cross-alignment mechanism based on absolute time sliding window and unit relative power segmentation (Power Binning) to reorganize the time slices of all units under the same operating conditions;
[0017] Multidimensional statistical features are extracted for each physical subcategory. A unidirectional residual clipping strategy is designed to be compared with the percentile of the full field baseline to construct a multidimensional feature matrix.
[0018] Furthermore, the curve recognition module also includes a dynamic threshold anomaly detection unit, configured as follows:
[0019] The isolated forest algorithm is used to score anomalies in the multidimensional feature matrix, and the sign is flipped on the scoring results so that high scores correspond to high anomalies and are aligned with the CPU path.
[0020] A three-branch adaptive threshold determination is performed based on the skewness and standard deviation of the decision scores of the entire sample.
[0021] A real alarm is confirmed to be output if and only if the decision score of a certain unit is lower than the dynamic threshold for N0 consecutive sliding time windows.
[0022] Listen for physically untrusted flags. When a "physically untrusted" flag is received within a certain time window, the abnormal contribution characteristics of the corresponding time window are re-scored. The scores of the two isolated forest decisions are compared before and after. The decision is determined as a false alarm or the original decision is maintained based on the magnitude of the score change. Specifically, if the score drops by more than 20%, it is determined as a false alarm; if the score changes by less than 5%, the original decision is maintained.
[0023] Furthermore, the curve recognition module also includes a diagnostic context construction unit, configured as follows:
[0024] The SHAP algorithm is used to perform feature attribution calculation on the anomaly scoring results of isolated forests, and the Top-K features with the highest contribution and their corresponding SHAP values are extracted.
[0025] The Top-K features are reverse-engineered to be actual measurements with physical units, and assembled into a structured diagnostic context according to a predefined template. The fields include device ID, fault detection time, abnormal component category and core feature measurement list.
[0026] Based on the preset feature-physical equation mapping table, a corresponding physical constraint equation is generated for each feature, serving as the physical basis and verification rule.
[0027] Furthermore, a component-level fault diagnosis and explanation report is generated by combining deep logical reasoning with the large model, and the fault diagnosis method for wind turbine equipment based on the large model is deployed to the early warning platform system; specifically including:
[0028] The large language model uses chain-of-thought reasoning to sequentially perform fault mode recognition, root cause tracing, and impact range assessment; it also retrieves historical similar cases from the fault back-label sample library established by the closed-loop learning module, uses the historical similar cases and their remaining lifetime labels as auxiliary diagnostic evidence, infers the remaining lifetime (RUL) corresponding to the current fault, and outputs preliminary operation and maintenance suggestions.
[0029] The reasoning results are aggregated into a structured diagnostic report, which includes information such as faulty component location, severity rating, physical mechanism explanation and recommended inspection items. This report is then sent to the mobile terminals of maintenance personnel via push notifications to achieve a rapid closed-loop response.
[0030] Furthermore, the curve recognition module also includes a visual diagnostic map generation and VLM visual semantic parsing unit, configured as follows:
[0031] Based on the list of abnormal units and the overall statistical baseline, high-frequency time-series data of abnormal units are visualized and diagnostic charts are generated according to physical categories and subcategories to obtain time-series curves.
[0032] The generated time series curve is used as a multimodal visual input and fed into a large visual language model. Under the constraints of system prompts and physical verification rules, the image is subjected to engineering semantic analysis, and the output is a judgment result containing true / false anomaly labels and structured explanation fields.
[0033] When the deviation of the VLM output theoretical value exceeds the threshold, the backoff mechanism is triggered, the sample is relabeled and sent back to the dynamic threshold anomaly detection unit to re-perform isolated forest anomaly detection.
[0034] Furthermore, the curve recognition module also includes a visual language model fine-tuning unit, configured as follows:
[0035] Collect manually verified and labeled samples, and encapsulate image paths, manual labels, and corresponding reasons into supervised learning samples;
[0036] Convert the samples into a multi-turn dialogue format that is fine-tuned by visual instructions, and connect it to the fine-tuning training framework data pipeline;
[0037] LoRA (Low-Rank Adaptation) technique is used for efficient supervised fine-tuning of parameters, training only the low-rank incremental parameters of the attention projection layer, and freezing the visual encoder and the language backbone; the specific steps include:
[0038] Collect manually checked and labeled samples from the running visual image screening process: scan the image directories of "real anomalies" and "fake anomalies", encapsulate the image path, manual label (real / fake) and corresponding reasons into supervised learning samples, and export them in JSONL format to ensure that "curve image → true value JSON text" correspond one by one;
[0039] JSONL is converted into a multi-turn dialogue format for visual instruction fine-tuning and integrated into the fine-tuning training framework's data pipeline; only the low-rank incremental parameters of the attention projection layer are trained, and the visual encoder and language backbone are frozen; the AdamW optimizer is used in conjunction with a mixed-precision training and gradient accumulation strategy.
[0040] In each round, the label-F1 is calculated on the validation set and the optimal checkpoint is retained. After fine-tuning, the LoRA adapter is loaded into the inference service.
[0041] Furthermore, the knowledge retrieval enhancement generation module is configured as follows:
[0042] Document fragmentation is performed on real daily operation and maintenance reports, fault handling reports, manufacturer technical manuals and expert experience databases. A fragmentation strategy combining sliding window and semantic boundary detection is adopted, and each fragment is vectorized and a multi-level semantic index architecture is established.
[0043] Using real-time fault state vectors as query input, a two-stage retrieval mechanism is used to first recall the candidate set using BM25 and then refine the ranking using Dense Retriever to retrieve Top-K relevant evidence fragments from the operation and maintenance knowledge base. The retrieved evidence is then concatenated with the fault context and input into a large language model to generate executable operation and maintenance suggestions.
[0044] A causal reasoning engine based on a directed acyclic graph (DAG) for key equipment of offshore wind turbines is constructed to calculate the return rate of intervention actions and attach the optimal intervention action as a hard constraint to the generation Prompt of the large model.
[0045] The retrieved evidence and fault context are concatenated and input into a large language model to generate executable operation and maintenance suggestions. The generated operation and maintenance suggestions are then given a comprehensive score for their feasibility.
[0046] The formula for calculating the overall feasibility score is as follows:
[0047]
[0048] In the formula, Match task To suggest a semantic match with the currently faulty task, Match env To suggest a fit with current environmental conditions, Match resource To match the availability of the required resources, Risk penaltyFor safety risk penalties, beta1, beta2, beta3, and beta4 are the weighting coefficients corresponding to each evaluation indicator.
[0049] Furthermore, the fusion decision module is configured as follows:
[0050] Feature vectors are extracted from each data modality using a corresponding feature encoder. A dynamic attention weighting mechanism is then used for multimodal feature fusion, with the fusion formula as follows:
[0051]
[0052] In the formula, alpha m (t) represents the attention weight of the m-th mode at time t, A m Let be the mapping function from each modal feature to the unified fusion space. Let M be the input feature vector of the m-th mode at time t, and M be the total number of modes participating in the fusion.
[0053] A confidence decay mechanism is implemented for missing modalities. When a missing or abnormal modal data is detected, the corresponding attention weight is adjusted according to the time decay factor, and the weight of the missing modality is redistributed to the other available modalities.
[0054] The probability of offshore units reaching their destination within a future time window is calculated using a multi-factor logistic regression model, with the following formula:
[0055]
[0056] In the formula, For the Sigmoid function, v t H is the wind speed forecast value. t For the effective wave height forecast value, T t B represents the seawater temperature. t The gamma0, gamma1, gamma2, gamma3, and gamma4 represent the available status of the vessels under maintenance, and the gamma4 are the regression coefficients corresponding to the input variables in the multivariate logistic regression model.
[0057] The reachability probability P acc (t) is combined with the fault severity level to construct a risk-accessibility decision matrix, enabling the automatic output of graded handling strategies and structured operation and maintenance work orders.
[0058] Furthermore, the edge-cloud collaboration module is configured as follows:
[0059] Deploy a lightweight early warning model that has undergone knowledge distillation and quantization compression at the edge to perform local fault detection and preliminary risk classification in real time;
[0060] When the communication link is normal, an adaptive fusion strategy is used to merge the detection results from the edge and the cloud. The fusion formula is as follows:
[0061]
[0062] In the formula, the fusion weight λ t Quality of Service (QoS) of the communication link t Adaptive determination, This represents the prediction result output by the edge node at time t. This represents the prediction result output by the cloud model at time t.
[0063] Under communication interruption conditions, the edge node switches to a fully offline early warning mode, independently completes alarm judgment, generates preliminary handling suggestions and emergency event records, and after the link is restored, the data is batch synchronized to the cloud for review and verification.
[0064] Furthermore, the closed-loop learning module is configured as follows:
[0065] An end-to-end auditable chain of decision-making evidence is formed based on work order execution feedback;
[0066] A fault back-labeling closed-loop mechanism is adopted. When a unit is confirmed to have an actual fault by a work order, the data of N days before the fault is automatically traced back, the remaining life is labeled according to the time step, and a fault back-labeling sample library is established as a priori reference for model fine-tuning.
[0067] An incremental learning strategy is adopted to update the sample pool and model parameters. The incremental training loss function includes the task loss for new samples, the experience replay loss, and the KL divergence regularization term for the output distribution of the new and old models.
[0068] It enables dynamic experience self-learning, vectorizes completed work orders in real time and injects them into the knowledge base, and uses a time-sensitivity decay factor to adjust the weight of knowledge fragments.
[0069] Compared with the prior art, the present invention has the following beneficial effects:
[0070] (1) Multi-source heterogeneous fusion to improve the coverage of early warning. By fusing four types of heterogeneous data, namely SCADA, CMS vibration, sea ice meteorology and operation and maintenance text, a multi-modal feature fusion framework is constructed, which effectively overcomes the problem of insufficient robustness of single-modal methods in sensor failure scenarios and improves the detection rate of early weak faults by about 30% (compared to traditional threshold methods).
[0071] (2) Enhanced VLM curve recognition with physical constraints reduces false alarm rate and manual review burden. Based on the Qwen-VL model fine-tuned by LoRA and combined with inference prompts embedded with physical constraint equations, the engineering semantics of the time-series fault curve is automatically parsed. This automates the curve recognition task that originally required manual interpretation by senior operation and maintenance engineers. With the help of rule post-processing, the false alarm rate is reduced to less than 10%, which greatly shortens the fault response cycle.
[0072] (3) RAG knowledge enhancement driven by causal inference introduces a causal inference engine into the retrieval enhancement generation process to generate intervention decisions and executable operation and maintenance suggestions. A two-stage retrieval mechanism combined with causal constraint verification of the knowledge graph is adopted to accurately match evidence fragments of the current fault scenario from the multi-level operation and maintenance knowledge base, calculate the return on intervention actions, and use the optimal action as a hard constraint to drive the language model to generate structured operation and maintenance suggestions including inspection objects, execution conditions, time windows, and resource lists, directly guiding on-site personnel to work efficiently. At the same time, it supports hypothetical deduction of What If, realizing a business-level decision-making closed loop from passive prediction to proactive intervention.
[0073] (4) Coordinated decision-making based on sea conditions to reduce the risk of waiting for maintenance. The severity of the fault and the sea accessibility assessment are incorporated into the hierarchical decision matrix to automatically match the emergency response plan for high-severity faults with the remote / UAV alternative disposal plan under low accessibility conditions, so as to maximize the timeliness of maintenance while ensuring safety.
[0074] (5) Edge-cloud collaboration ensures continuous early warning in extreme scenarios. The link adaptive fusion strategy enables the system to dynamically adjust the prediction weights of the edge and cloud based on the real-time communication quality. When communication is interrupted, the edge nodes operate independently, achieving zero-interruption early warning guarantee under extreme sea conditions.
[0075] (6) Closed-loop learning mechanism to ensure long-term system effectiveness. Based on incremental learning and evidence chain recording, the closed-loop learning mechanism enables the system to continuously accumulate experience from each work order execution feedback. Through experience replay and knowledge distillation, catastrophic forgetting is prevented, ensuring the continuous improvement of diagnostic accuracy and suggestion quality in long-term operation. At the same time, dynamic experience self-learning is realized. The key information of the completed work order is vectorized and injected into the knowledge base in real time. The timeliness decay factor is used to suppress outdated experience, so that the model can continuously absorb the latest operation and maintenance knowledge for incremental training, significantly reducing the computing power cost and labor cost of long-term operation and maintenance. Attached Figure Description
[0076] Figure 1 This is a schematic diagram of the overall system architecture according to an embodiment of the present invention;
[0077] Figure 2This is a flowchart of the curve recognition module in an embodiment of the present invention (covering data preprocessing, anomaly detection, diagnostic reasoning, visualization recognition, and model fine-tuning sub-steps).
[0078] Figure 3 This is a flowchart of the knowledge retrieval enhancement generation module according to an embodiment of the present invention;
[0079] Figure 4 This is a schematic diagram of the risk-accessibility decision matrix of the fusion decision module in an embodiment of the present invention;
[0080] Figure 5 This is a flowchart of the edge-cloud collaboration module according to an embodiment of the present invention;
[0081] Figure 6 This is a diagram illustrating the fault phenomena and reasoning process of an embodiment of the present invention.
[0082] Figure 7 This is a diagram showing the results of the candidate fault cause analysis in an embodiment of the present invention;
[0083] Figure 8 This is a diagram showing the results of historical case retrieval and investigation suggestions generated according to an embodiment of the present invention;
[0084] Figure 9 This is a comparison chart of vibration timing curves and multi-channel results before and after fault triggering in an embodiment of the present invention. Detailed Implementation
[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0086] I. System Overall Architecture
[0087] like Figure 1 As shown, this invention provides a fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model, consisting of six core functional modules: a data acquisition and processing module, a curve recognition module, a knowledge retrieval and enhanced generation module, a fusion decision-making module, an edge-cloud collaboration module, and a closed-loop learning module. The modules interact through standardized data interfaces, supporting distributed deployment and horizontal scaling, and are suitable for centralized intelligent operation and maintenance management scenarios in large-scale offshore wind farms.
[0088] 1. Data Acquisition and Processing Module
[0089] The data acquisition and processing module is responsible for collecting wind turbine operating status data from multiple heterogeneous data sources in real time, mainly including the following four categories: (1) SCADA high-frequency time series data, including more than 100 measurement points such as active power, speed, blade pitch angle, nacelle temperature, and vibration acceleration; (2) CMS vibration monitoring data, including broadband vibration time domain signals of various parts of the main bearing, gearbox, and generator; (3) Sea ice and meteorological data, including significant wave height, wind speed and direction, seawater temperature, and visibility; (4) Historical operation and maintenance text data, including structured and unstructured records such as daily operation and maintenance reports, fault handling reports, and spare parts ledgers.
[0090] 2. Curve Recognition Module
[0091] The workflow of the curve recognition module is as follows: Figure 2 As shown, it includes six sub-steps: data preprocessing and feature construction, dynamic threshold anomaly detection, diagnostic context construction, large model fault diagnosis reasoning, visualization diagnostic graph generation and VLM visual semantic parsing, and visual language model fine-tuning training, corresponding to steps S1 to S6.
[0092] (1) Data preprocessing, baseline alignment and multidimensional feature construction (step S1)
[0093] The system acquires high-frequency time-series SCADA data from wind turbines via a data acquisition and processing module. During the data cleaning phase, a dual-table fusion error correction mechanism is employed to simultaneously read data copies from the main data acquisition server and backup acquisition terminals. The difference between corresponding measurement point values in the two tables at the same time is calculated. When the difference exceeds ±5% of the physical reasonable range for that measurement point, an error correction procedure is triggered, automatically selecting a data source with a smaller deviation from the expected value calculated based on the SCADA physical model. This suppresses systematic errors caused by sensor drift and acquisition link failures at the data source. For dead values (N consecutive sampling points with identical values) and out-of-range abnormal values, the system automatically marks them as invalid and sets them to null values before interpolation (using forward linear interpolation; a data quality alarm is triggered when the interpolation ratio exceeds 20%). Periods where the grid-connected power is 5% lower than the rated power are identified as shutdown or off-grid conditions and removed in subsequent horizontal comparative analysis to ensure the physical consistency of the overall baseline statistics.
[0094] During the baseline alignment phase, a two-dimensional cross-alignment mechanism based on an absolute time sliding window and relative power segmentation of the units is established. Using a 4-hour sliding window step and rated power percentage as the power segmentation criterion, the power ratio sequences of each unit are divided into three operating condition intervals: low load (0-30%), medium load (30%-80%), and full load (80%-100%). Time-series data from all units falling within the same absolute time window and having the same operating condition label are aligned to form a two-dimensional data matrix for horizontal comparative analysis. Sparse slices with a data density of less than 50% are automatically discarded to ensure consistency in the physical benchmark for horizontal feature comparison.
[0095] In the feature construction phase, multi-dimensional statistical features such as mean, standard deviation, and maximum value are extracted for each physical subcategory (temperature, pressure, vibration, speed, etc.). For measurement points with redundant sensors (such as converter-side speed and generator-side speed), the difference between all redundant combinations is calculated, and the maximum absolute value is taken as the redundancy fusion feature to achieve cross-validation of sensor health at the physical level. Horizontal comparison adopts a unidirectional residual clipping strategy: using the 10th to 90th percentiles of the features of all units under the same time window and operating condition as the normal baseline, the residual is obtained by calculating the difference between the feature of each unit and the median of the baseline; for temperature features, only positive residuals are retained (high values indicate anomalies), for pressure features, only the absolute values of negative residuals are retained (low values indicate anomalies), and for vibration features, both-way absolute values of residuals are retained, forming a directional feature residual matrix oriented towards physical failure mechanisms.
[0096] (2) Dynamic threshold anomaly detection (step S2)
[0097] An isolated forest algorithm is used for unsupervised anomaly scoring of the feature residual matrix. In a GPU environment, cuML is prioritized for acceleration, and the output scores are sign-flipped (decision_score = max_score - raw_score). The maximum raw score (max_score) is subtracted from the raw score (raw_score) of each sample, resulting in higher transformation scores for samples with lower raw scores, thus indicating a higher degree of anomaly. The dynamic threshold is adaptively selected using a three-branch approach based on the skewness and standard deviation of the decision scores.
[0098] When the absolute value of the skewness is less than 0.5 and the standard deviation is less than 0.5 (nearly normal distribution), a conservative threshold is used:
[0099]
[0100] When the skewness is greater than 0 (right-skewed distribution), a skewness correction factor is introduced for adaptive downward adjustment:
[0101]
[0102] For all other cases (skewness not greater than 0), the lower limit of the standard box plot is used:
[0103]
[0104] In the formula, Skewness is the skewness of the decision score, IQR is the interquartile range, and Q... 25 This represents the 25th percentile of the current decision score set. The system will confirm and output a genuine alarm only if the isolated forest decision score of a certain unit is continuously lower than the above dynamic threshold and anomaly detection is triggered within N0 consecutive sliding time windows (default N0=8), thus avoiding false alarms caused by transient noise.
[0105] Simultaneously, the system monitors the physical closed-loop verification backoff signal from step S5. When a unit is marked as "physically untrustworthy" within a certain time window, the following operations are automatically performed: the abnormal contribution feature (feature column triggering physical deviation) in the feature residual matrix of that time window is set to zero (i.e., temporarily masked), and the isolated forest is rerun for scoring; the isolated forest decision scores are compared before and after, and if the score drops by more than 20%, the original anomaly is determined to be a false alarm of physical untrustworthiness, no real alarm is output, and the event is recorded as a negative sample; if the score change is less than 5%, the original anomaly determination is maintained, and the physical deviation information of the VLM is submitted to the large model in step S4 as additional evidence for comprehensive reasoning.
[0106] (3) Diagnostic context construction (step S3)
[0107] For abnormal units, the top 5 features contributing most to the anomaly determination (SHAP value) are extracted. The normalized residual values are then reverse-engineered to the original physical quantities (including current measured values, average baseline values, deviation amplitude, and historical trends over the past 7 days). These are formatted into contextual prompts that can be directly understood by LLM according to a predefined structured XML template, including fields such as: equipment ID, fault detection time, abnormal physical subcategory, core feature list (including measured values, baseline values, and deviation rates), and historical maintenance record summary (retrieved from the knowledge base). Based on a pre-defined feature-physical equation mapping table, corresponding physical constraint equations are generated for each feature, serving as the physical basis and verification rules. The feature name, the physical equation used, and the constraint conditions are mapped one-to-one and converted into a JSON structure.
[0108] (4) Large model reasoning (step S4)
[0109] After receiving the structured diagnostic context, the Large Language Model (LLM) sequentially performs fault mode identification (identifying the 1-3 most likely fault modes and providing a probability ranking) through a chain-of-thought reasoning mechanism, root cause tracing (analyzing the fault propagation path along the transmission chain), impact scope assessment (determining whether it may trigger a chain of faults), and handling priority determination (considering current sea conditions, spare parts availability, and similar events). Finally, it outputs a structured diagnostic report with fields including: faulty component location, severity rating (P1-P4), physical mechanism explanation, a list of recommended inspection items, and preliminary handling suggestions, which are then sent to the mobile terminals of maintenance personnel via push notification.
[0110] (5) Visual diagnostic map generation and VLM visual semantic parsing (step S5)
[0111] Based on the list of abnormal units output in step S2 and the overall statistical baseline, a multi-process parallel rendering method is used. The output directory is organized according to "physical category - subcategory - unit number". A standardized time series comparison chart is generated for each abnormal subcategory of each abnormal unit: the foreground curve is the normalized characteristic time series of the target unit, the background band area is the signal band at the 10th to 90th percentile of the overall baseline, the red dashed line marks the dynamic threshold position, and the green shading marks the shutdown / low load period (for VLM to identify interpretable shutdown). For categories with redundant sensors, an additional sub-chart of the difference in redundant channel combinations is drawn to visually display the sensor health status. All images are saved in PNG format at 1200 DPI resolution, with the file name rule "project_unit number_category_subcategory.png". If results with the same path already exist, they can be skipped to reduce redundant computing power consumption.
[0112] The generated diagnostic graph is encapsulated in Base64 encoding as an OpenAI-compatible multimodal message payload, and the Visual Language Model (VLM) inference service interface is invoked locally or privately deployed. System prompts are loaded from an external configuration file, including model role definitions ("You are a wind power operation and maintenance expert, responsible for identifying true anomalies in the time series curve"), physical verification rules (including physical equations, comparisons of measured and theoretical values, and fault data basis), output format constraints (mandatory JSON, including fields such as label / reason / confidence / evidence), and illusion suppression instructions ("If there are no obvious anomalies, return label: normal"). After parsing the JSON returned by the model, rule post-processing is performed: when the relative deviation in the graph is less than 5 percentage points below the overall baseline, or when the abnormal period falls entirely within the explainable downtime / off-grid period, the model's judgment result is forcibly corrected to normal, and a rule trigger explanation is added to the evidence field, thereby further reducing the false alarm rate.
[0113] To further enhance the forced coupling of physical mechanisms between modules, this invention establishes a bidirectional closed-loop verification mechanism between visual language model recognition and anomaly detection. Assume that step S3 generates K physical constraint equations for the current anomalous unit, each equation containing theoretical values. With tolerance In step S5, the measured physical quantities output by the VLM (such as peak frequency, temperature difference, and power change rate) are: If there exists any k such that:
[0114]
[0115] Then, closed-loop rollback is triggered. The specific steps are as follows: When the deviation between the anomaly judgment output by VLM (such as slope change, frequency peak) and the theoretical value in the physical constraint equation transmitted in step S3 exceeds a preset threshold (such as the deviation between the measured peak frequency of vibration spectrum and the theoretical frequency of bearing failure > 2%, or the deviation between the calculated temperature difference and the actual temperature difference > 10%), the system automatically triggers closed-loop rollback: the anomaly judgment is marked as "physically unreliable", its confidence weight is temporarily reduced, and the deviation feature identifier is recorded; then the deviation information and the original time series data are encapsulated into a rollback signal and sent back to step S2 to re-perform isolated forest anomaly detection, the feature column that triggers the deviation is removed from the feature residual matrix, the feature residual and isolated forest decision score are recalculated, and the results are compared with those before recalculation. The decision score is calculated after recalculation. If the recalculated decision score drops by more than 20%, the original anomaly is determined to be a false alarm due to physical unreliability, and no real alarm is output. This time window is then included in the negative sample pool of the closed-loop learning module. If the absolute value of the change is less than 5%, the original anomaly determination is maintained, and the physical deviation information is added as supplementary evidence to the large model diagnostic context in step S4. The diagnostic report is required to explain the possible causes of the deviation (such as sensor drift or atypical fault modes). This closed-loop mechanism ensures that anomaly detection and visual recognition no longer transmit data unidirectionally, but instead form a strong coupling relationship of mutual verification and correction through physical equations. This effectively reduces false alarms caused by visual artifacts or model illusions, and provides a physical basis for dynamic threshold adaptation.
[0116] (6) Fine-tuning and training of the visual language model (step S6)
[0117] Based on the pre-trained Qwen3-VL-8B model, the LoRA (Low-Rank Adaptation) strategy was used for efficient parameter fine-tuning. The LoRA hyperparameters were set as follows: rank r=16, scaling factor α=32, dropout=0.05, and the adaptation module covered all attention projection layers (q_proj, k_proj, v_proj, o_proj). Manually checked and labeled samples were collected from the existing visual image screening process: scanning the image directories for "real anomalies" and "fake anomalies" (or exporting them by merging screening results with the manual correction table). The image paths, manual labels (real / fake), and corresponding reasons were encapsulated as supervised learning samples and exported in JSONL format, ensuring a one-to-one correspondence between "curve image → ground truth JSON text". The training set consisted of manually checked "curve image - structured JSON ground truth" sample pairs, in a multi-turn dialogue JSONL file format (image + task prompts for the human side, and JSON string for the assistant side).
[0118] Optimization strategy: AdamW optimizer, initial learning rate 1×10⁻⁶ -4 The system employs cosine annealing scheduling, a warmup ratio of 0.05, 5 training epochs, an effective batch size of 8 per GPU (with 4 steps of gradient accumulation), mixed precision BF16, a gradient pruning threshold of 1.0, and weight decay of 0.01. In each epoch, the label-F1 score is calculated on the validation set, the best checkpoint is retained, and after fine-tuning, the business-side inference interface can be updated via hot reloading through the vLLM service, requiring no redeployment throughout the process.
[0119] 3. Knowledge Retrieval Enhancement Generation Module
[0120] The workflow of the knowledge retrieval enhancement generation module is as follows: Figure 3 As shown, it includes four sub-steps: knowledge base construction, knowledge graph construction, two-stage retrieval, and suggestion generation.
[0121] (1) Construction of a multi-level operation and maintenance knowledge base
[0122] Adaptive sharding is performed on the original documents: For structured documents (technical manuals, standard procedures), a sharding strategy prioritizing chapter boundary detection is adopted (maximum fragment length 512 tokens, overlap 128 tokens); for unstructured documents (daily maintenance reports, fault reports), sliding window sharding is adopted (window length 256 tokens, step size 128 tokens), and semantically coherent fragments across windows are merged through semantic similarity detection. All fragments are vectorized using a pre-trained embedding model in the Chinese domain to generate dense vectors, establishing a multi-level semantic indexing architecture that operates in parallel with both BM25 sparse index (coarse-grained recall) and FAISS vector index (fine-grained ranking), supporting daily incremental updates.
[0123] (2) Construction of knowledge graph in the field of wind power operation and maintenance
[0124] Using turbine equipment entities (blades, main shaft, gearbox, generator, converter, yaw system, pitch system, etc.), fault types (fatigue cracks, bearing wear, insulation breakdown, control failure, etc.), and maintenance solutions (component replacement, grease lubrication, parameter reconfiguration, etc.) as nodes, and directed edges for "component-fault" causal relationships, "fault-solution" mapping relationships, and "component-component" association propagation relationships, a domain knowledge graph (approximately 2,000 nodes and 8,000 relationships) covering all major drive chains of offshore wind power is constructed. The TransE embedding algorithm is used to map entities and relationships to a 128-dimensional low-dimensional vector space, supporting multi-hop graph reasoning and path interpretability output.
[0125] (3) Two-stage retrieval and suggestion generation
[0126] In the first stage, a real-time fault status vector (including fault type ID, component location code, severity level, and current sea state level) is used as the query. The Top-50 candidate knowledge fragments are recalled through BM25 sparse retrieval, and documents that do not match the current equipment model are filtered out. In the second stage, a Cross-Encoder reordering model is used to refine the Top-50 candidates, and the Top-K (default K=5) high-quality evidence fragments are selected. The knowledge graph path constraint is used for verification: a maximum of 3 hops of reasoning are performed on the graph along the current fault type to verify whether the maintenance plan described by the candidate fragment has a reasonable causal chain with the current fault. Candidates with broken causal chains are filtered out to improve the retrieval accuracy.
[0127] A Directed Acyclic Graph (DAG) for key equipment of offshore wind turbines is constructed based on an underlying causal network. Nodes are observable or intervention variables (such as power limitation and lubrication dosage), and edges represent the directions of causal relationships. Each edge is accompanied by the physical equation or empirical formula output in step S3. The nodes and edges of the DAG are predefined based on physical mechanisms, and the conditional probability parameters are updated using Bayesian methods using historical data (including RUL labels in the fault back-annotation sample library). It also supports Causal AI to assist business decision-making. Operations personnel can interact with the model to perform What-If hypothetical deductions, such as: "If the power of Unit 3 is limited by 10% and additional grease is applied, how many days will the RUL be extended?" The system will parse this query and map it to an intervention node on the DAG, call the causal inference engine to calculate and return the RUL probability distribution before and after intervention (Monte Carlo sampling) and the recommended execution window (combined with sea state accessibility), achieving an active closed loop from prediction to intervention.
[0128] The retrieved Top-K evidence fragments are first input into the causal inference engine, which then uses counterfactual reasoning to determine possible intervention actions. The engine calculates the change in RUL and the rate of return after intervention along the DAG, and calculates the optimal 1-3 intervention actions as a candidate suggestion set based on the ROI formula. This candidate set is then attached as a hard constraint to the generation prompt of the large model, requiring that the maintenance suggestions generated by the large model must belong to this set or be semantically equivalent to it. This ensures that the model will not give causally unreasonable or uneconomical suggestions, thus assisting maintenance personnel in finally confirming the work order and realizing a business-level decision-making closed loop from passive prediction to proactive intervention.
[0129] The formula for calculating ROI is:
[0130]
[0131] In the formula, ROI represents the rate of return on the intervention action. For possible intervention actions, To perform the action Change in RUL after intervention, Cost ) to perform the action Total resource cost of intervention.
[0132] The evidence fragments are then concatenated with the fault diagnosis context, and a large model is constructed using Prompt Engineering as input. This drives the Qwen-3.5B model to generate structured operation and maintenance recommendations, which include at least: inspection objects (specific components and measurement points), execution conditions (sea state requirements, required specialized qualifications), time window (recommended completion within X hours / day), What-If hypothetical extrapolation results, a list of required resources (spare parts models, quantities, special tools), and safety precautions. After the recommendations are generated, their executability is evaluated using a scoring function E. Recommendations below the threshold (E < 0.6) trigger a regeneration process until quality requirements are met or the maximum number of retries (default 3) is reached.
[0133] 4. Integrated Decision Module
[0134] The integrated decision-making module combines multi-source modal characteristics, sea condition accessibility assessment, and risk decision matrix to output tiered handling work orders.
[0135] (1) Multimodal feature encoding
[0136] SCADA structured time-series data is encoded using a temporal convolutional network (TCN), which captures time-dependent features at different time scales through dilated convolution, and the encoded output is a 128-dimensional feature vector. CMS vibration broadband signals are generated into a time-spectrum map using short-time Fourier transform (STFT, window length 256 points, overlap 128 points), and then frequency domain envelope features (including fundamental frequency, harmonic component amplitude and modulation sideband energy) are extracted by 1D-CNN, and the encoded output is a 64-dimensional feature vector. Sea ice meteorological data (significant wave height, wind speed, sea temperature, ship availability status) are encoded into a 32-dimensional environmental context feature vector through a 3-layer fully connected network. Maintenance text (fault history summary, maintenance record keywords) is labeled by a BERT-like model to obtain 768-dimensional semantic features, and then compressed to 64 dimensions through linear projection.
[0137] (2) Dynamic attention-weighted fusion and missing modality processing
[0138] The following dynamic attention-weighted fusion formula is used to aggregate multimodal features:
[0139]
[0140] In the formula, alpha m (t) represents the attention weight of the m-th mode at time t, A m Let be the mapping function from each modal feature to the unified fusion space. Let be the input feature vector of the m-th modality at time t, and M be the total number of modalities participating in the fusion; alpha is the attention weight for each modality. m(t) is dynamically calculated by the cross-modal attention network based on the current fault context, satisfying... When a missing or quality-abnormal modality is detected, the confidence decay mechanism adjusts the attention weight of that modality by an exponential decay factor (decay coefficient 0.5 / number of missing hours) and redistributes the missing weight evenly to the remaining available modalities, ensuring that the fusion results remain robust even in scenarios where some sensors fail.
[0141] (3) Sea state accessibility assessment and risk-accessibility decision matrix
[0142] Based on a multivariate logistic regression model, using the historical sea voyage records of the past 180 days (including weather conditions and actual sea voyage results) as the training set, the following accessibility probability model is fitted.
[0143] The formula for calculating the reachability probability is:
[0144]
[0145] In the formula, sigma is the Sigmoid function, and v t To forecast wind speed (m / s), H t To predict the significant wave height (m), T t B represents the seawater temperature (°C). t The system represents the availability status of the maintenance vessel (1 = available, 0 = unavailable). gamma0, gamma1, gamma2, gamma3, and gamma4 are the regression coefficients corresponding to the input variables in the multifactor logistic regression model. The model parameters are refitted monthly based on the latest historical data to adapt to seasonal sea state changes.
[0146] like Figure 4 As shown, the system correlates fault severity (divided into four levels, P1 to P4) with sea state reachability probability (P... acc (Using 0.5 as the dividing line) Jointly construct a 2×4 decision matrix: P1 level (urgent) and high accessibility (P acc ≥0.5) → Immediately dispatch maintenance vessels to conduct onboard repairs; P1 level but low accessibility (P acc<0.5) → Immediately initiate remote reset procedure + drone damage assessment + encrypted digital twin simulation; P2 level (severe) and high accessibility → planned deployment within 24 hours; P2 level and low accessibility → conditional deployment within 72 hours (waiting for weather window); P3~P4 level → included in monthly planned maintenance queue, continuously monitoring for degradation risk. The system automatically generates structured maintenance work orders based on the decision matrix results. Fields include: work order number, unit ID, fault level, handling plan, required resources, planned deployment time window, and safety procedure number, and pushes them to the Operations and Maintenance Management System (EAM) for execution. After the work order is executed, the feedback results filled in by the maintenance personnel are mapped to training labels to build a training sample pool. An incremental learning strategy combining experience replay and knowledge distillation is adopted, and a dynamic experience self-learning algorithm is added on this basis to absorb maintenance work orders and feedback generated by daily maintenance and repair in real time, enabling the RAG knowledge base to evolve autonomously, and hot updates are only performed when the new model is better than the previous online version.
[0147] 5. Edge-Cloud Collaboration Module
[0148] The workflow of the edge-cloud collaboration module is as follows: Figure 5 As shown, a three-layer architecture is adopted: offshore booster station edge node (L1), offshore platform cloud virtual machine (L2), and landside data center (L3).
[0149] At the edge (L1), a lightweight early warning model (with about 1 / 8 of the original cloud model) that has been knowledge distilled and INT8 quantized and compressed is deployed to perform local fault detection and preliminary risk classification in real time, with a response latency of less than 100ms, which meets the real-time requirements for safety monitoring of critical marine equipment.
[0150] When the communication link is normal, an adaptive fusion strategy is used to merge the detection results from the edge and the cloud:
[0151]
[0152] Fusion weight λ t Quality of Service (QoS) of the communication link t (Taking into account packet loss rate, round-trip time (RTT), and available bandwidth) adaptively determined. This represents the prediction result output by the edge node at time t. This represents the prediction result output by the cloud model at time t.
[0153] The formula for calculating the fusion weight is:
[0154]
[0155] In the formula, a and b are parameters jointly calibrated using historical link quality and prediction error. When communication quality is high (low latency, low packet loss), the cloud weight increases, using the full cloud model to improve prediction accuracy; when the link degrades, the edge weight automatically increases to ensure uninterrupted early warning.
[0156] In the event of a communication interruption, edge nodes automatically switch to a fully offline early warning mode, independently completing alarm judgment, generating preliminary handling suggestions, and recording emergency events. Once the link is restored, the accumulated early warning events, sensor data summaries (compressed and stored at 10-minute granularity), and local decision logs are batch synchronized to the cloud. The cloud-based full model then performs verification and updates the global unit health status map, and incorporates the samples from the offline period into the incremental training data pool of the closed-loop learning module.
[0157] 6. Closed-loop learning module
[0158] The closed-loop learning module constructs a full-link evidence chain recording mechanism: for each diagnosis and decision-making process, it fully records the model version number, input data summary (including sensor channels, time range, and data quality markers), retrieved evidence fragments and scores, generated original suggestions, manual verification and revision content, and final work order execution feedback, forming an end-to-end auditable decision evidence chain that supports post-event traceability and compliance review.
[0159] Sample quality assessment and automatic labeling: The feedback from maintenance personnel after work order execution (success / failure / partial revision) is automatically mapped to training labels—successful, unrevised suggestions are used as high-quality positive samples; failed or significantly revised suggestions, along with the manually revised correct version, are used as negative / revision sample pairs and included in the training sample pool. A maximum sample pool capacity is set (default 50,000 entries), and a priority elimination strategy that prioritizes rare fault categories is adopted to maintain the class balance of the training data.
[0160] Incremental learning of the model employs a strategy combining experience replay and knowledge distillation. The incremental training loss function is defined as:
[0161]
[0162] In the formula, L new For the task loss of new samples, L replay The empirical replay loss is KL(p) which is randomly sampled from the historical sample pool. old || p new () for the new and old models () , The KL divergence regularization term (to prevent catastrophic forgetting) of the output distribution of the new and old models respectively, ρ and This is the balance coefficient (default ρ=0.3). =0.1).
[0163] When the newly added sample pool accumulates more than 500 labeled samples, an incremental fine-tuning is triggered. After the fine-tuning is completed, the metrics (precision, recall, F1) are evaluated on the retention validation set. Hot updates are only performed when the new model is better than the current online version, ensuring a monotonic improvement in the quality of online services.
[0164] To address the issues of insufficient accuracy in remaining lifetime (RUL) labeling and the lack of fault labels, a fault back-labeling closed-loop mechanism is designed. When a unit experiences a confirmed fault via a work order, the system automatically backtracks to data from N days prior to the fault (default N=90) and labels the RUL values according to the time interval from the fault occurrence. A fault back-labeling sample library is then established as prior knowledge for model fine-tuning. For example: 1 day prior → RUL=1, 2 days prior → RUL=2... 90 days prior → RUL=90. This solves the problem of "no fault labels," making the model more accurate with repeated use.
[0165] To address the pain points of static delivery of large models, a dynamic experience-based self-learning algorithm was designed. This algorithm can absorb maintenance work orders and feedback generated from daily maintenance and repair in real time, supporting autonomous updates to the RAG knowledge base. Without the need for re-consuming computing power for model fine-tuning, the system backend automatically vectorizes and injects the daily newly added maintenance work orders and closed-loop records from the site into the knowledge base in real time, breaking through the limitations of traditional static thresholds. The underlying VAE model utilizes a sliding window to frequently absorb unit steady-state data for real-time incremental training. After maintenance personnel execute work orders and provide valid feedback, the system automatically extracts key information from the work orders, such as fault descriptions, executed operations, operation effects, and operator notes. This information is concatenated into natural language text (no more than 512 tokens in length), and a lightweight embedding model is used to generate a 768-dimensional vector in real time. This vector, along with the original text, timestamp, and work order number, is imported into the RAG knowledge base in step S7, and a timeliness decay factor (Score) is set.
[0166] The formula for calculating the time-related decay factor Score is as follows:
[0167]
[0168] In the formula, sim represents the cosine similarity between the query vector and the knowledge fragment. , For the current search time, This is the time when the knowledge fragment is added to the database. This exponential decay reduces the weight of experience older than 90 days by 60%, thus prioritizing the recommendation of recently validated knowledge.
[0169] The system attempts to match the operation plan in a new work order with existing causal edges in the causal directed acyclic graph (DAG): if a causal edge exists from the current fault type to the operation plan, the confidence of that edge is increased; if it is a new plan, it is initially marked as "to be verified," and only when at least three different fault cases subsequently use the plan successfully is it promoted to a formal knowledge node. All injected knowledge fragments are automatically merged and the index is rebuilt daily to ensure that they can be retrieved the next day. This achieves a complete closed loop of "execution → feedback → knowledge update → influencing subsequent decisions," enabling the system to update itself in real time and autonomously adapt to data drift caused by seasonal changes and sea state variations, achieving "autonomous operation and maintenance" of the system without manual parameter tuning.
[0170] 7. Examples of Intelligent System Analysis
[0171] To further illustrate the actual operation of the system of this invention, the system takes the operating data, fault phenomena, and related monitoring information of the offshore wind turbine to be diagnosed as input, and performs a comprehensive analysis by combining the diagnostic context, the operation and maintenance knowledge base, and multimodal monitoring data. For example... Figure 6 As shown, the system first analyzes the input fault phenomena and related monitoring information to form a fault phenomenon description and corresponding reasoning analysis results; based on the above analysis results, it further identifies possible fault modes and analyzes their causes to form candidate fault causes and corresponding analysis results, such as... Figure 7 As shown.
[0172] Based on the analysis of candidate fault causes, the system retrieves historical cases matching the current fault status from the operation and maintenance knowledge base, and generates corresponding troubleshooting suggestions by combining historical fault handling experience. Specific results are as follows: Figure 8 As shown. To further verify the fault analysis results, the system retrieved vibration monitoring data before and after the fault triggering time, and performed time-series comparative analysis on relevant measuring points and multiple vibration channels, as shown. Figure 9 As shown, this process helps in identifying abnormal characteristics and their changing trends. Through the above process, continuous intelligent analysis is achieved, from fault phenomenon analysis, candidate cause identification, historical case retrieval and troubleshooting suggestion generation to time-series data-assisted verification, providing a basis for subsequent fault handling and operation and maintenance decisions.
[0173] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.
Claims
1. A fault early warning and operation and maintenance system for offshore wind turbines based on a multimodal large model, characterized in that, include: The data acquisition and processing module is configured to acquire SCADA data, CMS vibration data, sea ice and meteorological data, historical operation and maintenance text data and fault curve data, and perform data cleaning, alignment and standardization processing. The curve recognition module is configured to use a visual language model fine-tuned from fault curve samples to recognize features in the time-series fault prediction curve, and output the fault type, recognition confidence and mechanism explanation after verification. The knowledge retrieval enhancement generation module is configured to retrieve evidence from the operation and maintenance knowledge base based on the retrieval enhancement generation mechanism, and then combine it with the knowledge graph constraint sorting to introduce causal inference to calculate the rate of return, driving the large language model to generate intervention decision inference and executable operation and maintenance suggestions; The fusion decision module is configured to perform joint scoring on multi-source heterogeneous fault characteristics, sea ice accessibility, causal inference and risk constraints based on a dynamic attention weighting mechanism, and output graded handling strategies and structured operation and maintenance work orders. The edge-cloud collaboration module is configured to perform edge prediction and cloud verification collaboration under communication delay or interruption conditions, and adopts a link adaptive fusion strategy to ensure the continuity of early warning; The closed-loop learning module is configured to form an evidence chain based on the feedback from work order execution and to update the sample pool and model parameters using an incremental learning strategy.
2. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model as described in claim 1, characterized in that, The curve recognition module includes a data preprocessing unit, which is configured as follows: A dual-table fusion error correction mechanism is used to read SCADA high-frequency time series data, and data cleaning and initial screening are performed based on dead value removal strategy and effective grid-connected power threshold. Establish a two-dimensional cross-alignment mechanism based on absolute time sliding window and unit relative power segmentation to reorganize the time slices of all units under the same operating conditions. Multi-dimensional statistical features are extracted for each physical subcategory. A one-way residual truncation strategy is designed to be compared with the percentile of the full-field baseline to construct a multi-dimensional feature matrix.
3. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 2, characterized in that, The curve recognition module further includes a dynamic threshold anomaly detection unit, configured as follows: The isolated forest algorithm is used to score anomalies in a multidimensional feature matrix, and the scores are then flipped to ensure that high scores correspond to high anomalies. A three-branch adaptive threshold determination is performed based on the skewness and standard deviation of the decision scores of all samples. A real alarm is confirmed to be output if and only if the decision score of a certain unit is lower than the dynamic threshold for N0 consecutive sliding time windows. Listen for physically untrusted flags. When a "physically untrusted" flag is received for a certain time window, the abnormal contribution characteristics of the corresponding time window are re-scored. The decision scores of the two isolated forest decisions are compared before and after, and the false alarm or the original decision is determined based on the magnitude of the score change.
4. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 3, characterized in that, The curve recognition module further includes a diagnostic context construction unit, configured as follows: The SHAP algorithm is used to perform feature attribution calculation on the anomaly scoring results of isolated forests, and the Top-K features with the highest contribution and their corresponding SHAP values are extracted. The Top-K features are reverse-engineered to be real measurements containing physical units, and then assembled into a structured diagnostic context according to a predefined template. Based on the preset feature-physical equation mapping table, a corresponding physical constraint equation is generated for each feature, serving as the physical basis and verification rule.
5. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 1, characterized in that, The curve recognition module also includes a visualization diagnostic graph generation and VLM visual semantic parsing unit, configured as follows: Based on the list of abnormal units and the overall statistical baseline, high-frequency time-series data of abnormal units are visualized and diagnostic charts are generated according to physical categories and subcategories to obtain time-series curves. The generated time series curve is used as a multimodal visual input and fed into a large visual language model. Under the constraints of system prompts and physical verification rules, the image is subjected to engineering semantic analysis, and the output is a judgment result containing true / false anomaly labels and structured explanation fields. When the deviation of the VLM output theoretical value exceeds the threshold, the backoff mechanism is triggered, the sample is relabeled and sent back to the dynamic threshold anomaly detection unit to re-perform isolated forest anomaly detection.
6. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 5, characterized in that, The curve recognition module also includes a visual language model fine-tuning unit, configured as follows: Collect manually verified and labeled samples, and encapsulate image paths, manual labels, and corresponding reasons into supervised learning samples; Convert the samples into a multi-turn dialogue format that is fine-tuned by visual instructions, and connect it to the fine-tuning training framework data pipeline; LoRA technique is used for efficient supervised fine-tuning of parameters, training only the low-rank incremental parameters of the attention projection layer, and freezing the visual encoder and the language backbone; In each round, the label-F1 is calculated on the validation set and the optimal checkpoint is retained. After fine-tuning, the LoRA adapter is loaded into the inference service.
7. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 1, characterized in that, The knowledge retrieval enhancement generation module is configured as follows: Document fragmentation is performed on real daily operation and maintenance reports, fault handling reports, manufacturer technical manuals and expert experience databases. A fragmentation strategy combining sliding window and semantic boundary detection is adopted, and each fragment is vectorized and a multi-level semantic index architecture is established. Using real-time fault state vectors as query input, a two-stage retrieval mechanism is used to first recall the candidate set using BM25 and then refine the ranking using DenseRetriever to retrieve Top-K relevant evidence fragments from the operation and maintenance knowledge base; A causal reasoning engine based on a directed acyclic graph for key equipment of offshore wind turbines is constructed to calculate the return rate of intervention actions and attach the optimal intervention action as a hard constraint to the generation Prompt of the large model. The retrieved evidence is concatenated with the fault context and input into a large language model to generate executable operation and maintenance suggestions. The generated operation and maintenance suggestions are then given a comprehensive score for their feasibility.
8. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 1, characterized in that, The fusion decision module is configured as follows: Feature vectors are extracted from each data modality using a corresponding feature encoder. A dynamic attention weighting mechanism is then used for multimodal feature fusion, with the fusion formula as follows: In the formula, alpha m (t) represents the attention weight of the m-th mode at time t, A m Let be the mapping function from each modal feature to the unified fusion space. Let M be the input feature vector of the m-th mode at time t, and M be the total number of modes participating in the fusion. A confidence decay mechanism is implemented for missing modalities. When a missing or abnormal modal data is detected, the corresponding attention weight is adjusted according to the time decay factor, and the weight of the missing modality is redistributed to the other available modalities. The probability of offshore units reaching their destination within a future time window is calculated using a multi-factor logistic regression model, with the following formula: In the formula, For the Sigmoid function, v t H is the wind speed forecast value. t For the effective wave height forecast value, T t B represents the seawater temperature. t The gamma0, gamma1, gamma2, gamma3, and gamma4 represent the available status of the vessels under maintenance, and the gamma4 are the regression coefficients corresponding to the input variables in the multivariate logistic regression model. The reachable probability P acc (t) Constructing a risk-reachability decision matrix combined with the fault severity level Severity, realizing the automatic output of hierarchical disposal strategy and structured operation and maintenance work order.
9. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 1, characterized in that, The edge-cloud collaboration module is configured as follows: Deploy a lightweight early warning model that has undergone knowledge distillation and quantization compression at the edge to perform local fault detection and preliminary risk classification in real time; When the communication link is normal, an adaptive fusion strategy is used to merge the detection results from the edge and the cloud. The fusion formula is as follows: In the formula, the fusion weight λ t Quality of Service (QoS) of the communication link t Adaptive determination, This represents the prediction result output by the edge node at time t. This represents the prediction result output by the cloud model at time t. Under communication interruption conditions, the edge node switches to a fully offline early warning mode, independently completes alarm judgment, generates preliminary handling suggestions and emergency event records, and after the link is restored, the data is batch synchronized to the cloud for review and verification.
10. The offshore wind turbine fault early warning and operation and maintenance system based on a multimodal large model according to claim 1, characterized in that, The closed-loop learning module is configured as follows: An end-to-end auditable chain of decision-making evidence is formed based on work order execution feedback; A fault back-labeling closed-loop mechanism is adopted. When a unit is confirmed to have an actual fault by a work order, the data of N days before the fault is automatically traced back, the remaining life is labeled according to the time step, and a fault back-labeling sample library is established as a priori reference for model fine-tuning. An incremental learning strategy is adopted to update the sample pool and model parameters. The incremental training loss function includes the task loss for new samples, the experience replay loss, and the KL divergence regularization term for the output distribution of the new and old models. It enables dynamic experience self-learning, vectorizes completed work orders in real time and injects them into the knowledge base, and uses a time-sensitivity decay factor to adjust the weight of knowledge fragments.
