AI Fusion Processing Method and System for Multi-Source Heterogeneous Data
By using an AI fusion processing method for multi-source heterogeneous data, combined with UAV images and fiber optic sensor data, the system can accurately distinguish between icing weight gain and structural damage to offshore wind turbine blades, solving the misjudgment problem in existing technologies and improving the system's recognition capability and operational reliability.
Patent Information
- Application Number
- CN202611324436.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
When offshore wind turbine blades are icy, visual images are obstructed and vibration signals are complex. Existing technologies cannot accurately distinguish between the weight gain from icing and structural damage, leading to misjudgment or neglect of actual cracks by the control system, resulting in power generation loss or blade breakage.
A multi-source heterogeneous data AI fusion processing method is adopted. By using UAV images and fiber optic sensor vibration waveforms, AI models are used to perform visual feature analysis and vibration feature analysis. Combined with AI fusion logic, the blade state is determined and corresponding control commands are generated.
Accurately identifying icing weight gain and structural damage under conditions of visual obstruction and confused vibration characteristics can avoid misjudgment and shutdown, prevent blade breakage, and improve the operational reliability of offshore wind turbines.
Smart Images

Figure CN123109979A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of wind power generation control technology, and in particular to AI fusion processing methods and systems for multi-source heterogeneous data. Background Technology
[0002] Offshore wind farms are typically deployed in areas far from land, where the turbine blades operate in a high-humidity, low-temperature marine environment for extended periods. On cold winter nights, moisture in the air easily condenses and freezes on the massive turbine blades. Because the ice layer is transparent and unevenly distributed, when drones equipped with cameras inspect the blades at night, the light from the supplemental lighting is strongly refracted and reflected on the ice surface, resulting in large areas of white, hazy light spots in the captured images. This visual interference prevents the inspection equipment from using image recognition technology to observe cracks on the blade surface or the true structural condition beneath the ice.
[0003] Furthermore, while fiber optic sensors installed inside the blades can collect vibration waveform data, the increased weight after icing and the reduced stiffness after structural damage both alter vibration characteristics. Existing data processing methods typically analyze visual images or vibration signals separately, making it difficult to accurately distinguish whether abnormal vibrations are caused by the added weight of ice or by internal blade structural fractures, especially under conditions of visual obstruction and complex superimposed interference. This leads the control system to easily misinterpret icing-induced vibration changes as structural damage, triggering unnecessary shutdowns and causing power generation losses, or to ignore actual cracks hidden by ice, causing the blades to operate with defects until they break. Summary of the Invention
[0004] In view of the aforementioned problems, this application is hereby filed.
[0005] Therefore, this application provides an AI fusion processing method and system for multi-source heterogeneous data, which can accurately distinguish between icing weight gain and structural damage under conditions of visual obstruction and confusion of vibration characteristics.
[0006] To solve the above-mentioned technical problems, this application provides the following technical solution: In the first aspect, this application provides an AI fusion processing method for multi-source heterogeneous data, including: acquiring multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms, wherein the multi-source heterogeneous data includes blade images captured by drones and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades; The visual feature analysis of the blade image is performed using an AI model to obtain visual feature data, and the vibration feature analysis of the blade vibration waveform is performed using the AI model to obtain vibration feature data. The visual feature data and the vibration feature data are fused and determined based on AI fusion logic to identify the fusion status of the wind turbine blades. In response to the fusion status indicator indicating a risk of damage, a target control command matching the fusion status indicator is generated and sent to the wind turbine main control system.
[0007] Preferably, the acquisition of multi-source heterogeneous data corresponding to offshore wind turbine blades includes: The image of the blade captured by the drone is acquired, and the vibration waveform of the blade collected by the fiber optic sensor is acquired simultaneously. Determine the location data of the shooting point of the blade image on the wind turbine blade; Based on the shooting point location data, a vibration segment corresponding to the shooting point location is extracted from the blade vibration waveform; Associate the blade image with the vibration segment and verify whether the acquisition time of the blade image and the acquisition time of the vibration segment meet the synchronization condition; In response to the satisfaction of the synchronization condition, the associated blade image and the vibration segment are identified as the multi-source heterogeneous data.
[0008] Preferably, the AI fusion processing method for multi-source heterogeneous data further includes: Acquire reference vibration waveforms from reference sensors installed at the top of the tower; Common-mode components are removed from the blade vibration waveform and the reference vibration waveform to obtain a pure vibration waveform. The pure vibration waveform is determined as the blade vibration waveform.
[0009] Preferably, the step of using an AI model to perform visual feature analysis on the leaf image to obtain visual feature data includes: The AI model is used to analyze the pixel brightness distribution in the leaf image to determine whether there are overexposed areas where the pixel brightness value is higher than the preset brightness condition. In response to the existence of the overexposed area, and the presence of a halo area at the edge of the overexposed area whose pixel color satisfies a preset rainbow color condition, a visual occlusion mark is generated. The location range data of the overexposed area and the visual occlusion marker are determined as the visual feature data.
[0010] Preferably, the step of using the AI model to perform vibration feature analysis on the blade vibration waveform to obtain vibration feature data includes: The AI model is used to perform frequency component analysis on the blade vibration waveform to determine the dominant frequency shift direction and high-frequency harmonic content; In response to the main frequency offset direction pointing to the frequency reduction direction, and the high-frequency harmonic content not exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as mass gain characteristics. In response to the high-frequency harmonic content exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as stiffness loss characteristics. The mass gain characteristic or the stiffness loss characteristic is determined as the vibration characteristic data.
[0011] Preferably, the step of fusing the visual feature data and the vibration feature data based on AI fusion logic to determine the fusion status identifier of the wind turbine blade includes: The visual feature data is determined to contain visual occlusion markers; In response to the vibration feature data being a mass gain feature, the AI fusion logic is used to determine that the wind turbine blade is in a pure icing state, and the pure icing state is identified as the fusion state identifier. In response to the vibration feature data being stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in a state of ice cover damage, and the damage risk state is determined as the fusion state identifier. Since the vibration feature data simultaneously includes mass gain features and stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in a state of icing and concurrent damage, and the severe damage risk state is identified as the fusion state identifier.
[0012] Preferably, the step of generating a target control command matching the fusion status identifier in response to the fusion status identifier indicating a risk of damage includes: In response to the fusion status being identified as a damage risk state or a severe damage risk state, an emergency stop command is generated, and the emergency stop command is determined as the target control command; In response to the fusion state being identified as a pure icing state, a de-icing operation command is generated; Obtain the current rotational speed data of the wind turbine blades, and determine the pitch angle adjustment amount based on the current rotational speed data; The de-icing operation command is generated based on the pitch angle adjustment amount, and the de-icing operation command is determined as the target control command.
[0013] Preferably, the AI fusion processing method for multi-source heterogeneous data further includes: After sending the target control command to the wind turbine main control system, the execution feedback signal after the wind turbine main control system executes the target control command is obtained; Based on the execution feedback signal, the updated blade vibration waveform collected by the fiber optic sensor is obtained; The AI model is used to analyze the updated blade vibration waveform and generate a secondary confirmation status identifier. In response to the secondary confirmation status indicating that the risk has not been eliminated, a long-term lock flag is generated and written to the operation log of the wind turbine blade.
[0014] Preferably, the step of analyzing the updated blade vibration waveform using the AI model to generate a secondary confirmation status identifier includes: In response to the target control command being a de-icing operation command, the main control system of the wind turbine is controlled to perform a disturbance action on the wind turbine blades; Acquire the transient vibration response data of the wind turbine blades when they perform the disturbance action; The AI model was used to analyze the decay rate of the transient vibration response data; In response to the decay rate meeting the preset health model conditions, a risk clearance flag is generated; In response to the decay rate not meeting the preset health model conditions, a risk persistence indicator is generated; The risk relief flag or the risk persistence flag is determined as the secondary confirmation status flag.
[0015] Secondly, this application also provides an AI fusion processing system for multi-source heterogeneous data, including: a data acquisition module for acquiring multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms, wherein the multi-source heterogeneous data includes blade images captured by drones and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades. The AI feature analysis module is used to perform visual feature analysis on the blade image using an AI model to obtain visual feature data, and to perform vibration feature analysis on the blade vibration waveform using the AI model to obtain vibration feature data. The AI fusion judgment module is used to perform fusion judgment on the visual feature data and the vibration feature data based on AI fusion logic to determine the fusion status identifier of the wind turbine blade. The control execution module is used to generate a target control command that matches the fusion status indicator in response to the indication of damage risk, and send the target control command to the wind turbine main control system.
[0016] Implementing this application will have the following beneficial effects: This application provides an AI fusion processing method and system for multi-source heterogeneous data. By fusing visual feature data and vibration feature data, this invention utilizes an AI model to accurately identify mass gain and stiffness loss characteristics based on the frequency shift direction and high-frequency harmonic content of the vibration waveform, even under conditions of visual obstruction. This processing method effectively eliminates the visual failure effects caused by nighttime light interference and ice cover, overcoming the limitation of single vibration signals in distinguishing between icing weight gain and structural damage. The system can accurately identify real cracks hidden by ice or accurately determine a pure icing state, avoiding unnecessary shutdowns due to misjudged structural damage and preventing blade breakage accidents caused by ignoring actual damage, thus improving the operational reliability of offshore wind turbines in harsh environments. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart of the AI fusion processing method for multi-source heterogeneous data involved in this application; Figure 2 This is a schematic diagram of the acquisition and preprocessing process of multi-source heterogeneous data in the AI fusion processing method for multi-source heterogeneous data involved in this application; Figure 3 This is a schematic diagram illustrating the process of using an AI model to perform visual and vibration feature analysis in the AI fusion processing method for multi-source heterogeneous data involved in this application. Figure 4 This is a schematic diagram of the AI fusion logic-based judgment and control instruction generation logic of the AI fusion processing method for multi-source heterogeneous data involved in this application.
[0019] Figure 5 This is a schematic diagram of the feedback and secondary confirmation mechanism after the execution of control instructions in the AI fusion processing method for multi-source heterogeneous data involved in this application. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0021] Example 1, referring to Figures 1-5This is the first embodiment of the present application, which provides an AI fusion processing method for multi-source heterogeneous data, including: This application provides a method that can effectively solve the problems mentioned above. The following will describe in detail how to implement the AI fusion processing method for multi-source heterogeneous data with multiple embodiments. Figure 1 A flowchart illustrating an AI fusion processing method for multi-source heterogeneous data is provided, including: Step: S1, acquire multi-source heterogeneous data corresponding to the wind turbine blades of the offshore wind farm. The multi-source heterogeneous data includes blade images taken by UAVs and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades. It should be noted that offshore wind farms operate in complex meteorological environments characterized by high humidity and high salt spray. During long-term operation, wind turbine blades are highly susceptible to external environmental erosion or internal material aging, leading to potential hazards. To accurately assess blade condition, it is necessary to acquire images of the blade surface taken by drones during inspections, and simultaneously acquire vibration waveforms collected by fiber optic sensors embedded within the blades. In this scenario, multi-source heterogeneous data refers to two sets of signals with completely different data formats and physical sources. The blade images captured by the drones are two-dimensional pixel matrix structures, transmitted back to the ground base station via a radio digital microwave link, and their data format is a compressed byte stream as specified by the Joint Image Experts Group. The blade vibration waveforms collected by the fiber optic sensors are one-dimensional time-series discrete voltage values, connected to an opto-demodulator via fiber optic patch cords. This demodulator converts the optical wavelength offset into an analog voltage signal, which is then quantized into a digital sequence by an analog-to-digital converter. Both exhibit heterogeneity in data structure, transmission medium, and physical dimensions; therefore, they are collectively referred to as multi-source heterogeneous data.
[0022] In this embodiment, the multi-source heterogeneous data includes high-resolution visual images captured by the UAV at different spanwise positions of the blade, and waveform data reflecting blade structural deformation collected in real time by fiber Bragg grating sensors laid along the inner skin of the blade. These two types of data reflect the current condition of the blade from the macroscopic surface visual dimension and the microscopic internal physical dimension, respectively. The fiber Bragg grating sensor is an optical measurement device in the prior art, which contains a grating region with periodically modulated refractive index. When the blade is strained, the change in grating spacing causes a shift in the center wavelength of the reflected light. The photoelectric demodulator captures this wavelength shift through an internal spectrometer or unbalanced Michelson interferometer architecture and linearly maps it into a voltage signal output.
[0023] Furthermore, jointly acquiring these two data sources as multi-source heterogeneous data sources can overcome the misjudgment problem that easily occurs with single data sources under complex weather conditions, providing fundamental data support for subsequent feature extraction and fusion state determination. In practical applications, image data captured by UAVs has high-resolution pixel matrix characteristics, with each pixel containing color quantization values for red, green, and blue channels, ranging from 0 to 255 unsigned integers. Meanwhile, waveform data acquired by fiber optic sensors has high-frequency discrete time series characteristics, with each sampling point containing a timestamp and the corresponding voltage amplitude, where the voltage amplitude is a floating-point number in volts.
[0024] In some embodiments, acquiring the multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms may further include the following steps: S1.1, acquire the image of the blade captured by the UAV, and simultaneously acquire the vibration waveform of the blade collected by the fiber optic sensor; S1.2, determine the location data of the shooting point of the blade image on the wind turbine blade; S1.3, Based on the shooting point location data, extract the vibration segment corresponding to the shooting point location from the blade vibration waveform; S1.4, associate the blade image with the vibration segment, and verify whether the acquisition time of the blade image and the acquisition time of the vibration segment meet the synchronization condition; S1.5, in response to satisfying the synchronization condition, the associated blade image and the vibration segment are determined as the multi-source heterogeneous data.
[0025] It is understandable that steps S1.1 to S1.5 constitute a complete operational sequence for multi-source data spatiotemporal alignment. Step S1.1 is responsible for the parallel acquisition of the raw data. The UAV's flight trajectory and the spatial distribution of the blades exist in their own coordinate systems. To ensure that the surface visual information and the internal physical vibration information can accurately correspond to the same local position on the blades, spatial and temporal alignment processing is required for the two independent data sources. The UAV records its current position through its onboard flight control module, which integrates a GPS receiver chip. This chip calculates the UAV's current longitude, latitude, and relative altitude by capturing pseudo-random codes broadcast by navigation satellites.
[0026] Specifically, as the drone flies along the spanwise direction of the wind turbine blade, it captures high-resolution images of the blade surface in real time. Each time the shutter is pressed to capture an image, the onboard flight control module records a GPS message at that instant. The latitude, longitude, and relative altitude data from the message are extracted and combined with the wind turbine tower base coordinates pre-stored in the flight control memory. Using the Pythagorean theorem in three-dimensional space, the straight-line spatial distance between the drone and the tower base station is calculated. This spatial distance is then projected onto the spanwise distance coordinate system of the blade using a pre-defined three-dimensional geometric model of the blade, calculating the distance in meters from the blade root to the image location on the wind turbine blade. This determines the image's location on the wind turbine blade. The location data is a structured data point containing spanwise distance in meters and a percentage of the chordwise position.
[0027] Furthermore, for the continuous long waveform data acquired by the fiber optic sensor, based on the spanwise distance (in meters) in the previously determined image capture point location data, a sensor deployment location mapping table pre-stored on the industrial control computer's hard drive is retrieved. This mapping table is a lookup table with two columns: the first column is the blade spanwise distance (in meters), and the second column is the unique identifier and data channel number of the fiber Bragg grating sensor corresponding to that distance. A specific example of this mapping table is shown in the table below: Spanning distance (meters) Raster Identity Identifier Data Channel Number 0.0 FBG-ROOT CH-01 5.0 FBG-SEG1 CH-02 10.0 FBG-SEG2 CH-03 15.0 FBG-SEG3 CH-04 20.0 FBG-TIP CH-05 Based on the spanwise distance in the shooting point location data, the row with the closest value is found in the first column. The grating identifier and data channel number in the second column of that row are read. Then, a waveform data segment with a preset time length of 2 seconds corresponding to the data channel number is extracted from the blade vibration waveforms continuously acquired from multiple channels as the vibration segment corresponding to the shooting point location.
[0028] Furthermore, the acquired blade images and the extracted vibration segments are data-bound by being labeled with the same association tag, which is an incrementing unsigned integer, for example, starting from 0 and incrementing by 1 each time. Then, the timestamp of the image acquisition is compared with the timestamp of the vibration segment's start, and the absolute value of the difference between the two timestamps is calculated, with the unit being milliseconds. It is then determined whether this absolute value of the difference is within a preset time deviation tolerance range to verify whether the two satisfy the synchronization condition. The time deviation tolerance range is defined as a closed interval consisting of a lower limit value and an upper limit value.
[0029] For example, during a certain inspection, a drone took an image of a leaf 15 meters from its root at 10:05:03:500 AM. The fiber optic sensor collected the waveform at a frequency of 1000 Hz. The waveform segment corresponding to the grating channel CH-04 at 15 meters was extracted from the waveform. The starting timestamp of this segment was 10:05:03:530, and the absolute value of the difference was 30 milliseconds. If the preset time deviation tolerance range has a lower limit of 10 milliseconds and an upper limit of 100 milliseconds, then 30 milliseconds is within this range and meets the synchronization condition.
[0030] It should be noted that the preset time deviation tolerance range is used to accommodate the inherent time difference between the UAV flight control clock and the fiber optic acquisition card hardware clock, as well as the data transmission delay. It is obtained by statistically analyzing the frequency error between the UAV flight control system clock crystal oscillator and the fiber optic acquisition system clock crystal oscillator, as well as the communication network transmission delay. The preferred setting range is 10 to 100 milliseconds. In the specific operation of this application, the communication delay parameters between the UAV and the fiber optic acquisition device are obtained. If the communication delay parameter is greater than 10 milliseconds, the upper limit of the preset time deviation tolerance range is set to 80 milliseconds; if the communication delay parameter is less than or equal to 10 milliseconds, the upper limit of the preset time deviation tolerance range is set to 20 milliseconds.
[0031] In practical applications, in response to the absolute value of the difference satisfying the time deviation tolerance range, the associated blade image and vibration segment are formally packaged into a data tuple. This data tuple is stored in memory in the form of a continuous address block, with image data first and waveform data second. It is determined as multi-source heterogeneous data for subsequent analysis, ensuring that the data relied upon in the feature extraction stage has strict consistency in physical space and occurrence time.
[0032] In this application, if the verification result does not meet the synchronization condition, the combination of the current image and the corresponding waveform is discarded, and the drone is triggered to hover and shoot again and to re-execute the timestamp comparison and truncation operation to prevent misaligned data from polluting the subsequent fusion judgment feature library.
[0033] Step S2: Use the AI model to perform visual feature analysis on the blade image to obtain visual feature data, and use the AI model to perform vibration feature analysis on the blade vibration waveform to obtain vibration feature data.
[0034] It should be noted that after acquiring spatially and temporally aligned multi-source heterogeneous data, feature parameters of different dimensions need to be extracted through analysis models to provide independent data support for subsequent fusion and judgment. The AI model is a set of feature judgment execution logic based on conditional branching statements and arithmetic operations, deployed in an edge computing gateway. This logic set is written in C language or programmable logic controller ladder diagram form, receiving structured pixel matrices and discrete waveform sequences as input, and outputting feature flags through fixed numerical comparisons and arithmetic operations. The edge computing gateway is an industrial-grade computer with a central processing unit and memory chips, executing compiled machine instructions to complete the model logic.
[0035] In this application, the analysis model performs visual feature analysis on the blade image and vibration feature analysis on the blade vibration waveform. The visual feature analysis mainly focuses on the brightness distribution and color edge morphology of pixels in the image, aiming to identify whether there are abnormal reflections or obstructions on the blade surface.
[0036] In practical applications, vibration feature analysis mainly focuses on the frequency composition and energy distribution of waveform data, aiming to identify whether the internal structure of the blade has undergone changes in mass distribution or stiffness reduction.
[0037] On the other hand, after the two types of features are extracted independently, visual feature data and vibration feature data are generated and output respectively, laying a data foundation for multi-dimensional joint assessment of blade health status and avoiding the limitations of a single data source under complex sea conditions.
[0038] In some embodiments, the step of using an AI model to perform visual feature analysis on the leaf image to obtain visual feature data may further include the following steps: S2.1, Analyze the pixel brightness distribution in the leaf image using the AI model to determine whether there are overexposed areas where the pixel brightness value is higher than the preset brightness condition; S2.2, in response to the existence of the overexposed area and the presence of a halo area at the edge of the overexposed area whose pixel color satisfies the preset rainbow color condition, a visual occlusion mark is generated. S2.3, the location range data of the overexposed area and the visual occlusion mark are determined as the visual feature data.
[0039] It should be noted that steps S2.1 to S2.3 constitute a complete operational sequence for visual feature extraction. Offshore wind farms are often located in areas where water vapor condenses. When ice or water film adheres to the blade surface, strong optical refraction and dispersion occur under sunlight. Therefore, the surface occlusion can be determined by identifying specific highlights and rainbow halos in the image.
[0040] Specifically, the process involves reading leaf images captured by a drone, scanning row by row and column by column according to pixel coordinates, extracting the red, green, and blue channel values for each pixel, adding the three channel values and dividing by three to obtain the pixel's brightness value. A two-dimensional array is constructed according to the image resolution, and the brightness value of each pixel is filled into the corresponding row and column position, thus forming the pixel brightness distribution matrix of the entire image. The pixel brightness distribution matrix is traversed, and the brightness value of each pixel is compared with a preset brightness condition. If the brightness values of all pixels in a continuous block are greater than the preset brightness condition, the continuous block is marked as an overexposed area, and the minimum horizontal coordinate, minimum vertical coordinate, maximum horizontal coordinate, and maximum vertical coordinate of the overexposed area in the image coordinate system are recorded. These four coordinate values are used as the location range data of the overexposed area.
[0041] Furthermore, after identifying an overexposed area, a pixel band with a width of 5 pixels is extended outward from the edge of the overexposed area, and the red, green, and blue channel values of all pixels within this pixel band are extracted. For adjacent pixels in the pixel band along the clockwise direction from the edge of the overexposed area, the differences in the red, green, and blue components between the next pixel and the previous pixel are calculated. If, along the clockwise direction from the edge of the overexposed area, the difference in the red component is negative, the difference in the green component is positive, and the difference in the blue component is positive, this data feature of alternating red, green, and blue components satisfies the preset rainbow color condition, indicating that there is a rainbow halo region at the edge of the overexposed area caused by light refraction. Based on this, a Boolean variable is set to true, generating a visual occlusion flag. The visual occlusion flag is a 1-byte storage unit, with a value of 1 representing true and a value of 0 representing false.
[0042] Furthermore, the location range data of the overexposed area and the generated visual occlusion marker are packaged and combined into a structure data, which is determined as the final visual feature data. This structure is arranged sequentially in memory, with the first 8 bytes storing four coordinate values, each occupying 2 bytes, and the 9th byte storing a Boolean visual occlusion marker.
[0043] Furthermore, the preset brightness condition is used to determine whether the image is overexposed. It is obtained based on statistics of the full-capacity and gain parameters of the drone camera's image sensor, with a preferred setting range of 200 to 240 grayscale values. In the specific operation of this application, the ambient light level of the current shooting environment is obtained. If the ambient light level is greater than 10,000 lux, the preset brightness condition is set to a grayscale value of 235; if the ambient light level is less than or equal to 10,000 lux, the preset brightness condition is set to a grayscale value of 210.
[0044] For example, if the red, green and blue components of a certain pixel are 200, 150 and 100 respectively, and the components of the clockwise adjacent pixels are 180, 170 and 120, the difference in red components is -20, the difference in green components is +20 and the difference in blue components is +20, this change characteristic meets the preset rainbow color conditions.
[0045] In this embodiment, the preset rainbow color condition is used to identify optical dispersion caused by the ice-water mixture. It is obtained based on experimental data of ice crystal refractive index and visible light wavelength decomposition. The preferred setting range is where the absolute value of the change in adjacent differences between the red, green, and blue components is between 5 and 20 gray values per pixel. In the specific operation of this application, the contrast parameter of the current image is obtained. If the contrast parameter is higher than the preset contrast upper limit, the preset rainbow color condition is set to an absolute value of 8 gray values per pixel for the change in adjacent differences. If the contrast parameter is lower than or equal to the preset contrast upper limit, the preset rainbow color condition is set to an absolute value of 15 gray values per pixel for the change in adjacent differences.
[0046] It should be noted that all pixel comparisons and difference calculations are performed by the arithmetic logic unit inside the central processing unit of the edge computing gateway. This unit contains adders, subtractors, and shift registers, and can complete one arithmetic operation in one clock cycle.
[0047] Specifically, the construction of the aforementioned pixel brightness distribution matrix and the calculation of the difference belong to the existing technology in the field of basic digital image processing. This application combines it with the specific optical refraction physical characteristics of sea ice to form a set of judgment logic that can be implemented through basic operations.
[0048] In some embodiments, the step of using the AI model to perform vibration feature analysis on the blade vibration waveform to obtain vibration feature data may further include the following steps: S2.4, The AI model is used to perform frequency component analysis on the vibration waveform of the blade to determine the direction of the main frequency shift and the content of high-frequency harmonics; S2.5, in response to the main frequency offset direction pointing to the frequency reduction direction, and the high frequency harmonic content not exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as mass gain characteristics. S2.6, In response to the high-frequency harmonic content exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as stiffness loss characteristics; S2.7, the mass gain characteristic or the stiffness loss characteristic is determined as the vibration characteristic data.
[0049] It should be noted that steps S2.4 to S2.7 constitute the complete operational sequence for vibration feature extraction. Wind turbine blades have a fixed natural vibration frequency during normal operation. When ice forms on the blade surface, increasing the overall mass, the vibration frequency decreases. However, when cracks or delamination occur in the internal material of the blade, leading to a decrease in stiffness, additional high-frequency vibration components are excited.
[0050] In this embodiment, frequency component analysis is used to analyze the captured blade vibration waveform. This involves multiplying the discrete waveform data in the time domain point-by-point with a preset frequency sine and cosine wave reference sequence and summing the results to calculate the energy amplitude of each frequency component. The frequency component with the largest amplitude is then extracted as the dominant frequency. This dominant frequency value is obtained, and the baseline dominant frequency value recorded under historical normal conditions is subtracted to obtain the dominant frequency difference. If the dominant frequency difference is negative, the dominant frequency offset direction points towards a decreasing frequency direction. The sum of the squares of the energy amplitudes of all frequency components within the preset high-frequency range is calculated and used as the high-frequency harmonic content. This operation is implemented using the mathematical principle of the Discrete Fourier Transform. Specifically, it iterates through the preset frequency sequence. For each target frequency, a corresponding sine and cosine discrete sequence is generated. The original waveform sequence is multiplied point-by-point with the sine discrete sequence and summed to obtain the imaginary energy. The original waveform sequence is then multiplied point-by-point with the cosine discrete sequence and summed to obtain the real energy. The square of the real energy is added to the square of the imaginary energy, and then the square root is taken to obtain the amplitude of the target frequency.
[0051] Furthermore, the direction of the dominant frequency offset is compared with a preset offset threshold. If the dominant frequency offset direction points towards a decreasing frequency, it indicates that the blade has become heavier overall. In this case, it is further determined whether the high-frequency harmonic content exceeds a preset harmonic condition. If the high-frequency harmonic content does not exceed the preset harmonic condition, it indicates that there is only a mass increase without material fracture, and the characteristic of the blade vibration waveform is assigned as a mass increase feature. The mass increase feature is represented by a constant 1 in memory.
[0052] Furthermore, if the high-frequency harmonic content exceeds the preset harmonic condition, it indicates that the blade has suffered a local stiffness loss, causing high-frequency abnormal noise. In this case, the characteristic of the blade vibration waveform is assigned as a stiffness loss characteristic. The stiffness loss characteristic is represented by a constant 2 in memory.
[0053] Furthermore, by using the determined mass gain characteristics or stiffness loss characteristics as the final vibration characteristic data output, the causes of blade anomalies can be distinguished from the physical root, providing clear physical parameter dimensions for subsequent fusion judgment.
[0054] For example, if the normal reference frequency of a blade is 2 Hz, the current measured frequency is 1.5 Hz, the frequency difference is -0.5 Hz, the frequency offset direction is downward, the sum of squares of the energy amplitude in the high frequency range is 30 volts squared, the preset harmonic condition is 50 volts squared, and the high frequency harmonic content does not exceed the standard, then it is determined to be a weight gain characteristic.
[0055] In this application, the preset harmonic condition is used to determine whether abnormal high-frequency vibrations caused by stiffness attenuation occur inside the blade. It is obtained by statistically analyzing the baseline high-frequency background noise during non-destructive testing of the wind turbine blades at the factory. The preferred setting range is 50 to 100 volts squared. In the specific operation of this application, the current operating speed of the wind turbine blades is obtained. If the operating speed is within the rated speed range, the preset harmonic condition is set to 80 volts squared; if the operating speed is below the rated speed range, the preset harmonic condition is set to 60 volts squared.
[0056] It is understandable that the frequency component analysis mentioned above belongs to the classic digital signal processing technology, which relies on hardware modules such as multipliers, accumulators and square root operators at its core, without the need for complex proprietary networks.
[0057] Step S3: Based on AI fusion logic, perform fusion judgment on the visual feature data and the vibration feature data to determine the fusion status identifier of the wind turbine blade; After acquiring visual and vibration feature data, single-dimensional features often only reflect one aspect of the blade's appearance. For example, visual features can only reveal surface anomalies, while vibration features can only reveal internal physical changes. Therefore, a fusion judgment logic is needed to cross-compare the two. The AI fusion logic is a set of conditional selection statements deployed in a microprocessor, physically implemented as multiple parallel numerical comparators and AND gates. Based on the presence or absence of visual occlusion markers and the specific category of vibration features, the fusion judgment logic constructs a two-dimensional joint judgment matrix. The two types of feature data are used as row and column numbers as inputs to the matrix, and the corresponding output enumeration values are directly obtained through a traversal search operation.
[0058] In this embodiment, the aforementioned two-dimensional joint decision matrix is stored in a read-only memory. The row index of the matrix is the vibration feature category code, the column index is the Boolean value of the visual occlusion identifier, and the data stored in the matrix is the fusion state identifier code. The content of the matrix is pre-written by engineers based on the principles of aeroelasticity before the industrial control computer leaves the factory.
[0059] Furthermore, based on the matching results, a fusion status identifier reflecting the current true physical state of the blades is output, thereby providing the most direct and accurate basis for determining which control strategy the wind turbine should adopt, avoiding unnecessary shutdowns or operation with damage due to misjudgment from a single data source.
[0060] In some embodiments, the step of fusing the visual feature data and the vibration feature data based on AI fusion logic to determine the fusion status identifier of the wind turbine blade may further include the following steps:
[0061] S3.1, Determine that the visual feature data contains a visual occlusion identifier; S3.2, In response to the vibration feature data being a mass gain feature, the AI fusion logic is used to determine that the wind turbine blade is in a pure icing state, and the pure icing state is identified as the fusion state identifier; S3.3, In response to the vibration feature data being stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in a state of ice covering damage, and the damage risk state is determined as the fusion state identifier. S3.4 In response to the vibration feature data simultaneously containing mass gain features and stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in an icing and damage state, and the severe damage risk state is identified as the fusion state identifier.
[0062] Understandably, steps S3.1 to S3.4 constitute the complete operational sequence for fusion determination. Icing on offshore wind farm blades not only alters their aerodynamic shape but may also mask internal fatigue cracks. Combining the determination process can strip away superficial interference and identify the true risk level.
[0063] Specifically, read the 9th byte of the aforementioned output visual feature data structure, parse the value of the Boolean variable stored in the byte, and if the value is 1, it is determined that the visual feature data contains a visual occlusion marker, indicating that the leaf surface is covered with reflective occlusions such as ice or water film.
[0064] Furthermore, the vibration feature data associated with the visual feature data is retrieved, and its feature category value is input into the conditional branch module of the fusion determination logic. When the vibration feature data is a mass increase feature, its category code is 1. In the row where the first column and the second column of the preset fusion determination lookup table are both 1, the fusion state identifier code in the third column is read as 01, which corresponds to a pure icing state. At this time, the fusion determination logic determines that the wind turbine blade is in a pure icing state, indicating that the blade has only increased in mass due to icing and has not suffered structural damage. A specific example of this lookup table is shown in the table below: Vibration feature category code Boolean value for visual occlusion indicator Fusion status code Corresponding state description 1 1 01 Purely frozen state 2 1 10 Ice covers the state of damage 3 1 11 Ice-related damage state 1 0 00 Normal state Furthermore, when the vibration feature data is a stiffness loss feature with a category code of 2, in the row where the first column of the lookup table is 2 and the second column is 1, the fusion state identifier code in the third column is read as 10. This code corresponds to the damage risk state. This indicates that the internal stiffness of the blade has decreased and is accompanied by high-frequency abnormal noise. Surface icing actually masks the internal structural damage. At this time, the fusion judgment logic is used to determine that the wind turbine blade is in a state of icing-covered damage, and the damage risk state is output as the fusion state identifier.
[0065] Specifically, since different spanwise positions of the same blade may exhibit different physical states, when different vibration segments are analyzed to reveal mass gain characteristics and stiffness loss characteristics, the codes for both are combined using a bitwise OR operation to form category code 3. A bitwise OR operation involves converting two values to binary representations and then performing a logical OR operation bit by bit; if any binary bit is 1, the corresponding bit in the result is 1. In the row where the first column of the lookup table is 3 and the second column is 1, the fusion status identifier code in the third column is read as 11. This code corresponds to a severe damage risk state. This indicates that the blade is not only severely iced but also has experienced concurrent material damage. In this case, the fusion determination logic is used to determine that the wind turbine blade is in an icing and concurrent damage state, and the severe damage risk state is output as the fusion status identifier.
[0066] In this application, the mapping relationship in the lookup table is obtained based on the aeroelasticity model of the wind turbine blades and statistical analysis of historical fault sample data. The preferred setting range covers four typical combinations of visual and vibration characteristics. In the specific operation of this application, the current operating age of the wind turbine blades is obtained. If the operating age is greater than 5 years, a priority mapping rule for stiffness loss characteristics is added to the lookup table. That is, when the vibration characteristic code is 3, regardless of the visual identifier, code 11 is forcibly output. If the operating age is less than or equal to 5 years, the default equal mapping rule is maintained.
[0067] Understandably, this multi-dimensional fusion judgment process effectively solves the engineering problem of visual inspection of offshore wind farms being easily affected by light interference, leading to missed internal damage.
[0068] It should be noted that by replacing complex implicit neural network reasoning with table lookup and bitwise logical operations, the decision-making process can be debugged and traced step by step by engineers, and non-technical personnel can also reproduce the logic by writing simple switch-case statements.
[0069] Furthermore, the storage medium of the lookup table is an electrically erasable programmable read-only memory, which supports rewriting and updating via a maintenance computer during wind turbine shutdown maintenance.
[0070] Step S4: In response to the fusion status indicator indicating a risk of damage, generate a target control command that matches the fusion status indicator and send the target control command to the wind turbine main control system.
[0071] It should be noted that after determining the fusion status identifier, differentiated control intervention measures must be taken according to different risk levels to maintain power generation efficiency as much as possible while ensuring the safety of the wind turbine. In this process, generating control commands refers to the microprocessor retrieving the corresponding standard function code from memory and combining it with the status identifier to form a data frame conforming to the industrial communication protocol. The wind turbine main control system is an industrial control unit containing a programmable logic controller (PLC), which runs a real-time operating system and executes corresponding physical actions by parsing the received data frames.
[0072] In practical applications, the risk level indicated by the fusion status flag is analyzed. If it indicates a non-destructive risk caused by purely environmental factors, a de-icing operation function code is loaded from the register. If it indicates a risk of structural damage, an emergency stop function code must be loaded from the register to cut off the load input.
[0073] On the other hand, the generated target control commands are packaged into binary messages conforming to industrial network communication standards and sent to the programmable logic controller (PLC) receive register of the wind turbine main control system via a fiber optic ring network switch, triggering the corresponding hardware action execution loop. This binary message contains a message header, function code, data payload, and cyclic redundancy check (CRC) code.
[0074] Furthermore, this closed-loop control process ensures the timely conversion from data perception to physical intervention, thus preventing catastrophic accidents.
[0075] In some embodiments, the step of generating a target control command matching the fusion status identifier in response to the fusion status identifier indicating a risk of damage may further include the following steps: S4.1, In response to the fusion state identifier being a damage risk state or a severe damage risk state, an emergency stop command is generated, and the emergency stop command is determined as the target control command; S4.2, In response to the fusion state identifier being a pure freezing state, a de-icing operation command is generated; S4.3, Obtain the current rotational speed data of the wind turbine blades, and determine the pitch angle adjustment amount based on the current rotational speed data; S4.4, Generate the de-icing operation command based on the pitch angle adjustment amount, and determine the de-icing operation command as the target control command.
[0076] It should be noted that steps S4.1 to S4.4 constitute the complete operational sequence for generating control commands. Damage to wind turbine blades and icing require drastically different handling methods. Damage necessitates immediate shutdown, while icing can be addressed by adjusting the aerodynamic attitude and utilizing centrifugal force to dislodge the ice.
[0077] Specifically, the obtained fusion status identifier code is parsed. When the code is 10 or 11, corresponding to a damage risk state or a severe damage risk state, it means that there are cracks or loss of stiffness inside the blade. At this time, an emergency stop command is directly generated. This command includes the target angle value of directly feathering the pitch angle to 90 degrees and the switching signal of closing the main shaft brake. The target angle value and the switching signal are written into the data load area of the industrial message, and the emergency stop command is identified as the target control command and issued.
[0078] Furthermore, when the fusion state is marked as 01, corresponding to the pure icing state, it means that the blades can eliminate potential hazards through aerodynamic ice removal during operation, which triggers the process of generating de-icing operation commands.
[0079] Furthermore, the real-time impeller speed data of the wind turbine main control system is read via the industrial bus and used as the current speed data. The corresponding target pitch angle is then looked up in a preset speed-to-pitch mapping table based on this current speed data. This mapping table is a two-column lookup table: the first column is the speed value in revolutions per minute (rpm), and the second column is the pitch angle in degrees. The current speed data is iterated through the first column of the lookup table to find the row with the closest speed, and the value in the second column of that row is read as the pitch angle adjustment amount. A specific example of this mapping table is shown in the table below: Current rotational speed (revolutions per minute) Pitch angle adjustment amount (degrees) 5 25 8 20 10 15 12 12
[0080] For example, if the current speed is 10 revolutions per minute, the required pitch angle adjustment is 15 degrees; if the current speed is 5 revolutions per minute, the required pitch angle adjustment is 25 degrees.
[0081] In this embodiment, the pitch angle adjustment amount obtained from the lookup table is filled into a specified byte position in the de-icing operation command data frame to generate a de-icing operation command containing the specific pitch angle. This de-icing operation command is then designated as the target control command and sent to the wind turbine main control system, instructing the pitch servo motor to perform the corresponding deflection action, thereby increasing the centrifugal force for ice removal from the blades by changing the angle of attack. The pitch servo motor is an AC servo motor with a brake, driven by a servo driver. After receiving the angle value in the command, the internal encoder uses a closed-loop adjustment mechanism to rotate the motor rotor to the corresponding angle.
[0082] It is understood that the mapping relationship in the speed-to-pitch mapping table is obtained based on statistical data from aerodynamic ice-throwing tests of wind turbine blades. The preferred setting range is that for every 1 revolution per minute (rpm) increase in speed, the pitch angle decreases by 2 to 3 degrees. In the specific operation of this application, the current ambient temperature is obtained. If the ambient temperature is below -10℃, it indicates strong ice adhesion, and the mapping relationship is set to decrease the pitch angle by 2 degrees for every 1 rpm increase in speed to maintain a larger angle of attack. If the ambient temperature is between -10℃ and 0℃, the mapping relationship is set to decrease the pitch angle by 3 degrees for every 1 rpm increase in speed.
[0083] It should be noted that the target control command needs to undergo cyclic redundancy check (CRC) calculation and be appended to the end of the data frame before being sent. The CRC is obtained by calculating using a generator polynomial based on international telecommunications standards, with a preferred setting of 16 check bits. In the specific operation of this application, the communication protocol version of the wind turbine main control system is obtained. If the version supports 32-bit CRC, the check bit is set to 32 bits; if the version does not support 32 bits, it remains at 16 bits.
[0084] Furthermore, by using this method of generating control commands based on state identifier matching, the adaptive dynamic response of the wind turbine control system is achieved.
[0085] Step S5: Acquire the reference vibration waveform collected by the reference sensor installed at the top of the tower; Understandably, the fiber optic sensor is embedded inside the blade, and the vibration waveform it collects includes not only the aerodynamic and structural vibrations of the blade itself, but also the mechanical background vibrations transmitted from the nacelle to the blade root. The reference sensor is a single-axis accelerometer, bolted to the flange at the top of the tower, and its electrical output is connected to the independent analog channel of the data acquisition card via a shielded cable. The single-axis accelerometer contains a mass and a piezoelectric ceramic crystal. When the tower vibrates, the mass compresses the piezoelectric ceramic crystal, generating a charge proportional to the acceleration. This charge is converted into a voltage signal by a charge amplifier.
[0086] In this application, in order to eliminate this background interference, a reference vibration waveform is obtained from a reference sensor installed at the top of the tower near the yaw bearing.
[0087] Furthermore, since the top of the tower is directly connected to the nacelle, the vibrations from the gearbox and generator inside the nacelle are directly transmitted to the top of the tower. Therefore, this reference vibration waveform can accurately reflect the background vibration baseline transmitted from the nacelle to the blade root.
[0088] In this embodiment, the reference vibration waveform is compared with the blade vibration waveform to serve as the data basis for extracting pure blade vibration, which can significantly improve the signal-to-noise ratio of subsequent feature extraction.
[0089] In some embodiments, the AI fusion processing method for multi-source heterogeneous data may further include the following steps: S5.1, Common mode components are removed from the blade vibration waveform and the reference vibration waveform to obtain a pure vibration waveform; S5.2, the pure vibration waveform is determined as the blade vibration waveform.
[0090] It should be noted that steps S5.1 to S5.2 constitute the complete operation sequence for common-mode interference removal. The blade vibration waveform acquired by the fiber optic sensor and the reference vibration waveform acquired by the reference sensor at the top of the tower are often modulated by the same mechanical vibration source in the nacelle at the same time. Therefore, there are highly overlapping common-mode interference components between the two in a specific frequency range.
[0091] Specifically, discrete numerical sequences of the blade vibration waveform and the reference vibration waveform within the same time window are extracted, and the sampling points of the two waveforms are aligned one by one according to the timestamp. For each aligned time point, the difference between the amplitude of the blade vibration waveform and the amplitude of the reference vibration waveform is calculated, resulting in a series of difference data points. Mathematically and physically, this means that the amplitude of the pure vibration waveform at a certain time point is equal to the amplitude of the blade vibration waveform at that time point minus the amplitude of the reference vibration waveform at that time point. This subtraction operation directly subtracts the background fluctuations transmitted from the nacelle from the blade vibration waveform. Since there may be slight differences in the sampling frequencies of the two waveforms, linear interpolation resampling of the reference vibration waveform is required before alignment to ensure that the number of sampling points and timestamps are completely consistent with the blade vibration waveform. Linear interpolation resampling refers to calculating the estimated amplitude at the intermediate time point based on the amplitude and time difference between two known sampling points.
[0092] Furthermore, a continuous waveform varying over time is reconstructed from the aforementioned difference data points, thereby eliminating the common-mode mechanical vibration components transmitted from the nacelle and obtaining a pure vibration waveform that only reflects the aeroelasticity and structural characteristics of the blade itself. The pure vibration waveform obtained after common-mode elimination is then reassigned as the blade vibration waveform, replacing the original waveform containing interference, for subsequent feature analysis and fusion determination.
[0093] Optionally, if the amplitude of the blade vibration waveform at a certain moment is 5 volts and the amplitude of the reference vibration waveform is 2 volts, then the pure amplitude is 3 volts, and so on to complete the full sequence reconstruction.
[0094] In this application, the difference calculation window length for common mode component elimination is set according to actual needs. In this embodiment, the preferred window length is 1 second.
[0095] Understandably, this common-mode rejection operation reduces the masking effect of background mechanical vibration on the extraction of blade features, and significantly improves the sensitivity of weak vibration feature recognition.
[0096] Furthermore, this method of using tower reference sensors for baseline subtraction does not require complex frequency domain calculations; it can achieve high-precision signal separation simply by relying on basic subtractor circuits or subtraction instructions within the processor.
[0097] Step S6: After sending the target control command to the wind turbine main control system, obtain the execution feedback signal after the wind turbine main control system executes the target control command; It should be noted that after the wind turbine main control system receives and executes a de-icing operation command or an emergency shutdown command, the physical state of the blades will change. For example, ice may fall off or vibration may disappear after shutdown. Closed-loop verification of the effect after execution is necessary. After executing the command, the wind turbine main control system will read the actual pitch angle or brake pressure through sensors and package it into an execution feedback signal.
[0098] In this embodiment, after sending the target control command to the wind turbine main control system, the system monitors the feedback frames on the communication bus in real time to obtain the execution feedback signal after the wind turbine main control system executes the target control command. This execution feedback signal is a binary message conforming to the communication protocol, containing the command execution status word and the actual physical quantity value.
[0099] Furthermore, the execution feedback signal includes the actual position angle data from the pitch angle sensor and the spindle brake status data, used to confirm whether the physical action has been executed correctly, providing a reference time point for the completion of the action for subsequent secondary verification based on vibration waveforms. The pitch angle sensor is an absolute encoder, installed on the rotating shaft of the pitch bearing, and outputs Gray code to represent the current actual rotation angle.
[0100] In some embodiments, the AI fusion processing method for multi-source heterogeneous data may further include the following steps: S6.1, Based on the execution feedback signal, obtain the updated blade vibration waveform collected by the fiber optic sensor; S6.2, Analyze the updated blade vibration waveform using the AI model to generate a secondary confirmation status identifier; S6.3 In response to the secondary confirmation status indicator indicating that the risk has not been eliminated, a long-term lock identifier is generated and the long-term lock identifier is written into the operation log of the wind turbine blade.
[0101] Steps S6.1 to S6.3 constitute the complete operation sequence of closed-loop verification and locking. After the feedback signal indication action is completed, the fiber optic acquisition card is immediately triggered to re-acquire a vibration data segment with a preset time length of 10 seconds, which is used as the updated blade vibration waveform.
[0102] Specifically, the analysis model is used to re-perform frequency component analysis and attenuation feature extraction on the updated blade vibration waveform to determine whether the current icing or damage state of the blade has been eliminated, and a corresponding secondary confirmation status identifier is generated. The secondary confirmation status identifier is an enumerated constant, including a 00 code representing that the risk has been eliminated and a 01 code representing that the risk still exists.
[0103] Furthermore, the secondary confirmation status flag is analyzed. If the flag indicates that the vibration characteristics have not returned to normal, it means the risk has not been eliminated, for example, the ice layer has not detached or the internal cracks are still expanding. In response to the risk not being eliminated, a long-term lock flag is generated. This flag is a fixed byte constant written to a specific address in non-volatile memory and appended to the end of the wind turbine main control system's operation log file as an operation log record for the wind turbine blades. The non-volatile memory is an electrically erasable programmable read-only memory, which can retain this byte constant even after the wind turbine is powered off and restarted.
[0104] For example, the long-term lock flag is converted into the hexadecimal value 0xFF and written to memory address 0x4000, and the timestamp and status code of the lock occurrence are appended to the runtime log file.
[0105] Understandably, after the long-term lockout flag is written, the wind turbine's main control system will read this address every time it powers on for self-test. If it finds 0xFF, it will reject the automatic start request for that blade. The blade must be manually inspected and the flag must be cleared by entering a password on the terminal before it can be restored to operation, thus completely eliminating the catastrophic consequences of operating with damage.
[0106] In this embodiment of the application, the step of using the AI model to analyze the updated blade vibration waveform and generate a secondary confirmation status identifier may further include the following steps: S6.21, in response to the target control command being a de-icing operation command, control the wind turbine main control system to perform a disturbance action on the wind turbine blades; S6.22, Obtain the transient vibration response data of the wind turbine blades when performing the disturbance action; S6.23, Analyze the decay rate of the transient vibration response data using the AI model; S6.24, In response to the attenuation rate satisfying the preset health model conditions, a risk clearance flag is generated; S6.25, in response to the decay rate not meeting the preset health model conditions, a risk persistence indicator is generated; S6.26, the risk relief identifier or the risk persistence identifier is determined as the secondary confirmation status identifier.
[0107] Understandably, steps S6.21 to S6.26 constitute the complete operational sequence for generating the secondary confirmation state. When the wind turbine blades perform de-icing and pitch control, a transient vibration response will occur due to the sudden change in aerodynamic load. The vibration of healthy blades will decay rapidly, while the vibration decay process of blades with residual ice or internal damage will be significantly slower due to changes in damping characteristics.
[0108] In this embodiment, in response to the target control command being a de-icing operation command, the wind turbine main control system performs small-amplitude reciprocating disturbance actions on the wind turbine blades according to a specific pitch sequence, for example, two consecutive pitch changes within a range of ±5 degrees. This reciprocating disturbance action is a set of control commands containing the target angle and execution time point, which is sent to the pitch servo driver.
[0109] Furthermore, during the execution of the disturbance action, high-frequency waveform data collected by the fiber optic sensor is continuously acquired, and the waveform segment within 5 seconds from the start of the disturbance action to the end of the disturbance action is extracted as transient vibration response data.
[0110] Specifically, the analysis of the attenuation rate of the transient vibration response data using an analytical model can be performed as follows: First, receive the feedback signal of the de-icing operation command, issue a pitch disturbance sequence command to the wind turbine main control system, and proceed to the second step. Second, determine whether fiber optic vibration data was collected during the execution of the disturbance sequence. If yes, proceed to the third step; otherwise, proceed to the sixth step. Third, extract the vibration peak value at the end of the disturbance sequence. This peak value is the maximum absolute value of the voltage amplitude in the extracted waveform segment, and proceed to the first step of the sub-process to calculate the attenuation rate. Fourth, determine whether the attenuation rate is greater than the lower limit of attenuation in the preset health model conditions. If yes, proceed to the fifth step; otherwise, proceed to the sixth step. Fifth, generate a risk clearance flag and end the process. Sixth, generate a risk persistence flag and end the process.
[0111] Further, the specific steps of the sub-process include: The first step of the sub-process involves extracting the amplitude of N consecutive sampling points starting from the vibration peak, calculating the difference between the amplitudes of adjacent sampling points, and then proceeding to the second step of the sub-process. The N consecutive sampling points are obtained based on statistics of the sampling frequency and transient response duration of the fiber optic sensor, with a preferred setting range of 50 to 200 sampling points. In the specific operation of this application, the current sampling frequency of the fiber optic sensor is obtained. If the sampling frequency is greater than 1000 Hz, the N consecutive sampling points are set to 100; if the sampling frequency is less than or equal to 1000 Hz, the N consecutive sampling points are set to 50. The second step of the sub-process involves calculating the absolute value of all differences and averaging them to obtain the average attenuation slope. This average attenuation slope is then returned to the aforementioned fourth step as the attenuation rate. The physical meaning of this calculation process is as follows: In the amplitude sequence of the N consecutive sampling points extracted above, the amplitude of the next sampling point is subtracted from the amplitude of the previous sampling point to obtain N minus 1 differences. The absolute value of each difference is taken and all of them are added together to obtain a sum. Then, the sum is divided by N minus 1 to obtain a quotient, which is the average attenuation slope, and its unit is volts per sampling interval.
[0112] For example, if the amplitude sequence of a transient response is 10 volts, 8 volts, 6 volts, 4 volts, and 2 volts, then the absolute difference values are 2, 2, 2, and 2, with a total of 8. Dividing by 4 yields an average attenuation slope of 2 volts per sampling interval, which is greater than the preset attenuation lower limit of 1.5 volts per sampling interval. In this case, a risk clearance flag is generated.
[0113] Furthermore, in response to the attenuation rate satisfying the preset health model condition, i.e. the average attenuation slope is greater than the preset attenuation lower limit, it indicates that the vibration converges rapidly, the blade structure damping is normal, and a risk relief indicator is generated.
[0114] Furthermore, in response to the attenuation rate not meeting the preset health model conditions, i.e., the average attenuation slope is less than or equal to the preset attenuation lower limit, it indicates that the vibration convergence is slow and the blade still has abnormal damping, thus generating a risk persistence indicator.
[0115] In this application, the risk deactivation flag or the risk persistence flag is defined as a secondary confirmation status flag. The risk deactivation flag is represented in memory as hexadecimal 0x00, and the risk persistence flag is represented as hexadecimal 0x01.
[0116] In this application, the lower limit of attenuation in the preset health model conditions is used to determine whether the damping ratio of the blade structure is within the healthy range. It is obtained by statistically analyzing the free vibration attenuation test data of the wind turbine blades at the time of manufacture, and its preferred setting range is 0.5 to 2.0 volts per sampling interval. In the specific operation of this application, the current ambient humidity parameter is obtained. If the ambient humidity is greater than 80%, it indicates that a water film may remain on the blade surface, leading to increased damping. Therefore, the lower limit of attenuation is set to 1.0 volts per sampling interval. If the ambient humidity is less than or equal to 80%, the lower limit of attenuation is set to 1.5 volts per sampling interval.
[0117] It should be noted that by introducing pitch disturbance and analyzing transient decay characteristics, the secondary confirmation method does not rely on the long-term accumulation of steady-state signals. It can quickly assess the true health status of the blades in a very short time after the de-icing action is performed, which greatly improves the timeliness of closed-loop feedback.
[0118] Appendix Figure 2 The process of acquiring and preprocessing multi-source heterogeneous data is illustrated. This step involves simultaneously acquiring images of UAV blades and vibration waveforms from fiber optic sensors, and extracting corresponding vibration segments based on the location of the capture points. After time synchronization verification, a tower reference waveform is introduced to remove common-mode components, thereby obtaining clean multi-source heterogeneous data and providing a highly reliable data foundation for subsequent feature analysis.
[0119] Appendix Figure 3 This paper demonstrates the process of using an AI model for visual and vibration feature analysis. In the visual dimension, visual occlusion is determined by detecting overexposed areas and iridescent halos at the edges of the image. In the vibration dimension, frequency component analysis extracts the dominant frequency shift direction and high-frequency harmonic content, respectively labeling mass gain and stiffness loss features. The parallel extraction of these two types of feature data provides multi-dimensional criteria for fusion and judgment.
[0120] Appendix Figure 4 The demonstration illustrates the AI-based fusion logic for judgment and control command generation. When visual features confirm occlusion, a fusion judgment is performed based on the combination of vibration features to accurately distinguish between pure icing, damage risk, and severe damage risk states. For different fusion state identifiers, de-icing operation commands or emergency stop commands are adaptively generated, enabling differentiated control responses to complex operating conditions.
[0121] Appendix Figure 5 The feedback and secondary confirmation mechanism after the control command is executed is illustrated. After the command is issued and the updated vibration waveform is obtained, a disturbance action is executed in response to the de-icing operation command, and the risk clearance status is assessed by analyzing the decay rate of the transient response. If the secondary confirmation indicates that the risk has not been cleared, a long-term lockout flag is generated and written to the operation log, forming a complete closed-loop safety control link.
[0122] Example 2: This example also provides an AI fusion processing system for multi-source heterogeneous data, including: a data acquisition module, used to acquire multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms, wherein the multi-source heterogeneous data includes blade images captured by drones and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades; The AI feature analysis module is used to perform visual feature analysis on the blade image using an AI model to obtain visual feature data, and to perform vibration feature analysis on the blade vibration waveform using the AI model to obtain vibration feature data. The AI fusion judgment module is used to perform fusion judgment on the visual feature data and the vibration feature data based on AI fusion logic to determine the fusion status identifier of the wind turbine blade. The control execution module is used to generate a target control command that matches the fusion status indicator in response to the indication of damage risk, and send the target control command to the wind turbine main control system.
[0123] The above-mentioned unit modules can be embedded in the processor of the electronic device in hardware form or independent of it, or they can be stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of the above modules.
[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. An AI fusion processing method for multi-source heterogeneous data, characterized in that, include: Acquire multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms. The multi-source heterogeneous data includes blade images captured by UAVs and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades. The visual feature analysis of the blade image is performed using an AI model to obtain visual feature data, and the vibration feature analysis of the blade vibration waveform is performed using the AI model to obtain vibration feature data. The visual feature data and the vibration feature data are fused and determined based on AI fusion logic to identify the fusion status of the wind turbine blades. In response to the fusion status indicator indicating a risk of damage, a target control command matching the fusion status indicator is generated and sent to the wind turbine main control system.
2. The AI fusion processing method for multi-source heterogeneous data according to claim 1, characterized in that: The acquisition of multi-source heterogeneous data corresponding to offshore wind turbine blades includes: The image of the blade captured by the drone is acquired, and the vibration waveform of the blade collected by the fiber optic sensor is acquired simultaneously. Determine the location data of the shooting point of the blade image on the wind turbine blade; Based on the shooting point location data, a vibration segment corresponding to the shooting point location is extracted from the blade vibration waveform; Associate the blade image with the vibration segment and verify whether the acquisition time of the blade image and the acquisition time of the vibration segment meet the synchronization condition; In response to the satisfaction of the synchronization condition, the associated blade image and the vibration segment are identified as the multi-source heterogeneous data.
3. The AI fusion processing method for multi-source heterogeneous data according to claim 1, characterized in that: The AI fusion processing method for multi-source heterogeneous data also includes: Acquire reference vibration waveforms from reference sensors installed at the top of the tower; Common-mode components are removed from the blade vibration waveform and the reference vibration waveform to obtain a pure vibration waveform. The pure vibration waveform is determined as the blade vibration waveform.
4. The AI fusion processing method for multi-source heterogeneous data according to claim 1, characterized in that: The step of using an AI model to perform visual feature analysis on the leaf image to obtain visual feature data includes: The AI model is used to analyze the pixel brightness distribution in the leaf image to determine whether there are overexposed areas where the pixel brightness value is higher than the preset brightness condition. In response to the existence of the overexposed area, and the presence of a halo area at the edge of the overexposed area whose pixel color satisfies a preset rainbow color condition, a visual occlusion mark is generated. The location range data of the overexposed area and the visual occlusion marker are determined as the visual feature data.
5. The AI fusion processing method for multi-source heterogeneous data according to claim 1, characterized in that: The step of using the AI model to perform vibration feature analysis on the blade vibration waveform to obtain vibration feature data includes: The AI model is used to perform frequency component analysis on the blade vibration waveform to determine the dominant frequency shift direction and high-frequency harmonic content; In response to the main frequency offset direction pointing to the frequency reduction direction, and the high-frequency harmonic content not exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as mass gain characteristics. In response to the high-frequency harmonic content exceeding the preset harmonic condition, the characteristics of the blade vibration waveform are determined as stiffness loss characteristics. The mass gain characteristic or the stiffness loss characteristic is determined as the vibration characteristic data.
6. The AI fusion processing method for multi-source heterogeneous data according to claim 1, characterized in that: The step of fusing and determining the fusion status identifier of the wind turbine blades based on AI fusion logic includes: The visual feature data is determined to contain visual occlusion markers; In response to the vibration feature data being a mass gain feature, the AI fusion logic is used to determine that the wind turbine blade is in a pure icing state, and the pure icing state is identified as the fusion state identifier. In response to the vibration feature data being stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in a state of ice cover damage, and the damage risk state is determined as the fusion state identifier. Since the vibration feature data simultaneously includes mass gain features and stiffness loss features, the AI fusion logic is used to determine that the wind turbine blade is in a state of icing and concurrent damage, and the severe damage risk state is identified as the fusion state identifier.
7. The AI fusion processing method for multi-source heterogeneous data according to claim 6, characterized in that: The step of generating a target control command matching the fusion status identifier in response to the indication of a risk of damage includes: In response to the fusion status being identified as a damage risk state or a severe damage risk state, an emergency stop command is generated, and the emergency stop command is determined as the target control command; In response to the fusion state being identified as a pure icing state, a de-icing operation command is generated; Obtain the current rotational speed data of the wind turbine blades, and determine the pitch angle adjustment amount based on the current rotational speed data; The de-icing operation command is generated based on the pitch angle adjustment amount, and the de-icing operation command is determined as the target control command.
8. The AI fusion processing method for multi-source heterogeneous data according to claim 7, characterized in that: The AI fusion processing method for multi-source heterogeneous data also includes: After sending the target control command to the wind turbine main control system, the execution feedback signal after the wind turbine main control system executes the target control command is obtained; Based on the execution feedback signal, the updated blade vibration waveform collected by the fiber optic sensor is obtained; The AI model is used to analyze the updated blade vibration waveform and generate a secondary confirmation status identifier. In response to the secondary confirmation status indicating that the risk has not been eliminated, a long-term lock flag is generated and written to the operation log of the wind turbine blade.
9. The AI fusion processing method for multi-source heterogeneous data according to claim 8, characterized in that: The step of using the AI model to analyze the updated blade vibration waveform and generate a secondary confirmation status identifier includes: In response to the target control command being a de-icing operation command, the main control system of the wind turbine is controlled to perform a disturbance action on the wind turbine blades; Acquire the transient vibration response data of the wind turbine blades when they perform the disturbance action; The AI model was used to analyze the decay rate of the transient vibration response data; In response to the decay rate meeting the preset health model conditions, a risk clearance flag is generated; In response to the decay rate not meeting the preset health model conditions, a risk persistence indicator is generated; The risk relief flag or the risk persistence flag is determined as the secondary confirmation status flag.
10. An AI fusion processing system for multi-source heterogeneous data, characterized in that, include: The data acquisition module is used to acquire multi-source heterogeneous data corresponding to the wind turbine blades of offshore wind farms. The multi-source heterogeneous data includes blade images taken by UAVs and blade vibration waveforms collected by fiber optic sensors installed inside the wind turbine blades. The AI feature analysis module is used to perform visual feature analysis on the blade image using an AI model to obtain visual feature data, and to perform vibration feature analysis on the blade vibration waveform using the AI model to obtain vibration feature data. The AI fusion judgment module is used to perform fusion judgment on the visual feature data and the vibration feature data based on AI fusion logic to determine the fusion status identifier of the wind turbine blade. The control execution module is used to generate a target control command that matches the fusion status indicator in response to the indication of damage risk, and send the target control command to the wind turbine main control system.
