A method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition

Abstract

This invention provides a method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition. It belongs to the technical field of real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition. This invention achieves microsecond-level synchronous acquisition by arranging three global shutter high-speed cameras around the wind turbine tower and using a high-frequency strobe laser illumination device. The acquired raw image stream is input into a deblurring adversarial reconstruction network to eliminate motion blur. At the edge computing nodes, the sharpened image sequence undergoes dynamic cropping and spatial dimensionality reduction encoding of the blade region. The dimensionality reduction feature vector and camera pose parameters are input into an adaptive holographic neural radiation field vibration network to reconstruct the three-dimensional voxel-level displacement field of the blade. This solves the technical problem of not being able to achieve accurate reconstruction of three-dimensional voxel-level real-time vibration displacement of wind turbine blades in rotational motion and complex outdoor environments.

Description

Technical Field

[0001] This invention belongs to the field of real-time displacement and vibration detection technology of wind turbine blades based on neural network visual recognition. Specifically, it relates to a method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition. Background Technology

[0002] Vibration and displacement detection of wind turbine blades is a core task in the health monitoring of wind power equipment. In current wind farm operation and maintenance, traditional detection methods are mainly divided into two categories: contact and non-contact. Contact methods collect vibration signals by installing accelerometers or strain gauges on the blade surface, while non-contact methods use monocular or binocular vision systems combined with feature point tracking algorithms to estimate the two-dimensional displacement of the blade. Some solutions introduce laser rangefinders or millimeter-wave radar to assist in ranging, which can obtain certain displacement data under static or low-speed rotation conditions.

[0003] However, the above methods face significant limitations in actual wind farm environments: contact sensors are limited by their installation location and can only acquire local point information, failing to reconstruct the three-dimensional displacement distribution of the entire field; two-dimensional vision methods suffer from severe motion blur when the blades rotate at high speeds, leading to the failure of feature point extraction; radar methods have insufficient resolution for slow, small-amplitude vibrations; and traditional stereo vision methods rely on explicit feature point matching and triangulation, which accumulate matching errors in outdoor environments with dynamic cloud background interference and severe light refraction, making it impossible to achieve sub-pixel-level three-dimensional displacement field reconstruction.

[0004] In existing technologies, due to the combined effects of motion blur caused by high-speed blade rotation, complex background interference, and non-rigid deformation, traditional methods cannot reconstruct the three-dimensional voxel-level vibration displacement field of the blade in real time from multi-view image sequences. In other words, existing technologies have the technical problem of not being able to achieve accurate reconstruction of the three-dimensional voxel-level real-time vibration displacement of wind turbine blades in rotational motion and complex outdoor environments. Summary of the Invention

[0005] In view of this, the present invention provides a method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition, which can solve the technical problem in the prior art that wind turbine blades cannot achieve accurate reconstruction of three-dimensional voxel-level real-time vibration displacement under rotational motion and complex outdoor environments.

[0006] This invention is implemented as follows: This invention provides a method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition, comprising the following steps:

[0007] Three global shutter high-speed cameras are arranged at 120-degree intervals around the wind turbine tower. At the same time, a high-frequency stroboscopic laser supplementary lighting device is installed on the top of the tower. The three global shutter high-speed cameras and the high-frequency stroboscopic laser supplementary lighting device achieve microsecond-level synchronous acquisition through a hardware-level clock trigger.

[0008] Three global shutter high-speed cameras acquire raw image streams of the blade rotation process at a frame rate of 200 frames per second. Simultaneously, the trigger electrical signal of the generator side exciter is recorded as a time reference label. The raw image stream is input into a deblurring adversarial reconstruction network for motion blur elimination processing to obtain a sharpened image sequence.

[0009] At the edge computing node, dynamic cropping of the leaf region is performed on the sharpened image sequence, the leaf contour features and leaf texture features are extracted and spatial dimensionality reduction encoding is performed to obtain the dimensionality reduction feature vector, and the dimensionality reduction feature vector is transmitted to the central server through the network.

[0010] The dimensionality-reduced feature vector and the camera pose parameters of three global shutter high-speed cameras are input into an adaptive holographic neural radiation field vibration network. The three-dimensional voxel-level displacement field of the blade is reconstructed through multi-view implicit neural representation, and the instantaneous displacement of the blade tip in the three-dimensional spatial coordinate system is calculated.

[0011] The Königsberg graph theory connected component spatiotemporal pruning algorithm is applied to suspected feature points in the three-dimensional voxel-level displacement field of the blade to remove dynamic cloud background interference and illumination refraction interference, while retaining the true vibration displacement trajectory.

[0012] The blade tip amplitude is calculated based on the actual vibration displacement trajectory. The current vibration state is determined by the amplitude adaptive adjustment function. When the blade tip amplitude exceeds the safety threshold, the parameter adjustment command is triggered and an early warning signal is generated.

[0013] The hardware-level clock trigger uses a GPS clock synchronization module. The three global shutter high-speed cameras and the high-frequency strobe laser illumination device all receive the pulse signal output by the GPS clock synchronization module as the trigger source. The pulse frequency of the pulse signal is 200Hz and the pulse rise edge error is less than 10 nanoseconds.

[0014] The trigger frequency of the high-frequency stroboscopic laser supplementary lighting device is synchronized with the exposure time of the three global shutter high-speed cameras. The trigger frequency is set to 200Hz. The single stroboscopic pulse width is obtained by experimentally testing the sharpness of the blade edge under different wind speed conditions. The minimum pulse width that reduces the radius of the blade edge blur kernel to less than 2 pixels is selected as the optimal value.

[0015] The deblurred adversarial reconstruction network adopts an encoder-decoder structure. The encoder part contains 5 convolutional layers, and the decoder part contains 5 deconvolutional layers. The deblurred adversarial reconstruction network introduces a skip connection structure to fuse the features of each layer of the encoder with the corresponding layer of the decoder. An adversarial discriminator is added to the last layer of the decoder. The adversarial discriminator is a 3-layer fully convolutional network that outputs a single-channel probability map.

[0016] The training loss function of the deblurred adversarial reconstruction network is composed of a weighted combination of content loss and adversarial loss. The content loss is the pixel-level mean square error between the clear image and the reconstructed image, and the adversarial loss is the binary cross-entropy output by the adversarial discriminator. The weight ratio of content loss to adversarial loss is set to 100:1.

[0017] Specifically, the dynamic cropping of the blade region involves: extracting edge contours for each frame of the sharpened image sequence using the Canny operator; performing a Hough linear transform on the extracted edge contours to identify the blade's main axis direction; expanding the blade's chord length to both sides with the main axis direction as the center to obtain the bounding box of the region of interest; and cropping the image region within the bounding box of the region of interest.

[0018] Specifically, the spatial dimensionality reduction encoding involves inputting the leaf region image into a lightweight convolutional neural network for feature extraction. The lightweight convolutional neural network contains three depthwise separable convolutional layers with output channels of 32, 64, and 128, respectively. After the last depthwise separable convolutional layer, a global average pooling layer is connected to compress the feature map into a 128-dimensional feature vector.

[0019] The core of the adaptive holographic neural radiation field vibration network adopts a multilayer perceptron structure with 8 fully connected layers, each layer has 256 neurons, the activation function is the ReLU function, the position encoding module uses sine and cosine functions to encode 10 frequency components of the three-dimensional spatial coordinates, and the time encoding module maps the timestamp to a 64-dimensional time feature vector.

[0020] The adaptive holographic neural radiation field vibration network employs a gradient adaptive jump mechanism during volume rendering to calculate the displacement temporal gradient of each three-dimensional point position. When the absolute value of the displacement temporal gradient is greater than a preset gradient threshold, the sampling point density is increased to four times the original density within a spherical region with a radius of 0.01 meters around the three-dimensional point position. The reference value of the preset gradient threshold is 0.015.

[0021] The Königsberg graph theory connected component spatiotemporal pruning algorithm abstracts each voxel in the three-dimensional voxel-level displacement field of the leaf as a vertex of the graph, and the displacement relationship between adjacent voxels is abstracted as a directed edge. The weight of the directed edge is defined as the Pearson correlation coefficient of the displacement vector between adjacent voxels. Strongly connected components are identified through depth-first search. When the weight of the directed edge is less than a preset correlation coefficient threshold, a dynamic pruning operation is performed. The reference value of the preset correlation coefficient threshold is 0.65.

[0022] In the Königsberg graph theory connected component spatiotemporal pruning algorithm, when performing depth-first search, the algorithm starts by pushing the voxel with the largest displacement amplitude onto the stack. During the traversal, it checks the weight of each directed edge. If the weight is less than a preset correlation coefficient threshold, the directed edge is pruned and the target vertex is marked as unvisited. The strongly connected component with the most vertices is retained, and the displacement values ​​of all voxels outside the strongly connected component are set to zero to complete the background stripping.

[0023] Specifically, the calculation of the blade tip amplitude involves: extracting the voxel displacement sequence corresponding to the blade tip position from the actual vibration displacement trajectory. The blade tip position is defined as the voxel point farthest from the rotation center along the main axis of the blade. The square root of the sum of the squares of the amplitudes of each axial component of the voxel displacement sequence in the three-dimensional spatial coordinate system is used to obtain the composite displacement amplitude. The blade tip amplitude is then extracted by applying the maximum value of a sliding window with a length of 200 frames to the composite displacement amplitude sequence.

[0024] The amplitude adaptive adjustment function calculates the adjustment coefficient based on three parameters: the current tip amplitude, the historical average amplitude, and the amplitude change rate. adjustment coefficient The calculation formula is: The reference amplitude The reference value is 0.5 meters, and the reference amplitude change rate is... The reference value is 0.1 .

[0025] The amplitude adaptive adjustment function is based on the adjustment coefficient. The value range of the adaptive holographic neural radiation field vibration network sampling point density and the deblurring intensity of the deblurring adversarial reconstruction network are dynamically adjusted. When the adjustment coefficient is... When the value is ≥2.5, the sampling point density is set to 8 times the baseline value, the deblurring intensity is set to ultra-high intensity mode, the anti-loss weight is increased to 5 times the baseline value, and the blade vibration warning signal is triggered at the same time.

[0026] The safety threshold is determined according to the wind turbine design specifications. When the blade tip amplitude exceeds 5% of the blade length, there is a risk of structural fatigue. When the blade tip amplitude is detected to exceed the safety threshold, the system sends an early warning signal to the wind farm monitoring center and records the complete three-dimensional voxel-level displacement field data and the clear image sequence of the blade at the moment of vibration exceeding the limit.

[0027] This invention integrates a defuzzification adversarial reconstruction network, an adaptive holographic neural radiation field vibration network, and a Königsberg graph theory connected component spatiotemporal pruning algorithm to construct a complete technical chain from motion fuzz elimination to three-dimensional voxel-level displacement field reconstruction and background interference removal. This solves the technical problem that wind turbine blades cannot achieve accurate reconstruction of three-dimensional voxel-level real-time vibration displacement in rotational motion and complex outdoor environments.

[0028] This invention addresses the shortcomings of traditional methods where motion blur leads to feature extraction failure by restoring high-frequency details of images through an adversarial reconstruction network, providing clear input for subsequent 3D reconstruction. It also addresses the issue of accumulated errors in traditional stereo vision feature point matching by learning the 3D displacement field directly from photometric consistency constraints through implicit neural representations, avoiding error propagation from explicit matching and triangulation. Furthermore, it addresses the inability to effectively remove dynamic background interference by using graph theory connectivity constraints to separate interfering voxels from the real vibration region through the spatial continuity of the actual vibration displacement.

[0029] In summary, the present invention solves the technical problem mentioned in the background art that wind turbine blades cannot achieve accurate reconstruction of three-dimensional voxel-level real-time vibration displacement under rotational motion and complex outdoor environments. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention.

[0031] Figure 2 This is a graph showing the change in blade tip amplitude over time. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0033] like Figure 1 The diagram shown is a flowchart of a real-time displacement and vibration detection method for wind turbine blades based on neural network visual recognition provided by the present invention. This method includes the following steps:

[0034] S01. Three global shutter high-speed cameras are arranged at 120-degree intervals around the wind turbine tower, and a high-frequency stroboscopic laser supplementary lighting device is installed on the top of the tower. The three global shutter high-speed cameras and the high-frequency stroboscopic laser supplementary lighting device achieve microsecond-level synchronous acquisition through a hardware-level clock trigger.

[0035] S02. Three global shutter high-speed cameras acquire the original image stream of the blade rotation process at a frame rate of 200 frames per second. The generator side exciter trigger electrical signal is recorded synchronously as a time reference label. The original image stream is input into a deblurring adversarial reconstruction network for motion blur elimination processing to obtain a sharpened image sequence.

[0036] S03. Perform dynamic cropping of the leaf region on the sharpened image sequence at the edge computing node, extract the leaf contour features and leaf texture features and perform spatial dimensionality reduction encoding to obtain the dimensionality reduction feature vector, and transmit the dimensionality reduction feature vector to the central server through the network.

[0037] S04. Input the reduced feature vector and the camera pose parameters of the three global shutter high-speed cameras into the adaptive holographic neural radiation field vibration network, and reconstruct the three-dimensional voxel-level displacement field of the blade through multi-view implicit neural representation to calculate the instantaneous displacement of the blade tip in the three-dimensional spatial coordinate system.

[0038] S05. Apply the Königsberg graph theory connected component spatiotemporal pruning algorithm to the suspected feature points in the three-dimensional voxel-level displacement field of the blade to remove dynamic cloud background interference and light refraction interference, and retain the real vibration displacement trajectory.

[0039] S06. Calculate the blade tip amplitude based on the actual vibration displacement trajectory, determine the current vibration state through the amplitude adaptive adjustment function, and trigger the parameter adjustment command and generate an early warning signal when the blade tip amplitude exceeds the safety threshold.

[0040] The trigger frequency of the high-frequency stroboscopic laser illumination device is synchronized with the exposure time of three global shutter high-speed cameras. The trigger frequency is set to 200Hz, and the single stroboscopic pulse width is set to 500 microseconds. The single stroboscopic pulse width value is obtained by experimentally testing the sharpness of the blade edge under different wind speed conditions. In the experiment, 10 sets of conditions with wind speeds ranging from 5 m / s to 25 m / s were selected. For each set of conditions, 20 discrete values ​​with pulse widths ranging from 100 microseconds to 1000 microseconds were tested. The blade edge blur kernel radius corresponding to each pulse width was recorded, and the minimum pulse width that reduces the blade edge blur kernel radius to less than 2 pixels was selected as the optimal value.

[0041] The deblurring adversarial reconstruction network adopts an encoder-decoder structure. The encoder part contains 5 convolutional layers to extract depth features of the blurred image, and the decoder part contains 5 deconvolutional layers to reconstruct the clear image. The deblurring adversarial reconstruction network introduces a skip connection structure to fuse the features of each layer of the encoder with the corresponding layer of the decoder. An adversarial discriminator is added to the last layer of the decoder to constrain the realism of the reconstructed image. The adversarial discriminator is a 3-layer fully convolutional network that outputs a single-channel probability map to distinguish between the real clear image and the reconstructed image.

[0042] The specific steps for establishing the training dataset for the deblurring adversarial reconstruction network include: building a blade model rotation platform in a laboratory environment, installing a 3-meter-long scaled-down model of a wind turbine blade on the platform, rotating the scaled-down model of the wind turbine blade at a speed of 10 to 60 revolutions per minute by a motor, capturing images of the moving scaled-down model of the wind turbine blade at different exposure times (1 to 10 milliseconds) using an experimental high-speed camera, and obtaining 6,000 sets of blurred images. Simultaneously, capturing images of the same scene at the same shooting angle using a microsecond-level ultra-high-speed camera to obtain corresponding clear images as labels, pairing the blurred images with the clear images to form training sample pairs, and dividing the training set and validation set into an 8:2 ratio.

[0043] The specific steps of training the deblurred adversarial reconstruction network include: inputting the training set into the deblurred adversarial reconstruction network, setting the batch size to 16, the initial learning rate to 0.0002, using the Adam optimizer for parameter updates, and using a weighted combination of content loss and adversarial loss. The content loss is the pixel-level mean square error between the clear image and the reconstructed image, and the adversarial loss is the binary cross-entropy output by the adversarial discriminator. The weight ratio of content loss to adversarial loss is set to 100:1. During training, the peak signal-to-noise ratio is evaluated on the validation set every 10 rounds. When the peak signal-to-noise ratio no longer increases for 5 consecutive rounds, the learning rate is reduced to 0.1 times the original value. The total number of training rounds is 200 rounds. The model parameters with the highest peak signal-to-noise ratio on the validation set are saved as the final model.

[0044] The specific steps of the dynamic cropping of the blade region include: performing edge detection on each frame of the sharpened image sequence, extracting edge contours using the Canny operator, performing Hough linear transform on the extracted edge contours to identify the blade's main axis direction, expanding the region of interest (ROI) bounding box to both sides with the blade's main axis direction as the center, the preset width being 1.2 times the blade chord length, and cropping the image region within the ROI bounding box as the blade region image.

[0045] The spatial dimensionality reduction encoding step specifically includes: inputting the leaf region image into a lightweight convolutional neural network for feature extraction. The lightweight convolutional neural network contains three depthwise separable convolutional layers with output channels of 32, 64, and 128 respectively, and a kernel size of 3×3. After the last depthwise separable convolutional layer, a global average pooling layer is connected to compress the feature map into a 128-dimensional feature vector. The data volume of the 128-dimensional feature vector is reduced to 0.05 times that of the original image.

[0046] The adaptive holographic neural radiation field vibration network has the following structure: The network input includes dimensionality-reduced feature vectors from three perspectives and corresponding camera pose parameters. The network core employs a multilayer perceptron structure, which contains eight fully connected layers, each with 256 neurons. The activation function is the ReLU function. The adaptive holographic neural radiation field vibration network takes three-dimensional spatial coordinates and time encoding as input. A position encoding module maps the three-dimensional spatial coordinates to a high-frequency feature space. This position encoding module uses sine and cosine functions to encode the ten frequency components of the three-dimensional spatial coordinates. The time encoding module maps timestamps to a 64-dimensional time feature vector. The multilayer perceptron structure outputs… The output includes volume density and RGB color. During volume rendering, a gradient adaptive jump mechanism is used to calculate the displacement temporal gradient of each 3D point position. When the absolute value of the displacement temporal gradient is greater than a preset gradient threshold, the sampling point density is increased to 4 times the original density in a spherical region with a radius of 0.01 meters around the 3D point position. The 3D voxels are projected onto the 2D image plane through differentiable volume rendering. The reprojection error is used as a supervision signal to optimize the network parameters. The preset gradient threshold is obtained by testing the network reconstruction accuracy on 10 sets of blade vibration data with different amplitudes. The candidate value of the preset gradient threshold ranges from 0.001 to 0.1. The threshold of 0.015, which minimizes the root mean square error between the reconstructed displacement field and the real displacement field, is selected as the optimal value.

[0047] The steps for establishing the training dataset of the adaptive holographic neural radiation field vibration network specifically include: selecting five different types of wind turbines in a real wind farm environment, arranging three experimental high-speed cameras around the tower of each wind turbine, and collecting image sequences of the blades under different wind speeds and vibration states. The wind speed range is from 5 m / s to 25 m / s, and the acquisition time for each wind turbine is 48 hours, resulting in a total of 30 million original images. At the same time, accelerometers are installed on the blade surface to measure the actual vibration displacement as label data. The collected image sequences are filtered according to the vibration amplitude, retaining image sequences with amplitudes in the range of 0.1 m to 2 m. The filtered image sequences are divided into segments of 5 seconds in length according to the time window, with each segment containing 1000 images, resulting in 15000 training samples. The training set, validation set, and test set are divided in a ratio of 7:2:1.

[0048] The training steps of the adaptive holographic neural radiation field vibration network specifically include: inputting the image sequence and camera pose parameters of the training set into the adaptive holographic neural radiation field vibration network, setting the batch size to 4, the initial learning rate to 0.0005, using the AdamW optimizer for parameter updates, and the loss function consisting of three parts: reprojection loss, displacement consistency loss, and temporal smoothing loss. The reprojection loss calculates the mean square error between the network-rendered image and the real image, the displacement consistency loss calculates the absolute error between the network-predicted displacement and the displacement measured by the accelerometer, and the temporal smoothing loss constrains the gradient of displacement changes between adjacent frames. The weight ratio of reprojection loss, displacement consistency loss, and temporal smoothing loss is set to 1:10:0.1. During training, the displacement prediction error is evaluated on the validation set every 5 rounds. When the displacement prediction error no longer decreases for 3 consecutive rounds, the learning rate is reduced to 0.5 times the original value. The total number of training rounds is 100 rounds. The model parameters with the smallest displacement prediction error on the validation set are saved as the final model.

[0049] The adaptive holographic neural radiation field vibration network achieves accurate modeling of non-rigid body vibration by fusing implicit neural representations with a multilayer perceptron structure. Traditional 3D reconstruction methods rely on explicit geometric representations, which limit the reconstruction accuracy of rapidly deforming blade vibration scenes. Implicit neural representations express physical quantities in 3D space as continuous functions, and the network learns the mapping relationship between 3D spatial coordinates, volume density, and RGB color through a neural network. During training, the adaptive holographic neural radiation field vibration network automatically learns the vibration patterns of the blade surface. The gradient adaptive jump mechanism enables the network to automatically increase the sampling point density in areas of intense vibration, thereby capturing sub-pixel-level minute displacements. The introduction of viewpoint constraints and time encoding modules enables the adaptive holographic neural radiation field vibration network to implicitly reconstruct the spatiotemporal evolution of the leaf's three-dimensional voxel-level displacement field from a two-dimensional image sequence. Compared with traditional stereo vision methods, the adaptive holographic neural radiation field vibration network avoids the cumulative errors of feature point matching and triangulation, and directly learns the leaf's three-dimensional voxel-level displacement field from photometric consistency constraints. This makes the displacement measurement accuracy not limited by image resolution. At the same time, the differentiable volume rendering framework allows end-to-end gradient optimization, enabling the adaptive holographic neural radiation field vibration network to simultaneously optimize geometric reconstruction and motion estimation. As a result, it maintains stable displacement detection performance even under extreme conditions of low-light motion blur and complex background interference.

[0050] The principle of the Königsberg graph theory connected component spatiotemporal pruning algorithm is as follows: each voxel in the three-dimensional voxel-level displacement field of the blade is abstracted as a vertex of a graph, and the displacement relationship between adjacent voxels is abstracted as a directed edge. The weight of the directed edge is defined as the correlation coefficient of the displacement vector between adjacent voxels. Based on the connectivity theory in graph theory, the real vibration displacement is spatially represented as a highly connected strongly connected component, while background interference and noise are spatially isolated. The Königsberg graph theory connected component spatiotemporal pruning algorithm first constructs a global displacement correlation graph, and then uses a depth-first search to traverse all vertices in the global displacement correlation graph to identify the strongly connected components in the global displacement correlation graph. For connected components, the weight of each directed edge is calculated during traversal. When the weight is less than a preset correlation coefficient threshold, a dynamic pruning operation is performed to cut off the directed edge and mark the corresponding voxel as an isolated point. The dynamic pruning operation is performed iteratively until the global displacement correlation graph structure converges. The largest strongly connected component that is finally retained is the real vibration displacement region. The preset correlation coefficient threshold is obtained by performing a grid search on 50 sets of test data containing different levels of background interference. The candidate value range of the preset correlation coefficient threshold is 0.3 to 0.9, with a step size of 0.05. The threshold of 0.65, which maximizes the harmonic average of the accuracy of real displacement recognition and the false interference rejection rate, is selected as the optimal value.

[0051] The specific implementation steps of the Königsberg graph theory connected component spatiotemporal pruning algorithm include: performing voxel traversal on the three-dimensional voxel-level displacement field of the leaf, storing the three-dimensional spatial coordinates and displacement vector of each voxel as vertex attributes, searching for all neighboring voxels within a spatial neighborhood radius of 0.05 meters for each voxel, calculating the Pearson correlation coefficient between the displacement vectors of the current voxel and its neighboring voxels as the directed edge weight, constructing the adjacency matrix of the global displacement correlation graph, initializing the depth-first search stack, pushing the voxel with the largest displacement amplitude onto the stack, performing a depth-first search traversal, checking the weight of each directed edge during the depth-first search traversal, pruning the directed edge if the weight of the directed edge is less than 0.65 and marking the target vertex as unvisited, continuing to traverse other directed edges until the depth-first search stack is empty, counting the traversed vertex set as strongly connected components, retaining the strongly connected component with the largest number of vertices, and setting the displacement values ​​of all voxels outside the strongly connected component to zero to complete background stripping.

[0052] The Königsberg graph theory connected component spatiotemporal pruning algorithm achieves efficient background interference removal through graph theory connectivity constraints. Traditional image segmentation methods rely on pixel-level color or texture features, which suffer from poor feature stability in outdoor environments with dynamic cloud background interference and severe light refraction interference. The Königsberg graph theory connected component spatiotemporal pruning algorithm transforms the problem into graph connectivity analysis. Utilizing the spatial continuity of vibration displacement, the real leaf vibration forms a continuous displacement field in space, with highly correlated displacement vectors between adjacent positions. In contrast, the spurious displacements generated by cloud movement and light refraction exhibit random spatial distribution. Since there is a lack of physical correlation between quantities, the Königsberg graph theory connected component spatiotemporal pruning algorithm constructs a global displacement correlation graph and performs strongly connected component decomposition to encode the physical continuity constraints of vibration in a graph theory manner. The dynamic pruning operation iteratively eliminates low-correlation connections based on directed edge weights, preventing isolated interference points from forming connected paths. The time complexity of the depth-first search is linear, which significantly improves computational efficiency compared to traditional global optimization methods. The Königsberg graph theory connected component spatiotemporal pruning algorithm reduces computational resource consumption while maintaining high-precision interference elimination, enabling real-time vibration detection to be realized on edge computing nodes.

[0053] The specific steps for calculating the blade tip amplitude include: extracting the voxel displacement sequence corresponding to the blade tip position from the actual vibration displacement trajectory, wherein the blade tip position is defined as the voxel point farthest from the rotation center along the blade's main axis; performing time-domain analysis on the voxel displacement sequence; calculating the amplitude of each axial component of the voxel displacement sequence in the three-dimensional spatial coordinate system; taking the square root of the sum of the squares of the three axial components to obtain the composite displacement amplitude; applying a sliding window maximum value extraction to the composite displacement amplitude sequence, wherein the window length is set to 200 frames corresponding to 1 second; and using the maximum displacement amplitude within the window as the blade tip amplitude at the current moment.

[0054] The amplitude adaptive adjustment function is used to dynamically adjust the sampling point density of the adaptive holographic neural radiation field vibration network and the deblurring intensity of the deblurring adversarial reconstruction network according to the leaf tip amplitude. The amplitude adaptive adjustment function calculates the adjustment coefficient based on three parameters: the current leaf tip amplitude, the historical average amplitude, and the amplitude change rate. The amplitude adaptive adjustment function is described as follows: ;in This represents the current blade tip amplitude, in meters. The reference amplitude is 0.5 meters, obtained by statistically analyzing the median of the blade tip amplitude distribution under normal operating conditions. The average historical amplitude over the past 60 seconds, in meters. This represents the rate of change of amplitude, expressed in meters per second. It is calculated as the difference between the current tip amplitude and the tip amplitude of the previous second. The reference amplitude change rate, with a value of 0.1 m / s, was obtained by analyzing the median of the amplitude change rate distribution of 100 sets of blade vibration data; when the adjustment coefficient... When the value is less than 0.8, the sampling point density remains at the baseline value, the deblurring intensity is set to low intensity mode, and the adversarial loss weight is reduced to 0.5 times the baseline value; when the adjustment coefficient is... When ∈ [0.8, 1.5), the sampling point density is set to twice the baseline value, the deblurring intensity is set to medium intensity mode, and the adversarial loss weight remains at the baseline value; when the adjustment coefficient When ∈ [1.5, 2.5), the sampling point density is set to 4 times the baseline value, the deblurring intensity is set to high intensity mode, and the adversarial loss weight is increased to 2 times the baseline value; when the adjustment coefficient When the value is ≥2.5, the sampling point density is set to 8 times the baseline value, the deblurring intensity is set to ultra-high intensity mode, the anti-loss weight is increased to 5 times the baseline value, and the blade vibration warning signal is triggered at the same time.

[0055] The safety threshold is the upper limit of the blade tip amplitude. According to the wind turbine design specifications, there is a risk of structural fatigue when the blade tip amplitude exceeds 5% of the blade length. For a wind turbine with a blade length of 60 meters, the safety threshold is set at 3 meters. The value of the safety threshold is determined by referring to the blade deformation limit under extreme load conditions in the wind turbine design standard IEC61400-1. When the blade tip amplitude is detected to exceed the safety threshold, the system immediately sends an early warning signal to the wind farm monitoring center, and at the same time records the complete three-dimensional voxel-level displacement field data of the blade and the cleared image sequence at the moment of vibration exceeding the limit for subsequent fault diagnosis and analysis.

[0056] The hardware-level clock trigger uses a GPS clock synchronization module. The three global shutter high-speed cameras and the high-frequency strobe laser illumination device all receive the pulse signal output by the GPS clock synchronization module as the trigger source. The pulse signal has a pulse frequency of 200Hz and a pulse rise edge error of less than 10 nanoseconds, ensuring that the deviation of the original image stream acquired by the three global shutter high-speed cameras on the time axis is less than 50 microseconds. The deviation limit is calculated by determining that the leaf tip displacement is 3.5 mm within 50 microseconds under the condition of a leaf tip linear velocity of 70 m / s. The leaf tip displacement is less than the physical size of a single pixel, which is 5 mm, thus meeting the sub-pixel level displacement measurement accuracy requirements.

[0057] The blade contour feature is the edge contour geometry information extracted from the blade region image using the Canny operator, and the blade texture feature is the statistical distribution information of surface grayscale changes in the blade region image.

[0058] The suspected feature points are voxel points in the three-dimensional voxel-level displacement field of the blade with a displacement amplitude greater than 0.01 meters. The displacement amplitude threshold of 0.01 meters is obtained by statistically analyzing the 95th percentile of the noise displacement distribution in the static blade test data.

[0059] The dynamic cloud background interference refers to the false displacement signal generated in the image by the movement of clouds, and the light refraction interference refers to the drastic change in image brightness caused by the refraction of sunlight through the atmosphere.

[0060] The parameter adjustment instructions include increasing the sampling point density to 8 times the baseline value and increasing the deblurring intensity to an ultra-high intensity mode, which are used to improve displacement detection accuracy when the blade tip amplitude exceeds the safety threshold.

[0061] In addition, the present invention also provides a real-time displacement and vibration detection system for wind turbine blades implemented by a computer. The computer is provided with a readable storage medium, which stores program instructions. When the program instructions are run in the computer, they are used to execute the above-mentioned real-time displacement and vibration detection method for wind turbine blades.

[0062] The specific implementation of step S01 is as follows: Supports are fixed at 120-degree intervals along the circumference of the wind turbine tower's outer wall. Three global shutter high-speed cameras are mounted on these supports, ensuring that the optical axes of all three cameras point towards the center area of ​​the blade's sweeping plane. This arrangement ensures that at least two cameras can obtain clear frontal images of the blade at any rotation angle, thus providing sufficient multi-view constraints for subsequent 3D reconstruction. A high-frequency stroboscopic laser illumination device is fixedly installed at the top of the tower, with its illumination angle covering the entire rotation range of the blade. The hardware-level clock trigger uses a GPS clock synchronization module. This module simultaneously outputs a 200Hz pulse trigger signal to the three global shutter high-speed cameras and the high-frequency stroboscopic laser illumination device. The pulse rise edge error is controlled within 10 nanoseconds, ensuring that the synchronization deviation between the exposure time of the three cameras and the stroboscopic time of the illumination device on the time axis is less than 50 microseconds, meeting the sub-pixel-level time synchronization accuracy requirements under blade tip linear velocity conditions.

[0063] The specific implementation of step S02 is as follows: Three global shutter high-speed cameras continuously acquire the original image stream of the blade rotation process at a frame rate of 200 frames per second, driven by GPS trigger signals. The global shutter mechanism ensures that the exposure time of all pixels in the same frame is completely consistent, avoiding the inter-line timing distortion caused by the rolling shutter. At the same time, the system synchronously acquires the trigger electrical signal of the generator-side exciter, and uses the timestamp of this electrical signal as the time reference label of the image sequence for phase alignment during subsequent vibration frequency analysis. The original image stream is then fed into a deblurring adversarial reconstruction network, which employs an encoder-decoder structure. The encoder extracts multi-scale depth features of the blurred image through 5 convolutional layers, while the decoder restores spatial resolution and reconstructs a sharp image layer by layer through 5 deconvolutional layers. Skip connections directly transmit the features from each layer of the encoder to the corresponding layer of the decoder, preserving high-frequency edge information. A 3-layer fully convolutional adversarial discriminator is added at the end of the decoder, using binary cross-entropy adversarial loss to constrain the visual realism of the reconstructed image. The weight ratio of content loss to adversarial loss is set to 100:1. The Adam optimizer is used during training, with an initial learning rate of 0.0002, for a total of 200 training epochs. The model parameters with the highest peak signal-to-noise ratio on the validation set are saved, and the final output is a sequence of sharpened images.

[0064] The specific implementation of step S03 is as follows: At the edge computing node, dynamic cropping of the blade region is performed frame by frame on the sharpened image sequence. First, the Canny operator is applied to each frame to extract the edge contour. After the Canny operator suppresses noise through Gaussian filtering, the pixel gradient magnitude and direction are calculated, and double threshold non-maximum suppression is used to obtain the edge response of a single pixel width. Then, Hough linear transform is performed on the extracted edge contour, and the blade main axis direction is identified by accumulating votes in the parameter space. The width of the blade chord is expanded to both sides by 1.2 times the width of the main axis direction to form the bounding box of the region of interest, and the region is cropped as the blade region image. The leaf region image is then input into a lightweight convolutional neural network for spatial dimensionality reduction encoding. This network contains three depthwise separable convolutional layers with output channels of 32, 64, and 128 respectively, and a kernel size of 3×3. The depthwise separable convolution decomposes standard convolution into channel-wise convolution and pointwise convolution, significantly reducing the computational cost. The last depthwise separable convolutional layer is followed by a global average pooling layer, which compresses the feature map of arbitrary size into a 128-dimensional feature vector, reducing the data volume to 0.05 times that of the original image. The 128-dimensional reduced feature vector is then transmitted to the central server via the network.

[0065] The specific implementation of step S04 is as follows: The central server receives 128-dimensional reduced feature vectors from three perspectives and the corresponding camera pose parameters, and inputs the above data into an adaptive holographic neural radiation field vibration network. The core of this network is a multilayer perceptron structure containing 8 fully connected layers, with 256 neurons in each layer, and the ReLU function is used as the activation function. The network takes three-dimensional spatial coordinates and time encoding as input. The position encoding module uses sine and cosine functions to encode 10 frequency components of the three-dimensional coordinates, mapping low-frequency spatial coordinates to high-frequency feature space, enabling the network to learn detailed high-frequency geometric information; the time encoding module maps timestamps to 64-dimensional time feature vectors, enabling the network to implicitly model the temporal vibration dynamics of the leaf. The network outputs volume density and RGB color, and projects three-dimensional voxels onto a two-dimensional image plane through differentiable volume rendering, using reprojection error as a supervision signal to optimize network parameters. During the volume rendering process, a gradient adaptive jump mechanism is introduced to calculate the displacement temporal gradient of each 3D point. When the absolute value exceeds the preset gradient threshold of 0.015, the sampling point density is increased to 4 times the original density in a spherical region with a radius of 0.01 meters around the point, thereby capturing the sub-pixel level micro-displacement in the area of ​​severe vibration, and finally outputting the 3D voxel level displacement field of the blade and the instantaneous displacement of the blade tip.

[0066] The specific implementation of step S05 is as follows: The Königsberg graph theory connected component spatiotemporal pruning algorithm is applied to the three-dimensional voxel-level displacement field of the blade. First, all suspected feature voxels with displacement amplitudes greater than 0.01 meters are traversed, and the three-dimensional spatial coordinates and displacement vectors of each voxel are stored as vertex attributes of the graph. For each voxel, neighboring voxels within a spatial neighborhood radius of 0.05 meters are searched, and the Pearson correlation coefficient between the current voxel and the displacement vectors of its neighboring voxels is calculated as the directed edge weight, constructing an adjacency matrix of the global displacement correlation graph. Then, a depth-first search stack is initialized, and traversal begins from the voxel with the largest displacement amplitude. During the traversal, the weight of each directed edge is checked; if the weight is less than the preset correlation coefficient threshold of 0.65, the edge is pruned, and the target vertex is marked as unvisited. This process is iterated until the stack is empty. The set of traversed vertices is counted as strongly connected components, and the strongly connected component with the most vertices is retained. The displacement values ​​of all voxels outside the strongly connected components are set to zero, completing the background interference removal and preserving the true vibration displacement trajectory.

[0067] The specific implementation of step S06 is as follows: Extract the voxel displacement sequence corresponding to the blade tip position from the actual vibration displacement trajectory. The blade tip position is defined as the voxel point furthest from the rotation center along the blade's principal axis. Calculate the displacement amplitude of each of the three axial components of the voxel displacement sequence in a three-dimensional coordinate system. Take the square root of the sum of the squares of the three axial components to obtain a synthetic displacement amplitude sequence. Then, apply a sliding window of 200 frames (corresponding to 1 second) to the synthetic displacement amplitude sequence, and take the maximum value within the window as the blade tip amplitude at the current moment. The amplitude adaptive adjustment function adjusts according to the current blade tip amplitude. Historical average amplitude over the past 60 seconds With the rate of change of amplitude Calculate the adjustment coefficient ,in accordance with The value range dynamically adjusts the sampling point density of the adaptive holographic neural radiation field vibration network and the deblurring intensity of the deblurring adversarial reconstruction network; when When the vibration amplitude is ≥2.5, a blade vibration early warning signal is triggered synchronously. When the blade tip amplitude exceeds the safety threshold, the system sends an early warning signal to the wind farm monitoring center and records the three-dimensional voxel-level displacement field data and the clear image sequence of the blade at the moment of vibration exceeding the limit for subsequent fault diagnosis and analysis.

[0068] It should be noted that the key technologies of this invention include: the deblurring adversarial reconstruction network, through the joint constraint of skip connections and adversarial discriminators, restores the high-frequency information of the leaf edges erased by high-speed rotation in motion-blurred images, enabling the subsequent 3D reconstruction network to obtain a clear input containing accurate contour information, thus fundamentally eliminating the limitation of traditional methods where feature extraction fails due to blurring; the adaptive holographic neural radiation field vibration network encodes 3D spatial coordinates and timestamps into continuous function inputs, bypassing the cumulative error path of explicit feature point matching and triangulation through implicit neural representation, and the gradient adaptive jump mechanism further automatically densifies sampling in areas of severe vibration, enabling displacement measurement accuracy to break through the bottleneck of image resolution discretization; the Königsberg graph theory connected component spatiotemporal pruning algorithm transforms the spatial continuity of vibration displacement into strongly connected component constraints in graph theory, separating isolated interference voxels from highly connected real vibration regions through depth-first search and dynamic edge pruning, achieving high-precision background removal while maintaining linear time complexity. The three key technologies are linked together: front-end blur removal provides quality assurance for mid-end 3D reconstruction; mid-end implicit representation provides a complete 3D displacement field for back-end interference removal; and back-end graph theory pruning provides a clean vibration signal for final amplitude calculation. The three technologies work together to enable the entire detection link to maintain stable sub-pixel-level 3D displacement reconstruction capability under extreme combined interference of motion blur, complex backgrounds, and large deformations of non-rigid bodies. This is something that no single technology can achieve independently.

[0069] It should be noted that this invention also solves the following technical problem: In wind turbine blade vibration detection systems, as the blade vibration state dynamically changes from normal to severe vibration conditions, the fixed-parameter detection system suffers from sparse sampling points and insufficient deblurring intensity under high-amplitude conditions, leading to a significant decrease in detection accuracy as vibration intensity increases. This invention addresses this by using an amplitude adaptive adjustment function to calculate the adjustment coefficient based on a weighted combination of three parameters: the current blade tip amplitude, the historical average amplitude, and the amplitude change rate. ,in accordance with The system automatically increases the sampling point density of the adaptive holographic neural radiation field vibration network from the baseline value to 8 times in the specified interval, and increases the adversarial loss weight of the deblurred adversarial reconstruction network from the baseline value to 5 times in the specified interval. This enables the system to automatically enhance the sensing accuracy when the vibration intensifies and maintain a low computational load when the vibration is stable. This achieves an adaptive dynamic match between detection accuracy and computational resource consumption with the vibration state, and solves the technical problem of uneven accuracy of fixed parameter detection systems over a wide amplitude range.

[0070] Specifically, the principle of this invention is as follows: The fundamental reason why this invention can solve the above-mentioned technical problems lies in the fact that three core technical links form a mutually supportive logical closed loop. First, the deblurring adversarial reconstruction network, through the joint constraints of the encoder-decoder structure and the adversarial discriminator, restores the motion-blurred image to a clear image containing high-frequency information of the blade edge, ensuring the input quality of the subsequent neural radiation field network from the source. Second, the adaptive holographic neural radiation field vibration network uses three-dimensional spatial coordinates and time encoding as inputs of continuous functions, implicitly learns the volume density and color distribution of the blade surface through a multilayer perceptron, and introduces a gradient adaptive jump mechanism to automatically encrypt sampling in areas of severe vibration, so that the displacement measurement accuracy is not limited by the discretization of image resolution, fundamentally breaking through the accuracy bottleneck of traditional explicit feature point matching methods in non-rigid large deformation scenarios. Finally, the Königsberg graph theory connected component spatiotemporal pruning algorithm utilizes the physical characteristics that the real vibration displacement has high spatial connectivity while the background interference voxels are isolated, and separates the interference voxels from the real vibration region through depth-first search and dynamic edge pruning, ensuring the purity of the final displacement field. The three steps work sequentially and are logically consistent, making the present invention theoretically complete.

[0071] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0072] The specific implementation of step S01 is as follows: Three global shutter high-speed cameras are arranged circumferentially around the wind turbine tower, with the tower axis as the center, at 120-degree intervals. The horizontal distance between the three cameras and the tower is equal to ensure multi-view coverage of the entire blade rotation area. A high-frequency stroboscopic laser illumination device is installed on the top of the tower, with the illumination wavelength range covering the camera's photosensitive band. All three global shutter high-speed cameras and the high-frequency stroboscopic laser illumination device are connected to a Global Positioning System (GPS) clock synchronization module, receiving a 200Hz pulse signal output from this module as a unified trigger source. The pulse rise edge error is less than 10 nanoseconds, ensuring that the time deviation of the original image stream acquired by the three cameras is less than 50 microseconds, meeting the sub-pixel displacement measurement accuracy requirements. The trigger frequency of the high-frequency stroboscopic laser illumination device is set to 200Hz, and the single stroboscopic pulse width is set to 500 microseconds. This pulse width value is determined by... Up to 25 Under 10 working conditions within the wind speed range, the radius of the blur kernel at the blade edge was determined for 20 discrete pulse width values ​​ranging from 100 microseconds to 1000 microseconds. The minimum pulse width that reduces the blur kernel radius to less than 2 pixels was selected as the optimal value.

[0073] The specific implementation of step S02 is as follows: Three global shutter high-speed cameras synchronously acquire the original image stream of the blade rotation at a frame rate of 200 frames per second. Simultaneously, the trigger electrical signal of the generator-side exciter is recorded as a time reference label. The original image stream is then input into a deblurring adversarial reconstruction network. This network adopts an encoder-decoder structure, with the encoder containing 5 convolutional layers and the decoder containing 5 deconvolutional layers. Skip connections are introduced between each layer. The last layer of the decoder is supplemented with a 3-layer fully convolutional adversarial discriminator, outputting a single-channel probability map to distinguish between the real, clear image and the reconstructed image. The training dataset was collected from a laboratory blade scale-down model rotation platform, with a motor driving a 3-meter scale-down blade at 10... Up to 60 Rotation was achieved using an experimental high-speed camera with exposure times ranging from 1 millisecond to 10 milliseconds, resulting in 6000 sets of blurred images. Simultaneously, clear images of the same scene were captured using a microsecond-level ultra-high-speed camera and paired to form training samples. The training and validation sets were divided in an 8:2 ratio. Network loss function. Content loss and the loss of combat The weighted combination is expressed by the following formula:

[0074] ;

[0075] In the formula, The total loss of the deblurred adversarial reconstruction network is a dimensionless quantity. The pixel-level mean square error between the sharpened and reconstructed images is expressed in units of 1 (normalized pixel intensity squared). To counteract the binary cross-entropy output of the discriminator, it is a dimensionless quantity. As a reference normalized measure for content loss, take The mean of the initial training batches, in units of same To counteract the loss, a reference normalization is taken. The mean of the initial training batch is a dimensionless quantity; the weight ratio of the two is 100:1. The training batch size is set to 16, the initial learning rate is 0.0002, and an adaptive moment estimation optimizer is used. The peak signal-to-noise ratio is evaluated on the validation set every 10 epochs. If there is no improvement for 5 consecutive epochs, the learning rate is reduced to 0.1 times the original value. A total of 200 epochs are trained, and the model parameters with the highest peak signal-to-noise ratio on the validation set are saved.

[0076] The specific implementation of step S03 is as follows: Cannibalistic edge detection is performed on each frame of the sharpened image sequence. The main axis direction of the blade is identified through Hough linear transform. The blade is then expanded to both sides with a preset width of 1.2 times the chord length, centered on the main axis direction, to crop the region of interest image. The cropped image is input into a lightweight convolutional neural network containing three depthwise separable convolutions. The number of output channels for the three layers are 32, 64, and 128, respectively, and the kernel size is 3×3. Finally, a global average pooling layer is applied to compress the feature vector to a 128-dimensional feature vector. The data volume is reduced to 0.05 times that of the original image and transmitted to the central server via the network. In the formula, The 128-dimensional reduced feature vector output by the edge computing node contains spatial encoding information of leaf contour features and texture features.

[0077] The specific implementation of step S04 is as follows: The 128-dimensional feature vectors from three perspectives... An adaptive holographic neural radiation field vibration network is constructed based on the corresponding camera pose parameters. The core of this network is a multilayer perceptron with eight fully connected layers, each layer containing 256 neurons, and the activation function is a linear rectified function. (Three-dimensional spatial coordinates) After being mapped to the high-frequency feature space by the location encoding module, the location encoding formula is as follows:

[0078] ;

[0079] In the formula, For position encoding vectors, the dimension is 60, and for , , The three coordinate components are respectively based on their frequency coefficients. ( It is formed by calculating the sine and cosine values ​​and then piecing them together. , , These are the three components of the three-dimensional spatial coordinates, with units of . Here, is the frequency coefficient, a dimensionless quantity, and is related to the coordinate components (unit: ). In the product obtained after multiplying, and Together they constitute the angular frequency factor. The input to the position encoding function here is actually the coordinate normalized value, and the coordinates have been adjusted according to the scene scale reference value. (The empirical value is the blade length, in units of...) Normalized to a dimensionless quantity before substitution, the actual substituted value is... , , This ensures that the independent variable of the trigonometric function is a dimensionless quantity. The time encoding module will use timestamps. (Unit is) After normalization, it is mapped to a 64-dimensional time feature vector. The normalization method is ,in This is a time reference value; the empirical value is 1. The actual input to the network is a position encoding vector. With time feature vector spliced ​​vector The dimension is 124. The network output volume density... (Unit is) ) and color value (Normalized intensity, a dimensionless quantity), a two-dimensional image is synthesized through differentiable volume rendering, with reprojection error used as a supervisory signal to optimize parameters. A gradient adaptive jump mechanism calculates the temporal gradient of displacement at each 3D point. ,when Intensify sampling when the following conditions are met:

[0080] ;

[0081] In the formula, For the first The displacement time-series gradient of a 3D point, in units of For the first A three-dimensional point at time... The combined displacement amplitude, in units of The preset gradient threshold is in units of After searching a grid of 10 sets of vibration data with different amplitudes, the optimal value of 0.015 was selected. ;when At that point, the radius is 0.01. Within the spherical region, the sampling point density is increased to four times the original density to capture sub-pixel-level minute displacements. The total loss during training of the adaptive holographic neural radiation field vibration network. It consists of three parts, and the formula is expressed as follows:

[0082] ;

[0083] In the formula, The total loss of the adaptive holographic neural radiation field vibration network is a dimensionless quantity. The mean squared error between the network-rendered image and the real image, expressed in units of 1 (normalized pixel intensity squared). The absolute error between the network-predicted displacement and the displacement measured by the accelerometer, in units of... The constraint loss is the gradient of displacement change between adjacent frames, in units of for The reference normalized value is taken as The mean of the initial training batches, in units of same for The reference normalized value is taken as The mean value on the initial training batch, in units of for The reference normalized value is taken as The mean value on the initial training batch, in units of The weights of the three components are 1:10:0.1. The batch size is set to 4, the initial learning rate is 0.0005, and a weight decay adaptive moment estimator optimizer is used. The displacement prediction error is evaluated on the validation set every 5 rounds. If the error does not decrease for 3 consecutive rounds, the learning rate is reduced to 0.5 times the original value. The training is carried out for a total of 100 rounds, and the model parameters with the smallest displacement prediction error on the validation set are saved.

[0084] The specific implementation of step S05 is as follows: for the displacement amplitude in the three-dimensional voxel-level displacement field of the blade greater than 0.01... A global displacement correlation graph is constructed from the suspected feature voxels, where each voxel is abstracted as a vertex, and the search neighborhood radius is 0.05. Calculate voxels from adjacent voxels within the range. With voxels Pearson correlation coefficient of displacement vector As the weight of a directed edge, the formula is expressed as follows:

[0085] ;

[0086] In the formula, voxels With voxels The Pearson correlation coefficient between the displacement vectors is a dimensionless quantity. voxels In the Frame displacement components, in units of voxels In the Frame displacement components, in units of voxels exist Mean displacement within the frame, in units of voxels exist Mean displacement within the frame, in units of For the number of frames in the time window, After constructing the adjacency matrix, a depth-first search traversal is performed starting from the voxel with the largest displacement magnitude. During the traversal, if... The directed edge is then pruned, and the target voxel is marked as unvisited until the search stack is empty. Finally, the strongly connected component with the most vertices is retained, and the displacement values ​​of the remaining voxels are set to zero to complete background stripping. A preset correlation coefficient threshold of 0.65 is determined by performing a grid search on 50 sets of test data containing varying degrees of background interference, with candidate values ​​ranging from 0.3 to 0.9 and a step size of 0.05. The threshold with the highest harmonic mean between the accuracy of true displacement recognition and the false interference removal rate is selected.

[0087] The specific implementation of step S06 is as follows: extract the blade tip voxel displacement sequence from the actual vibration displacement trajectory. The blade tip position is defined as the voxel point farthest from the rotation center along the blade's principal axis. Calculate the composite displacement amplitude of each axial displacement component in the three-dimensional coordinate system. The formula is expressed as follows:

[0088] ;

[0089] In the formula, For the first The composite displacement amplitude of the leaf tip voxel, in units of , , The first Leaf tip voxels in , , Axial displacement components, in units of .right The sequence is processed using a sliding window with a window length of 200 frames to extract the maximum value, thus obtaining the tip amplitude at the current moment. Amplitude adaptive adjustment coefficient The calculation formula is expressed as follows:

[0090] ;

[0091] In the formula, Here, is the amplitude adaptive adjustment coefficient, and is a dimensionless quantity. This represents the current blade tip amplitude, in units of... For reference amplitude, the empirical value is 0.5. The median of the blade tip amplitude distribution under normal operating conditions was obtained by statistical analysis. The average historical amplitude over the past 60 seconds, in units of The amplitude change rate is expressed in units of 1 / 2. Take the difference between the current blade tip amplitude and the blade tip amplitude of the previous second. The empirical value for the rate of change of amplitude is 0.1. The value was obtained by analyzing the median of the amplitude change rate distribution of 100 sets of blade vibration data; the first term of the formula... The second item reflects the overall level of the current amplitude and historical amplitude. Both reflect the dynamic trend of amplitude change and are dimensionless quantities. The sampling point density is maintained at the baseline value, the deblurring intensity is set to low intensity mode, and the adversarial loss weight is reduced to 0.5 times the baseline value; when The sampling point density is set to twice the baseline value, the deblurring intensity is set to medium intensity mode, and the adversarial loss weight is kept at the baseline value; when The sampling point density is set to 4 times the baseline value, the deblurring intensity is set to high intensity mode, and the adversarial loss weight is increased to 2 times the baseline value; when The sampling point density was set to 8 times the baseline value, the deblurring intensity was set to ultra-high intensity mode, the anti-loss weight was increased to 5 times the baseline value, and a blade vibration warning signal was triggered simultaneously. The safety threshold was based on wind turbine design standards, with a blade length of 60 mm. Time setting is 3 ,when When the safety threshold is exceeded, an early warning signal is immediately sent to the wind farm monitoring center, and complete displacement field data and clear image sequences at the time of exceeding the limit are recorded for fault diagnosis and analysis.

[0092] To better understand and implement this invention, the following is a specific application scenario of the invention, Example 2: To illustrate the effect of the invention, technicians set up a test environment and conducted a real-time displacement vibration detection experiment on a horizontal axis wind turbine with a blade length of 60 meters and a rated speed of 12 revolutions per minute in a wind farm. The technical solution of this invention was used to record and analyze the blade vibration process of the wind turbine during a strong gust of wind.

[0093] In terms of experimental hardware configuration, three global shutter high-speed cameras were mounted on a fixed bracket 18 meters from the center of the tower. The azimuth interval between adjacent cameras was strictly 120 degrees, the lens focal length was set to 50mm, and the physical size calibration result corresponding to a single pixel was 5.2mm, meeting the spatial resolution requirements for sub-pixel displacement measurement. A high-frequency stroboscopic laser illumination device was installed at the top of the tower. The pulse width of a single stroboscopic laser was determined to be 500μs after experimental testing of blade edge sharpness within a wind speed range of 5m / s to 25m / s. The GPS clock synchronization module simultaneously outputs a 200Hz pulse trigger signal to the three cameras and the illumination device. The measured pulse rise time error was 7 nanoseconds, and the synchronization deviation between image frames of the three cameras was 32μs, which is better than the design specifications.

[0094] During a strong gust of wind, the wind speed rapidly increased from 8 m / s to 21 m / s in approximately 90 seconds, then recovered to 10 m / s in about 40 seconds. Three cameras captured a raw image stream at a rate of 200 frames per second throughout the event, resulting in 26,000 raw images. After processing by a deblurring adversarial reconstruction network, the average radius of the blur kernel at the blade edges decreased from 8.3 pixels before processing to 1.7 pixels after processing, effectively restoring edge details. After dynamic cropping and spatial dimensionality reduction encoding of the blade region were completed at the edge computing nodes, the amount of data transmitted to the central server per frame decreased from the original 6.2 MB to approximately 4.8 KB, a significant reduction that met the bandwidth constraints for real-time transmission.

[0095] The central server runs an adaptive holographic neural radiation field vibration network to process the dimensionality-reduced feature vector and camera pose parameters, reconstructing a three-dimensional voxel-level displacement field of the blade. During the wind speed increase phase, the system detects a continuous increase in the amplitude of the composite displacement at the blade tip, and the adjustment coefficient... As the value gradually increases from 0.72 to 2.61, the system sequentially enters four working modes: low intensity, medium intensity, high intensity, and ultra-high intensity. The sampling point density gradually increases from the baseline value to 8 times, and the deblurring intensity adversarial loss weight gradually increases from the baseline value to 5 times.

[0096] In the Königsberg graph theory connected component spatiotemporal pruning algorithm processing stage, the system constructs a global displacement correlation graph for the suspected feature point voxels identified in the displacement field, performs depth-first search and performs dynamic edge pruning with a preset correlation coefficient threshold of 0.65. The final proportion of the number of the largest strongly connected component voxels retained to the total number of suspected feature points, as well as the distribution of isolated voxels that are judged as background interference and removed, are shown in Table 1.

[0097] Table 1. Statistical table of voxel connected components of suspected feature points at different wind speed stages.

[0098]

[0099] As shown in Table 1, as the wind speed increases, the number of isolated interference voxels generated by the dynamic cloud background and light refraction increases. However, the Königsberg graph theory connected component spatiotemporal pruning algorithm can always effectively separate isolated interference voxels from the maximum strongly connected component by utilizing the spatial continuity constraint of the real vibration displacement, thus ensuring the purity of the real vibration displacement trajectory.

[0100] Based on the preserved true vibration displacement trajectory, the system calculates the changes in the blade tip amplitude sequence as follows: Figure 2As shown in the figure, the horizontal axis represents time, and the vertical axis represents the blade tip amplitude, allowing for a direct observation of the dynamic response process of the blade tip amplitude as wind speed changes. During the peak wind speed phase, the maximum sliding window value of the composite blade tip amplitude reached 2.71m, lower than the safety threshold of 3m corresponding to a blade length of 60m. The system maintained an ultra-high intensity operating mode but did not trigger the final warning signal, indicating that the structural load of the wind turbine during this strong gust event remained within a safe range. The amplitude adaptive adjustment function is based on... The real-time changes in values ​​correctly drive the dynamic switching of parameters at all levels, ensuring the balance of detection accuracy across a wide amplitude range.

[0101] Compared to traditional contact sensors and 2D vision detection methods, this invention fundamentally bypasses the error accumulation mechanism of explicit feature point matching in non-rigid large deformation scenarios through implicit neural representation, making displacement measurement accuracy unconstrained by image resolution discretization. The deblurring adversarial reconstruction network recovers the high-frequency geometric information of the edges of high-speed rotating blades, overcoming the fundamental defect of traditional methods where feature extraction fails due to motion blur. The Königsberg graph theory connected component spatiotemporal pruning algorithm uses the physical continuity constraint of vibration displacement to replace traditional pixel-level color texture segmentation, fundamentally eliminating the pollution of the displacement field by outdoor dynamic backgrounds. The amplitude adaptive adjustment function maintains the balance of detection accuracy over a wide amplitude range through dynamic parameter adjustment, compensating for the performance degradation of traditional fixed parameter systems under extreme conditions.

[0102] It should be noted that the variables involved in this invention are explained in detail in Table 2.

[0103] Table 2 Variable Explanation Table

[0104]

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition, characterized in that, Includes the following steps: Three global shutter high-speed cameras are arranged at 120-degree intervals around the wind turbine tower. At the same time, a high-frequency stroboscopic laser supplementary lighting device is installed on the top of the tower. The three global shutter high-speed cameras and the high-frequency stroboscopic laser supplementary lighting device achieve microsecond-level synchronous acquisition through a hardware-level clock trigger. Three global shutter high-speed cameras acquire raw image streams of the blade rotation process at a frame rate of 200 frames per second. Simultaneously, the trigger electrical signal of the generator side exciter is recorded as a time reference label. The raw image stream is input into a deblurring adversarial reconstruction network for motion blur elimination processing to obtain a sharpened image sequence. At the edge computing node, dynamic cropping of the leaf region is performed on the sharpened image sequence, the leaf contour features and leaf texture features are extracted and spatial dimensionality reduction encoding is performed to obtain the dimensionality reduction feature vector, and the dimensionality reduction feature vector is transmitted to the central server through the network. The dimensionality-reduced feature vector and the camera pose parameters of three global shutter high-speed cameras are input into an adaptive holographic neural radiation field vibration network. The three-dimensional voxel-level displacement field of the blade is reconstructed through multi-view implicit neural representation, and the instantaneous displacement of the blade tip in the three-dimensional spatial coordinate system is calculated. The Königsberg graph theory connected component spatiotemporal pruning algorithm is applied to suspected feature points in the three-dimensional voxel-level displacement field of the blade to remove dynamic cloud background interference and illumination refraction interference, while retaining the true vibration displacement trajectory. The blade tip amplitude is calculated based on the actual vibration displacement trajectory. The current vibration state is determined by the amplitude adaptive adjustment function. When the blade tip amplitude exceeds the safety threshold, the parameter adjustment command is triggered and an early warning signal is generated. The training loss function of the deblurred adversarial reconstruction network is composed of a weighted combination of content loss and adversarial loss. The content loss is the pixel-level mean square error between the clear image and the reconstructed image, and the adversarial loss is the binary cross-entropy output by the adversarial discriminator. The weight ratio of content loss to adversarial loss is set to 100:

1. The spatial dimensionality reduction encoding specifically involves: inputting the leaf region image into a lightweight convolutional neural network for feature extraction. The lightweight convolutional neural network contains three depthwise separable convolutional layers with output channels of 32, 64, and 128, respectively. After the last depthwise separable convolutional layer, a global average pooling layer is connected to compress the feature map into a 128-dimensional feature vector. The core of the adaptive holographic neural radiation field vibration network adopts a multilayer perceptron structure containing 8 fully connected layers, with 256 neurons in each layer. The activation function is the ReLU function. The position encoding module uses sine and cosine functions to encode 10 frequency components of the three-dimensional spatial coordinates. The time encoding module maps the timestamp to a 64-dimensional time feature vector. The adaptive holographic neural radiation field vibration network employs a gradient adaptive jump mechanism during volume rendering. It calculates the displacement temporal gradient of each three-dimensional point position. When the absolute value of the displacement temporal gradient is greater than the preset gradient threshold, the sampling point density is increased to 4 times the original density within a spherical region with a radius of 0.01 meters around the three-dimensional point position. The reference value of the preset gradient threshold is 0.

015.

2. The method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition according to claim 1, characterized in that, The hardware-level clock trigger uses a GPS clock synchronization module. The three global shutter high-speed cameras and the high-frequency strobe laser illumination device all receive the pulse signal output by the GPS clock synchronization module as the trigger source. The pulse frequency of the pulse signal is 200Hz, and the pulse rise edge error is less than 10 nanoseconds.

3. The method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition according to claim 2, characterized in that, The trigger frequency of the high-frequency stroboscopic laser supplementary lighting device is synchronized with the exposure time of the three global shutter high-speed cameras. The trigger frequency is set to 200Hz. The single stroboscopic pulse width is obtained by experimentally testing the sharpness of the blade edge under different wind speed conditions. The minimum pulse width that reduces the radius of the blade edge blur kernel to less than 2 pixels is selected as the optimal value.

4. The method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition according to claim 3, characterized in that, The deblurred adversarial reconstruction network adopts an encoder-decoder structure. The encoder part contains 5 convolutional layers, and the decoder part contains 5 deconvolutional layers. The deblurred adversarial reconstruction network introduces a skip connection structure to fuse the features of each layer of the encoder with the corresponding layer of the decoder. An adversarial discriminator is added to the last layer of the decoder. The adversarial discriminator is a 3-layer fully convolutional network that outputs a single-channel probability map.

5. The method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition according to claim 4, characterized in that, The specific dynamic cropping of the blade region is as follows: for each frame of the sharpened image sequence, the edge contour is extracted using the Canny operator, the extracted edge contour is subjected to Hough linear transform to identify the blade main axis direction, and the preset width of 1.2 times the blade chord length is extended to both sides with the blade main axis direction as the center to obtain the bounding box of the region of interest, and the image region within the bounding box of the region of interest is cropped.

6. The method for real-time displacement and vibration detection of wind turbine blades based on neural network visual recognition according to claim 5, characterized in that, The Königsberg graph theory connected component spatiotemporal pruning algorithm abstracts each voxel in the three-dimensional voxel-level displacement field of the leaf as a vertex of the graph, and the displacement relationship between adjacent voxels is abstracted as a directed edge. The weight of the directed edge is defined as the Pearson correlation coefficient of the displacement vector between adjacent voxels. Strongly connected components are identified through depth-first search. When the weight of the directed edge is less than a preset correlation coefficient threshold, a dynamic pruning operation is performed. The reference value of the preset correlation coefficient threshold is 0.65.

Images