Deep learning-based offshore wind power foundation equipment stress monitoring and early warning system

Abstract

The application discloses a deep learning-based offshore wind power foundation equipment stress monitoring and early warning system, comprising a data acquisition module, a mapping module, a fringe image prediction module, a fusion module, a model construction module, a stress module and an early warning module. The application converts the obtained multi-dimensional monitoring data into first fringe images consistent with the space-time of photoelastic fringe images through the mapping module and the fringe image prediction module, then fuses the first fringe images with the photoelastic fringe images, and uses the stress module to predict the stress distribution of offshore wind power foundation equipment. Abnormal features in the stress distribution map are identified, and potential risks are judged in advance and early warning is issued in combination with a safety threshold. The application fuses the information missing of the first fringe images and the photoelastic fringe images in the same space-time, so that the fused images enable the stress prediction model to capture more key clues related to stress distribution, and the stress distribution can be more accurately predicted.

Description

Technical Field

[0001] This invention relates to the field of offshore wind power monitoring technology, specifically a deep learning-based stress monitoring and early warning system for offshore wind power infrastructure equipment. Background Technology

[0002] Offshore wind power infrastructure has long been subjected to multiple challenges from extreme marine environments: in addition to dynamic loads such as wind, waves, and currents, as well as its own gravity, it also faces complex conditions such as seawater corrosion and sea ice impact, resulting in a spatiotemporal dynamic evolution of the structural stress field. This complex stress environment not only exacerbates structural fatigue damage but may also trigger catastrophic failures, directly threatening the safe operation and economic benefits of offshore wind farms.

[0003] Existing monitoring technologies face significant limitations: traditional sensor monitoring methods, such as strain gauges and accelerometers, are affected by the strong corrosiveness and high humidity of the marine environment, resulting in problems such as large monitoring blind spots, short equipment lifespan, and data transmission delays, making it difficult to meet the needs of full-scale, high-precision monitoring. Numerical simulation methods based on finite element analysis, due to the high uncertainty of marine environmental parameters, suffer from low model accuracy and poor computational efficiency, making it impossible to achieve real-time dynamic assessment of stress states.

[0004] In terms of early warning mechanisms, current monitoring systems generally adopt static threshold early warning strategies, which are unable to capture the nonlinear dynamic characteristics of stress evolution and are difficult to quantify the coupled effects of environmental factors, resulting in a high false alarm rate and insufficient lead time. Summary of the Invention

[0005] The purpose of this invention is to provide a stress monitoring and early warning system for offshore wind power infrastructure based on deep learning. By fusing photoelastic fringe images and multi-dimensional data, the system predicts stress distribution, thereby improving the accuracy and reliability of the prediction.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A deep learning-based stress monitoring and early warning system for offshore wind power foundation equipment includes:

[0008] The data acquisition module is used to collect multidimensional monitoring data and photoelastic fringe images of offshore wind power infrastructure equipment, and to preprocess the collected data. The multidimensional monitoring data includes: strain data collected by distributed fiber optic sensors; vibration data collected by accelerometers; temperature data collected by temperature sensors; wave force, wind speed, and salinity data collected by environmental sensors; strain data collected by distributed fiber optic sensors; and vibration data collected by accelerometers.

[0009] The mapping module establishes a spatial coordinate mapping between the pixels of the photoelastic stripe image and the sensor installation points, based on the three-dimensional model of the offshore wind power infrastructure equipment.

[0010] The stripe image prediction module converts multidimensional monitoring data into spatially aligned data with the photoelastic stripe image based on the spatial coordinate mapping between the pixels of the photoelastic stripe image and the sensor installation points, and then uses the stripe image prediction model to generate the first stripe image.

[0011] The fusion module uses a multi-scale feature fusion network to fuse the first fringe image with the photoelastic fringe image at the same time scale to obtain a fused photoelastic fringe image.

[0012] The model building module constructs a stress prediction model; the stress prediction model includes an encoder and a decoder.

[0013] The stress module inputs the fused photoelastic fringe image into the stress prediction model to obtain a stress distribution map;

[0014] The early warning module identifies abnormal features in the stress distribution map and, in conjunction with safety thresholds, anticipates potential risks and issues warnings in advance.

[0015] Photoelastic fringe imaging utilizes the photoelastic effect, where transparent or translucent materials exhibit birefringence under stress. A circularly polarized light field system can transform the stress distribution into a visual, isochromatic fringe pattern. The density, color, and shape of the fringe directly reflect the magnitude and direction of the stress. Although key structures of offshore wind power foundation equipment (such as concrete bases and composite material connectors) are mostly made of non-transparent materials, stress information can be indirectly transmitted by embedding transparent photoelastic sensing elements (such as PMMA materials) in areas of stress concentration (such as near bolt connections and welds). Alternatively, the stress transmission medium on the equipment surface (such as coatings or bonded photoelastic films) can be monitored, and the internal stress of the equipment can be inverted through the fringe pattern of the medium.

[0016] According to the above technical solution, a three-dimensional rectangular coordinate system is constructed based on the three-dimensional model of the offshore wind power infrastructure equipment to determine the position coordinates of the stress detection device installation point. The mapping relationship of the three-dimensional rectangular coordinate system in the photoelastic fringe image is calculated using a pinhole camera model.

[0017] ;

[0018] In the formula, This represents the sensor's three-dimensional coordinates in the world coordinate system. K represents the pixel coordinates on the two-dimensional image plane after the sensor's three-dimensional coordinates in the world coordinate system are projected through the camera. Represents the extrinsic parameter matrix. Represents the rotation matrix. Represents the translation vector. This represents the depth of the camera coordinate system.

[0019] Camera intrinsic parameter calibration: Zhang Zhengyou calibration method is adopted. Multiple sets of images are collected under different postures using a checkerboard calibration board. The camera intrinsic parameter matrix K is solved by least squares fitting, including core parameters such as focal length and principal point coordinates.

[0020] According to the above technical solution, the stripe image prediction module performs the following steps:

[0021] Based on the projection relationship from the three-dimensional rectangular coordinate system to the pixels of the photoelastic fringe image, the position coordinates of the stress detection device installation point in the corresponding pixels of the photoelastic fringe image are determined. The physical quantities of the stress detection device in the multidimensional monitoring data are assigned to the corresponding pixels, generating a sensor pixel map of the same size as the fringe image. When assigning values ​​to the corresponding pixels, floating-point numbers need to be rounded to integer pixels. Specifically, after normalizing the multidimensional monitoring data to 0-1, a weighted fusion mapping can be used to map the data to pixels, or the multidimensional monitoring data can be stuffed into the R, G, and B channels, or the multidimensional monitoring data can be reduced to 1 / 3 dimension using PCA / autoencoder before mapping to grayscale or RGB pixels. For areas with sensor coverage, the nearest neighbor interpolation method can be used to assign the physical quantities collected by a single sensor to pixels centered on the sensor location. Pixel region; For pixel filling rules without sensor coverage, Kriging interpolation can be used to fit and generate pixel values ​​for the uncovered area based on the physical values ​​and spatial positions of surrounding sensors, ensuring the spatial continuity of stress distribution.

[0022] Extract the temporal features of multidimensional monitoring data, compress the temporal features into a single-channel feature map through convolution, and then fuse it with the sensor pixel map through spatial broadcasting or coordinate weighting to generate a spatiotemporal feature map;

[0023] The sensor pixel image and spatiotemporal features are input into the stripe image prediction model to generate the first stripe image.

[0024] The multidimensional data includes auxiliary information such as temperature, pressure, and ocean waves. By integrating this data, the generated stripe image can more comprehensively reflect the internal physical state of an object, avoiding the loss of key information due to the limitations of a single data source.

[0025] At the same time, since the two images need to be fused, the first stripe image needs to be spatiotemporally aligned with the image to be fused.

[0026] According to the above technical solution, the steps for constructing the stripe image prediction model are as follows:

[0027] Multidimensional monitoring data is generated through finite element simulation, and then corresponding photoelastic fringe images are generated using optical simulation software. Since offshore wind power data is difficult to obtain, finite element simulation data can be used; however, real data can also be used if available. Different finite element models can be constructed based on different offshore wind turbine foundation structures. The construction steps of the finite element model are existing technologies and will not be described in detail here. For example, in the finite element simulation, the bottom of the offshore wind turbine foundation equipment uses fixed constraints (simulating the connection between the pile foundation and the seabed) to restrict the translational degrees of freedom in the X, Y, and Z directions and the rotational degrees of freedom around the three axes, simulating the fixed connection effect between the foundation and the seabed; the area where the equipment contacts the seawater uses hydrodynamic pressure constraints, and the seawater density is taken as... The hydrodynamic pressure is calculated according to the Morrison equation and applied to the foundation surface; the connection between the foundation and the tower, and between the tower and the engine room, is bonded to simulate the rigid connection effect of flange bolts, with no relative slippage or deformation; the main body of the foundation is made of Q355 marine engineering steel with an elastic modulus of [missing information]. Poisson's ratio 0.3, density Allowable stress 230MPa; the concrete foundation pad is made of C40 marine concrete with an elastic modulus of 230MPa. Poisson's ratio is 0.2.

[0028] Load conditions: Covering seven common offshore load conditions (static loads: equipment self-weight, foundation self-weight; dynamic loads: 5-12 level wind loads, 2-5m wave height loads, tidal current loads, equipment operation vibration loads), while also superimposed random fatigue loads (simulating stress accumulation under long-term offshore conditions). For example:

[0029] The seismic load was set according to the design intensity of 7 degrees for offshore wind power projects, with a horizontal seismic acceleration of 0.1g and a vertical seismic acceleration of 50% of the horizontal acceleration, and was applied using the spectral analysis method.

[0030] The foundation, tower, nacelle, and blades are subjected to a self-weight load (total self-weight 2000kN), and the underwater section is simultaneously subjected to seawater buoyancy load, etc.

[0031] Simulated sensor pixel maps and spatiotemporal feature maps are obtained based on simulated multidimensional monitoring data;

[0032] Simulated sensor pixel images, spatiotemporal feature maps, and photoelastic fringe images are used as training samples, and a generative adversarial network is used to train a fringe image prediction model.

[0033] The stripe image prediction model includes a generator that converts an input image into a target image;

[0034] A registration network that adjusts the image output by the generator to align it spatially with the target image;

[0035] The transformation module transforms the image output by the generator based on the deformation field predicted by the registration network to obtain a distorted image, thereby achieving spatial alignment of the image.

[0036] The discriminator is used to determine whether the generated image is close enough to the real target image, and can also evaluate the spatial alignment between the generated image and the target image.

[0037] The sensor pixel map and spatiotemporal feature map are consistent with the photoelastic stripe image to be fused in the spatial dimension, and the stripe image prediction model registration network and transformation module are used to ensure that the generated first stripe image is also consistent in the spatial dimension.

[0038] According to the above technical solution, the fusion module performs the following steps:

[0039] The first stripe image and the photoelastic stripe image at the same time scale are input into the multi-scale feature fusion network. The feature extraction layer of the multi-scale feature fusion network extracts multi-scale features using convolutional layers. Then, the weights of the first stripe image and the photoelastic stripe image are dynamically allocated through spatial attention mechanism and channel attention mechanism, and the multi-scale features are fused according to the weights. Finally, the fused multi-scale features are reconstructed into a fused photoelastic stripe image using deconvolution.

[0040] According to the above technical solution, the encoder of the stress prediction model includes 5 encoding stages. Each encoding stage extracts features through convolution operation and downsamples through convolution with stride N, while doubling the number of channels to compress the input RGB stripe image. The value range of N can be limited to N=2 to 5 (positive integer) to balance the completeness of feature extraction and computational efficiency.

[0041] The decoder contains five decoding stages, which correspond one-to-one with the five encoding stages of the encoder. Each decoding stage expands the feature map size by upsampling and fuses the features of the corresponding stage of the encoder by skip connections, and finally outputs a single-channel stress distribution map.

[0042] A single photoelastic fringe image or limited physical parameters may lead to errors due to the limited nature of the data. By fusing the first fringe image with the photoelastic fringe image, a more comprehensive reflection of the fine fringe patterns of local stress concentration and the distribution of the global stress field can be obtained.

[0043] The abnormal features in the stress distribution map include stress concentration, over-threshold regions, and abrupt change trends;

[0044] An alert is issued if the stress value in a certain area of ​​the distribution map exceeds the safety threshold.

[0045] A warning is issued if the difference between the average stress in the stress concentration area and the average stress in the surrounding area is greater than or equal to the safety difference A.

[0046] By comparing the stress distribution map with historical stress distribution maps, an early warning will be issued if there is a continuous increase in local stress, the gradual appearance of high stress points in areas that were originally low stress, or a sudden asymmetry in stress distribution.

[0047] According to the above technical solution, a storage medium is provided for storing computer-executable instructions, which, when executed, implement the aforementioned deep learning-based stress monitoring and early warning system for offshore wind power infrastructure.

[0048] According to the above technical solution, a computer program product is provided, the computer program product includes a computer program, which, when executed by a processor, implements the above-mentioned deep learning-based stress monitoring and early warning system for offshore wind power infrastructure.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention complements the information missing of the first fringe image and the photoelastic fringe image at the same time scale by fusing them, thereby obtaining a more comprehensive and accurate fused fringe image. The fused fringe image contains richer stress-related features. At the same time, fusing multi-dimensional data into the photoelastic fringe image can enhance the robustness of the image and reduce the influence of interference factors. The fused image enables the stress prediction model to capture more key clues related to stress distribution at the same time, which can more accurately predict stress distribution, improve the reliability of the results, and better reflect the stress situation as a whole. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0051] Figure 1 This is a schematic diagram of the stress monitoring and early warning system for offshore wind power foundation equipment based on deep learning, as described in this invention.

[0052] Figure 2 This is the execution flowchart of the stress monitoring and early warning system for offshore wind power foundation equipment based on deep learning, as described in this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Offshore wind power equipment is typically constructed from opaque steel and concrete. Therefore, transparent photoelastic sensors are embedded in stress concentration areas such as bolt connections and weld seams near the sea level to prevent visual obstruction. For opaque steel and concrete, the photoelastic sensors can be patch-applied. A deep learning-based stress monitoring and early warning system for offshore wind power foundations is then used for stress monitoring and early warning. The structure of this deep learning-based stress monitoring and early warning system is as follows: Figure 1 As shown, it includes a data acquisition module, a mapping module, a stripe image prediction module, a fusion module, a model building module, a stress module, and an early warning module.

[0055] Specific steps for stress monitoring and early warning using a deep learning-based offshore wind power foundation equipment stress monitoring and early warning system ( Figure 2 )include:

[0056] S1, the data acquisition module collects multi-dimensional monitoring data and photoelastic fringe images of the offshore wind power foundation equipment, and preprocesses the collected data; the multi-dimensional monitoring data includes: strain data, vibration data, temperature data, and wave data. Specifically:

[0057] Photoelastic stripe images were collected by embedding transparent photoelastic sensing elements in stress concentration areas such as bolt connections and weld seams of offshore wind power foundation equipment on the sea surface.

[0058] For opaque steel and concrete, photoelastic elements can be patch-applied to collect photoelastic stripe images. Specifically, local transparent window treatment is performed on the surface of the component, such as using high-transparency optical glass to encapsulate the window, or embedding transparent photoelastic sensing elements near welds. For opaque steel and concrete, photoelastic strip areas can be used, which are sealed to the steel structure body to meet marine corrosion and waterproof requirements. The photoelastic elements are embedded inside the optical glass in the window area, and the camera collects the stress data of the steel structure surface through the optical glass.

[0059] For different structures of offshore wind power equipment, ANSYS finite element simulation software can be used to construct a correlation model between surface stress and internal stress of each structure. The simulation simulates actual load conditions such as wind, waves and currents at sea, obtains the correspondence between surface stress and internal stress of components under different loads, stress transfer coefficient, stress concentration coefficient, and establishes a standardized stress conversion database.

[0060] For the surface stress data of steel structures collected by patch-type photoelastic elements, the data is first compared and calibrated with the measured data of strain sensors to eliminate contact errors between the patch and the surface of the component. The calibrated surface stress data is then used to extrapolate and calculate the actual stress value inside the steel structure in real time based on a standardized stress conversion database, and converted into the corresponding photoelastic fringe image.

[0061] The actual stress values ​​inside the steel structure obtained by simulation can be continuously compared with long-term measured data and fatigue test data of similar equipment, and the stress conversion database can be dynamically corrected by using the gradient descent method.

[0062] The parameters of the photoelastic components installed on different offshore wind turbine structures vary. For example, the core physical parameters for photoelastic simulation near the weld are as follows: the photoelastic component is made of polycarbonate material, and its material parameters are: refractive index n=1.586, photoelastic coefficient... The Poisson's ratio is 0.37, and the elastic modulus is 2.4 GPa. In the simulation, a monochromatic parallel light source (wavelength 532 nm) was used, and a linear polarizer was employed with the polarization direction orthogonal to the analysis direction. The relationship between the order of the photoelastic fringes and the stress is as follows: (N is the stripe order, λ is the wavelength of the light source, and d is the thickness of the photoelastic element, which is 5 mm).

[0063] Meanwhile, for multidimensional monitoring data, strain data can be collected using BX120-3AA foil strain gauges and JM3812 marine engineering strain acquisition equipment; vibration data can be collected using ST-3003 triaxial temperature and vibration integrated sensor; temperature data can be collected using marine engineering armored PT1000 platinum resistance sensor; and wave data can be collected using CSB-1000 ultrasonic wave height meter and other equipment.

[0064] For multidimensional monitoring data, outliers in each dimension were removed using the 3σ criterion and replaced with the mean of the five adjacent valid data points. Missing values ​​were then imputed using linear interpolation, and finally standardized using an improved Z-Score standardization algorithm. The processing formula is as follows: In the formula, Represented as standardized values, Represented as the original value, This represents the median of the data. This is represented as the interquartile range. Simultaneously, a sliding window deduplication filter (window length set to 5s, step size 1s) is applied to the collected time-series data to further reduce data drift interference. Furthermore, using the sampling timestamp of one dimension as a benchmark, time synchronization calibration is performed on the data of other dimensions to eliminate pseudo-drift caused by minute differences in sensor sampling, ensuring the spatiotemporal consistency of multidimensional data.

[0065] S2. Utilize the mapping module to establish a spatial coordinate mapping between the pixels of the photoelastic stripe image and the sensor mounting points. Specifically:

[0066] A three-dimensional rectangular coordinate system was constructed based on the three-dimensional model of the offshore wind power infrastructure equipment. The position coordinates of the stress detection device installation point were determined. The mapping relationship of the three-dimensional rectangular coordinate system to the photoelastic fringe image was calculated using a pinhole camera model.

[0067] ;

[0068] In the formula, This represents the sensor's three-dimensional coordinates in the world coordinate system. This represents the pixel coordinates on the two-dimensional image plane corresponding to the sensor's three-dimensional coordinates in the world coordinate system after being projected by the camera. This represents the sensor's three-dimensional coordinates in the world coordinate system, and K represents the camera intrinsic parameter matrix. Represents the extrinsic parameter matrix. Represents the rotation matrix. Represents the translation vector. This represents the depth of the camera coordinate system.

[0069] When calculating the mapping relationship between the three-dimensional Cartesian coordinate system and the photoelastic fringe image using a pinhole camera model, lens distortion correction is required. Lens distortion correction includes radial distortion and tangential distortion, and the correction formula is as follows: ;

[0070] in, , , The radial distortion coefficient is... , The tangential distortion coefficients are obtained by fitting actual measurement data from the calibration board, thus eliminating the influence of lens distortion on pixel coordinate mapping.

[0071] Simultaneously, multiple static reference markers with known 3D coordinates can be arranged on the photoelastic element. By acquiring the image coordinates of these reference points in real time, the real-time extrinsic parameter matrix of the camera can be deduced using the PnP algorithm. The updated extrinsic parameter matrix is ​​then used to correct all projection calculations, compensating for camera pose drift caused by vibration and temperature drift. Alternatively, extrinsic parameters of multiple consecutive frames can be smoothed and filtered to improve the stability of pose estimation. Another approach is to establish a mathematical model of how camera intrinsic parameters and distortion parameters change with temperature, and correct these parameters in real time using temperature sensor data. Alternatively, high-speed shutter and global shutter cameras can be used at the image acquisition end to reduce motion blur; and at the data processing end, noise reduction and edge enhancement can be performed on the stripe image to improve the robustness of feature point detection.

[0072] S3. Using the stripe image prediction module, the multidimensional monitoring data is used to generate the first stripe image through the stripe image prediction model, specifically including:

[0073] Based on the projection relationship from the three-dimensional rectangular coordinate system to the pixels of the photoelastic fringe image, the position coordinates of the stress detection device installation point and the data collected by other sensors are determined in the corresponding pixels of the photoelastic fringe image. Specifically:

[0074] After standardizing the time-synchronized and pre-processed strain data, vibration data, temperature data, and wave data (a total of 4 dimensions), the data is input into the PCA / autoencoder model to reduce the dimensionality of the 4-dimensional multi-dimensional monitoring data to 1 / 3 of the target dimension (i.e., 2 dimensions, since 1 / 3 of the 4 dimensions takes 2 effective feature dimensions), ensuring that the data after dimensionality reduction retains more than 95% of the core features of the original data.

[0075] After performing 0-1 normalization on the 2D reduced data, the grayscale pixel value of each sampling point is assigned to the corresponding pixel position according to the projection relationship from the 3D Cartesian coordinate system to the pixels of the photoelastic fringe image.

[0076] For a single sensor data point, the value is assigned to a value centered on the sensor's projection point. Pixel area;

[0077] If the mounting points of multiple sensors, after projection, are mapped to the same pixel or the same [pixel / segment] of the photoelastic fringe image... For local pixel regions, collision resolution can be achieved using a distance-weighted average method, for example, by averaging the distance between each sensor and that pixel / The Euclidean distance to the center of the region is used as a weighting factor; the closer the distance, the larger the weight (weighting value = 1 / Euclidean distance from the sensor to the center pixel, adding a minimum value to prevent the denominator from being 0). The weighted average of the data from all sensors is calculated, and this value is used as the final assignment for the pixel / region. When using the distance-weighted averaging method to resolve multi-sensor mapping conflicts, if the Euclidean distance d between a sensor's mounting point and the center of the pixel / local region after projection is 0 (i.e., the sensor is precisely mapped to the center pixel), the weight of that sensor is preferentially set to 1. The relative weights of the remaining sensors are calculated according to their Euclidean distances. Then, the weights of all sensors are normalized and calibrated. Finally, the weighted average of the pixel assignment is calculated using the normalized weights. If multiple sensor data are mapped to the same pixel / region, and the distance-weighted average is greater than or equal to the saturation threshold of 0.9, the original 2D dimensionality-reduced data from the sensors that were not distance-weighted are scaled proportionally to the specified values. Then, the distance-weighted average method is used again to assign values.

[0078] For sensorless areas, Kriging interpolation can be used to fill grayscale values ​​(for example, using effective sensor data within a 5×5 pixel range around the sensorless area as samples, a Kriging interpolation model is constructed, and the pixel data values ​​of the sensorless area are predicted through spatial correlation analysis to complete the filling). The interpolation is then validated. (Normalized error), if If the interpolation result is valid, then the interpolation result is valid; if In this case, a virtual sensor is added to the sensorless area, with the average data value of the surrounding area as the initial value of the virtual sensor, and it is dynamically corrected during the model training process to ensure that the error of the filling data is within a controllable range.

[0079] The generated sensor pixel image and photoelastic stripe image are subjected to feature fusion and alignment processing. Specifically, the global brightness mean and variance of the sensor pixel image and the photoelastic stripe image are calculated respectively, and then brightness alignment calibration is performed on all pixel values ​​of the sensor pixel image. If it is an RGB pixel image, the above calibration operation is performed on the R, G and B channels respectively to ensure that the brightness and color distribution characteristics of the sensor pixel image and the photoelastic stripe image are consistent, and finally a sensor pixel image of the same size and pixel position matching the photoelastic stripe image is generated.

[0080] Based on the physical quantities and spatial locations of surrounding sensors, pixel values ​​for the uncovered areas are fitted and generated, producing a sensor pixel map of the same size as the stripe image. The stress detection device includes strain gauges and sensors for stress detection.

[0081] The temporal features of the multidimensional monitoring data are extracted and compressed into a single-channel feature map through convolution. Then, the feature map is fused with the sensor pixel map through spatial broadcasting or coordinate weighting to generate a spatiotemporal feature map. Specifically, the multidimensional monitoring data is preprocessed to be dimensionless. Based on the current time, a temporal window of length T is selected to construct a multidimensional temporal input tensor. Those skilled in the art can adjust the temporal window according to the sampling frequency (e.g., when the sampling frequency is 10Hz, T=128 corresponds to a 12.8-second window).

[0082] A convolution operation is applied to the multidimensional temporal input tensor to compress the temporal features into a single-channel feature map; then, the single-channel feature map is fused with the sensor pixel map through spatial broadcasting or coordinate weighting to generate a spatiotemporal feature map.

[0083] Spatial broadcasting distributes the values ​​of a single-channel temporal feature map to each spatial location in the sensor pixel map, thus binding temporal features to spatial locations. Coordinate weighting generates a spatial weight matrix based on the spatial coordinates of the sensor pixel map, weights the single-channel temporal feature map, and then fuses it with the sensor pixel map. Spatial broadcasting and coordinate weighting are existing technologies and will not be described in detail here.

[0084] The sensor pixel image and spatiotemporal features are input into the stripe image prediction model to generate the first stripe image.

[0085] The stripe image prediction model was trained using mini-batch gradient descent (batch size=16) with Adam as the optimizer. The initial learning rate was 0.001, decreasing by 10% every 100 epochs. The total training epochs were 500. An early stopping strategy was employed (training stopped if the validation set loss did not decrease for 20 consecutive epochs). The steps for building the stripe image prediction model are as follows:

[0086] In the finite element simulation, the bottom of the offshore wind power foundation equipment is fixedly constrained (simulating the connection between the pile foundation and the seabed) to restrict the translational degrees of freedom in the X, Y, and Z directions and the rotational degrees of freedom about the three axes, simulating the fixed connection effect between the foundation and the seabed; the area where the equipment contacts the seawater is constrained by hydrodynamic pressure, and the seawater density is taken as... The hydrodynamic pressure is calculated according to the Morrison equation and applied to the foundation surface; the connection between the foundation and the tower, and between the tower and the engine room, is bonded to simulate the rigid connection effect of flange bolts, with no relative slippage or deformation; the main body of the foundation is made of Q355 marine engineering steel with an elastic modulus of [missing information]. Poisson's ratio 0.3, density Allowable stress 230MPa; the concrete foundation pad is made of C40 marine concrete with an elastic modulus of 230MPa. Poisson's ratio is 0.2. Load conditions include: static loads (equipment self-weight, foundation self-weight); dynamic loads (5-12 level wind load, 2-5m wave height load, tidal current load, equipment operation vibration load), and superimposed random fatigue loads.

[0087] Based on the above boundary conditions and load conditions, multidimensional monitoring data are generated by combining finite element simulations, and then corresponding photoelastic fringe images are generated using optical simulation software.

[0088] Simulated sensor pixel maps and spatiotemporal feature maps are obtained based on simulated multidimensional monitoring data;

[0089] Simulated sensor pixel images, spatiotemporal feature maps, and photoelastic fringe images are used as training samples, and a generative adversarial network is used to train a fringe image prediction model.

[0090] The stripe image prediction model includes a generator that transforms the input image into the target image;

[0091] The registration network adjusts the image output by the generator to align it spatially with the target image.

[0092] The transformation module transforms the image output by the generator based on the deformation field predicted by the registration network to obtain a distorted image, thereby achieving spatial alignment of the image.

[0093] The discriminator is used to determine whether the generated image is close enough to the real target image, and can also evaluate the spatial alignment between the generated image and the target image.

[0094] The registration network used to train the stripe image prediction model employs a combination of multi-scale mean square error loss and deformation smoothing loss. In the formula, for The architecture uses L1 loss for the generator and cross-entropy loss for the discriminator, while also introducing perceptual loss, resulting in a total loss. In the formula, Indicates L1 loss, Represents cross-entropy loss, Indicates perceived loss. and This represents the weight hyperparameter. The deformation field constraint uses rigid body transformation constraints, limiting the rotation angle of the deformation field to no more than... The translation amount should not exceed 3 pixels to avoid mapping distortion caused by excessive deformation.

[0095] S4. The first stripe image is fused with the photoelastic stripe image at the same time scale using the multi-scale feature fusion network in the fusion module to obtain the fused photoelastic stripe image. Specifically, the first stripe image and the photoelastic stripe image at the same time scale are input into the multi-scale feature fusion network. After the feature extraction layer of the multi-scale feature fusion network extracts multi-scale features using convolutional layers, the weights of the first stripe image and the photoelastic stripe image are dynamically allocated through spatial attention mechanism and channel attention mechanism, and the multi-scale features are fused according to the weights. Then, the fused multi-scale features are reconstructed into the fused photoelastic stripe image using deconvolution.

[0096] Among these, the sampling frequency of the stripe image can be used as a unified time reference for the same time scale, aligning all dimensional data to the timestamp at that frequency. Specifically, using the sampling timestamp of the photoelastic stripe image as the reference, vibration, temperature, and wave monitoring data with different sampling frequencies are aligned to the same time scale as the stripe image through interpolation resampling or time-series window aggregation, resulting in the first stripe image and the photoelastic stripe image at the same time scale.

[0097] After standardizing and preprocessing the first fringe image and the photoelastic fringe image at the same time scale, the first fringe image is passed through the first, second and third level convolutional blocks in sequence to output the corresponding low, medium and high scale feature maps respectively; similarly, the photoelastic fringe image uses the same convolutional block parameters as the first fringe image (weight sharing or independent initialization is acceptable) to output the corresponding scale feature map; wherein, the convolutional kernel is initialized with He normal initialization, the BN layer momentum is set to 0.9, and the slope of the ReLU activation function is set to 0 (no leakage).

[0098] In the channel dimension, the aligned first fringe image feature map and the photoelastic fringe image feature map at the same scale are stitched together, preserving feature information from all spatial locations. The stitched feature map is then processed using a spatial attention mechanism (…). Feature compression is performed using convolutional kernels to ensure the output spatial size matches the input; the number of convolutional kernels is set to 1 to retain only spatial information. The compressed feature map is then mapped to... using the sigmoid activation function. The interval is used to obtain the spatial weight value of each pixel position (the larger the weight, the more important the feature at that position). The attention mask is applied to the original feature map to strengthen the features of key regions and weaken the features of irrelevant regions.

[0099] At the same scale, global average pooling is performed on the feature maps of the first stripe image and the photoelastic stripe image for spatial attention mechanism (compressing the two-dimensional features of each channel into a single value by compressing the input RGB stripe image), resulting in channel feature vectors. A fully connected layer (FC) is used to perform a dimensionality reduction and then dimensionality increase transformation on the vectors to simulate the dependencies between channels, and the channel weight vectors are output through the Sigmoid activation function.

[0100] For each scale feature, the spatial weight and channel weight are multiplied separately to obtain the weighted feature, and a dynamic fusion coefficient is set. (Output from attention mechanism) , (The coefficients of another image are used to fuse weighted features, and then cross-scale fusion (such as stitching or addition) is performed on the fused features at all scales to obtain the final multi-scale fused features. The fused multi-scale features are low-resolution feature maps, which need to be upsampled through deconvolution to restore them to the original image size, resulting in the final fused photoelastic stripe image. For example, multi-layer deconvolution (transposed convolution) is used to gradually enlarge the feature maps. Batch normalization (BN) and ReLU activation are added during the deconvolution process to avoid gradient vanishing and improve the reconstruction quality; finally, through... Convolution adjusts the number of channels, outputting a fused photoelastic stripe image with the same size as the input image.

[0101] S5. Construct a stress prediction model in the model building module; the stress prediction model includes an encoder and a decoder.

[0102] The encoder consists of 5 encoding stages. Each encoding stage extracts features through convolution operations and downsamples the input RGB striped image by using convolution with a stride of N, while doubling the number of channels.

[0103] The decoder consists of 5 decoding stages, which correspond one-to-one with the 5 encoding stages of the encoder. Each decoding stage expands the feature map size by upsampling and fuses the features of the corresponding stage of the encoder through skip connections, and finally outputs a single-channel stress distribution map.

[0104] The stress prediction model is trained using the stress cloud map (after pixelation) from finite element simulation and the calibrated results of measured stress data as training labels. The encoder of the stress prediction model uses convolution with a stride of 2 as a unified downsampling method in each encoding stage. Simultaneously, deep features related to equipment stress are extracted through convolution operations. After downsampling is completed in each encoding stage, the number of feature map channels is doubled, thereby achieving feature dimensionality expansion and information enrichment.

[0105] The decoder and encoder have five completely symmetrical encoding stages. Each decoding stage achieves upsampling through transposed convolution, and the feature maps of the corresponding stages of the encoder are directly fused into the feature maps of the same stage of the decoder through skip connections, compensating for feature loss during downsampling and ensuring the detailed accuracy of the stress distribution map. Its physical constraint mechanism introduces stress balance constraints and material strength constraints to ensure that the predicted stress distribution satisfies the static equilibrium equation and that the predicted stress value does not exceed the yield strength of the steel of the wind power equipment (345MPa for Q345 steel).

[0106] S6. Input the fused photoelastic fringe image into the stress prediction model of the stress module to obtain the stress distribution map.

[0107] S7. The early warning module identifies abnormal features in the stress distribution map or combines them with safety thresholds to predict potential risks and issue early warnings. Specifically,

[0108] An alert is issued if the stress value in a certain area of ​​the distribution map exceeds the safety threshold.

[0109] A warning is issued if the difference between the average stress in the stress concentration area and the average stress in the surrounding area is greater than or equal to the safety difference A.

[0110] A warning is issued if the stress concentration area is significantly higher than the surrounding area.

[0111] By comparing the stress distribution map with historical stress distribution maps, an early warning will be issued if there is a continuous increase in local stress, the gradual appearance of high stress points in areas that were originally low stress, or a sudden asymmetry in stress distribution.

[0112] For example: when the stress value in a certain area is higher than that in the surrounding area When the average stress value of a pixel area reaches 50% or higher, a regional stress anomaly warning is triggered. This is triggered when the stress change rate of five consecutive sampling points... The stress at 30 consecutive sampling points increases monotonically and the total increase is The stress difference at symmetrical points is determined by Mutation to The number of abnormal pixels increases for 10 consecutive sampling points and the total increment is... At each pixel, trend warnings are triggered for sudden stress change, continuous increase, sudden asymmetry, and gradual appearance of anomalies, respectively.

[0113] To address image acquisition errors caused by harsh conditions such as high vibration, high salt spray, and strong light at sea, multiple error mitigation measures can be adopted: Anti-strong light: The camera is equipped with an automatic light-adjusting filter that automatically adjusts the filter transmittance according to the intensity of light at sea. At the same time, backlighting technology is used to supplement the lighting and avoid overexposure or underexposure of images caused by direct strong light. Image denoising: The acquired photoelastic stripe images are first processed by adaptive median filtering and wavelet denoising algorithm to eliminate image noise caused by vibration and waves, thereby improving image quality.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0115] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based stress monitoring and early warning system for offshore wind power foundation equipment, characterized in that, include: The data acquisition module is used to collect multi-dimensional monitoring data and photoelastic stripe images of offshore wind power infrastructure equipment, and to preprocess the collected data. The multidimensional monitoring data includes: strain data, vibration data, temperature data, and ocean wave data; The mapping module establishes a spatial coordinate mapping between the pixels of the photoelastic fringe image and the sensor installation points based on the 3D model of the offshore wind power infrastructure. Specifically, this includes: constructing a 3D Cartesian coordinate system based on the 3D model of the offshore wind power infrastructure; determining the position coordinates of the stress detection device installation points; and calculating the mapping relationship of the 3D Cartesian coordinate system in the photoelastic fringe image using a pinhole camera model. ; In the formula, This represents the sensor's three-dimensional coordinates in the world coordinate system. K represents the pixel coordinates of the sensor on the two-dimensional image plane, and K represents the camera intrinsic parameter matrix. Represents the extrinsic parameter matrix. Represents the rotation matrix. Represents the translation vector. This represents the depth of the camera coordinate system; The stripe image prediction module, based on the spatial coordinate mapping between the pixels of the photoelastic stripe image and the sensor installation points, converts the multidimensional monitoring data into a form spatially aligned with the photoelastic stripe image, and then generates a first stripe image using a stripe image prediction model; the stripe image prediction module performs the following steps: Based on the projection relationship from the three-dimensional rectangular coordinate system to the pixels of the photoelastic fringe image, the position coordinates of the stress detection device installation point in the corresponding pixels of the photoelastic fringe image are determined. The physical quantities of the stress detection device in the multidimensional monitoring data are assigned to the corresponding pixels to generate a sensor pixel map of the same size as the fringe image. Extract the temporal features of multidimensional monitoring data, compress the temporal features into a single-channel feature map through convolution, and then fuse it with the sensor pixel map through spatial broadcasting or coordinate weighting to generate a spatiotemporal feature map; The sensor pixel image and spatiotemporal features are input into the stripe image prediction model to generate the first stripe image; The fusion module uses a multi-scale feature fusion network to fuse the first fringe image with the photoelastic fringe image at the same time scale to obtain a fused photoelastic fringe image. The model building module constructs a stress prediction model; the stress prediction model includes an encoder and a decoder. The stress module inputs the fused photoelastic fringe image into the stress prediction model to obtain a stress distribution map; The early warning module identifies abnormal features in the stress distribution map and, in conjunction with safety thresholds, anticipates potential risks and issues warnings in advance.

2. The deep learning-based stress monitoring and early warning system for offshore wind power foundation equipment according to claim 1, characterized in that: The steps for constructing the stripe image prediction model are as follows: Multidimensional monitoring data is generated through finite element simulation, and then corresponding photoelastic fringe images are generated using optical simulation software. Simulated sensor pixel maps and spatiotemporal feature maps are obtained based on simulated multidimensional monitoring data; Simulated sensor pixel images, spatiotemporal feature maps, and photoelastic fringe images are used as training samples, and a generative adversarial network is used to train a fringe image prediction model.

3. The deep learning-based stress monitoring and early warning system for offshore wind power foundation equipment according to claim 1, characterized in that: The steps performed by the fusion module include: The first stripe image and the photoelastic stripe image at the same time scale are input into the multi-scale feature fusion network. The feature extraction layer of the multi-scale feature fusion network extracts multi-scale features using convolutional layers. Then, the weights of the first stripe image and the photoelastic stripe image are dynamically allocated through spatial attention mechanism and channel attention mechanism, and the multi-scale features are fused according to the weights. Finally, the fused multi-scale features are reconstructed into a fused photoelastic stripe image using deconvolution.

4. The deep learning-based stress monitoring and early warning system for offshore wind power foundation equipment according to claim 1, characterized in that: The encoder of the stress prediction model contains 5 encoding stages. Each encoding stage extracts features through convolution operations and downsamples through convolution with a stride of N, while doubling the number of channels to compress the input RGB stripe image. The decoder contains five decoding stages, which correspond one-to-one with the five encoding stages of the encoder. Each decoding stage expands the feature map size by upsampling and fuses the features of the corresponding stage of the encoder by skip connections, and finally outputs a single-channel stress distribution map.

5. A storage medium for storing computer-executable instructions, characterized in that: When the computer-executable instructions are executed, they implement the deep learning-based stress monitoring and early warning system for offshore wind power infrastructure equipment as described in any one of claims 1-4.

6. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the deep learning-based stress monitoring and early warning system for offshore wind power infrastructure as described in any one of claims 1-4.

Images