An offshore wind turbine gearbox fault diagnosis method based on image recognition

Abstract

The application discloses a kind of offshore wind turbine gear box fault diagnosis methods based on image recognition, it is related to gear box fault diagnosis technical field, the reliability of offshore wind gear box crack diagnosis is significantly improved by the imaging optimization of the coupling of multiple physical fields and the innovation of deep learning architecture;The synergy of polarization filtering and dynamic frequency domain filtering effectively suppresses the optical artifact interference of salt crystallization layer;Double generator adversarial network realizes artifact feature decoupling, avoids crack information loss;Frequency domain branch captures crack spectral energy distribution, and spatial branch combines stress distribution priori to strengthen key area detection, solve the cross-scale identification problem of micro-point corrosion and macroscopic broken tooth;Adaptive frequency band selection and deformable convolution design enable the algorithm to cope with different salt mist concentration, complex working conditions of crystalline morphology, reduce the dependence on hardware modification.

Description

Technical Field

[0001] The present invention relates to the technical field of gearbox fault diagnosis, and in particular to an offshore wind turbine gearbox fault diagnosis method based on image recognition. Background Art

[0002] Offshore wind turbine gearboxes are exposed to high salt fog and high humidity in marine environments for long periods of time. Salt fog particles continuously deposit on the gear surfaces and form a crystal layer, which affects visual fault diagnosis. In recent years, with breakthroughs in deep learning technology, image recognition solutions based on high-resolution industrial cameras and convolutional neural networks have gradually become mainstream. They detect cracks by capturing subtle changes in the tooth surface texture. However, the complex artifacts formed by salt fog crystallization on the tooth surface, such as dendritic or layered structures, are highly similar to the morphological characteristics of real cracks, making it difficult for existing algorithms to effectively distinguish between the two during the feature extraction stage. Especially after the typhoon season, the salt crystal layer can reach a thickness of millimeters, further obscuring the true morphology of the tooth surface and causing the loss of fault feature information.

[0003] To cope with salt fog interference, some technical routes focus on multimodal data fusion and physical enhanced learning. For example, multispectral imaging technology is used to analyze the reflectivity differences between salt crystals and metal substrates in different bands. Another team has introduced polarized light imaging to suppress mirror reflection noise by using the modulation characteristics of the crystal layer on the polarization state. However, in such solutions, multispectral imaging requires dedicated optical hardware and is difficult to deploy in the narrow gearbox cabin. Polarization imaging is sensitive to the thickness of the crystal layer, and the risk of failure increases sharply when the crystal morphology is irregular. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] The present invention provides an offshore wind turbine gearbox fault diagnosis method based on image recognition to solve the problem that the crystallization artifact interference of the gear surface caused by the offshore salt spray environment is limited in the existing image diagnosis methods in terms of artifact separation and cross-scale feature extraction.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] An embodiment of the present invention provides a method for diagnosing faults in an offshore wind turbine gearbox based on image recognition, which includes:

[0008] Step S1, collecting gear meshing surface image data using a multi-band combined light source and an industrial camera, wherein the multi-band combined light source includes a visible light band and a short-wave infrared band;

[0009] Step S2, preprocessing the image data, including dynamic interference removal based on polarization filtering and noise suppression based on frequency domain filtering;

[0010] Step S3, input the preprocessed image into a crystallization artifact separation module, which contains a generative adversarial network and a deformable convolution layer, for extracting crack features from the salt mist crystallization coverage area;

[0011] Step S4, multi-scale fusion of the separated crack features through a spatio-temporal feature extraction network, which contains a frequency domain analysis branch and a spatial domain attention branch;

[0012] Step S5, input the fused features into a classification model to output the fault type and severity;

[0013] Step S6, generate maintenance decision instructions according to the classification results, and update the gearbox health state database synchronously.

[0014] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the visible light band is 450-680nm, the short-wave infrared band is 900-1700nm, the light source adopts an alternating pulse trigger mode, and the irradiation time ratio of visible light to short-wave infrared is 1:3.

[0015] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the preprocessing in step S2 specifically includes:

[0016] The linear polarization reflection image of the gear surface is obtained through the polarization filtering device, and the specular reflection interference of the salt crystallization layer is suppressed;

[0017] The linear polarization reflection image is subjected to frequency domain filtering, and a band-pass filter with adaptive bandwidth is used to retain the frequency band related to crack features;

[0018] In step S2, the center frequency of the band-pass filter is dynamically adjusted according to the thickness of the salt crystallization layer, the thickness detection is realized by analyzing the absorption rate change of the short-wave infrared band image, and the adjustment rule is that the center frequency of the band-pass filter shifts to high frequency direction with the increase of the thickness of the salt crystallization layer.

[0019] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the generative adversarial network of the crystallization artifact separation module adopts a double generator architecture:

[0020] The first generator is used to generate a synthetic image with salt mist crystallization artifacts, and the second generator is used to generate a gear surface image without artifacts;

[0021] The deformable convolution layer dynamically adjusts the convolution kernel shape in the decoding stage to match the morphological changes of the crystallization artifacts;

[0022] In the double generator architecture, the input of the first generator is the tooth surface image without crystallization artifacts and a random noise vector, and the output is a synthesized image superimposed with crystallization artifacts; the input of the second generator is a real image with crystallization artifacts, and the output is a tooth surface image after artifact separation; the two generators share a feature extraction layer, and realize adversarial training through a gradient inversion layer.

[0023] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, in step S3, the pretreated image is input into a crystallization artifact separation module composed of an adversarial generation network and a deformable convolution layer to extract crack features from the salt mist crystallization covered area, which is divided into the following two parts:

[0024] a) Double generator image mapping:

[0025] The first generator fuses the non-artifact tooth surface image with random noise to generate a synthesized image with salt mist crystallization artifacts:

[0026]

[0027] wherein, represents the synthesized image with artifacts, G1(·,·;θ G1 represents the first generator network mapping, x represents the input non-artifact real tooth surface image, represents a random noise vector subject to a standard multivariate normal distribution, I is a k×k identity matrix consistent with the dimension of the noise vector, and θ G1 represents the network weight of the first generator, Enc(·) represents the shared encoder mapping, represents a matrix for projecting a k-dimensional noise z into a d-dimensional encoded feature space, k represents the dimension of the random noise vector, d represents the number of encoded feature channels, R represents the real number field, and Dec1(·) represents the decoder mapping of the first generator;

[0028] wherein,

[0029]

[0030] wherein, w r,s represents the weight of the s-th noise component mapped to the r-th encoded feature dimension;

[0031] The second generator removes artifacts and restores the real texture of the tooth surface based on the shared encoder features:

[0032]

[0033] wherein, represents the tooth surface image after removing artifacts, G2(·;θ G2) represents the second generator network mapping, y represents the input real image with artifacts, θ G2 represents the network weight of the second generator, Dec2(·) represents the decoder mapping of the second generator;

[0034] b) deformable convolution adaptive offset:

[0035] In the decoding stage, a deformable convolution layer is introduced to adaptively learn the sampling offset according to the intermediate feature to match the shape of the crystal artifact:

[0036] Δp(p n )=H off (F mid (p0+p n );θ off ),

[0037]

[0038] where Δp(p n ) represents the learned offset of the relative position p n , H off (·; θ off ) represents the offset generation network mapping, F mid represents the intermediate feature map of the decoder, p0 represents the center position of the current output pixel, represents a set of relative positions in the standard convolution sampling grid, θ off represents the weight of the offset generation network, F out (p0) represents the value of the output feature map at position p0, w(p n ) represents the weight coefficient of the convolution kernel at the relative position p n , and F in represents the input feature map.

[0039] In the formula,

[0040] H off (F; θ off )=σ(W off F+b off ),

[0041] where F here is the intermediate feature vector extracted at F mid (p0+p n ), represents the fully connected weight matrix of the offset network, represents the bias vector of the offset network, σ represents the activation function, and ReLU and LeakyReLU are selected, C represents the number of feature channels, N represents the number of sampling points, R is the standard sampling grid, and |R|=N.

[0042] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the spatial-temporal feature extraction network comprises:

[0043] The curvelet transform is performed on the input image to extract multi-directional and multi-scale frequency domain features.

[0044] The frequency domain components corresponding to the crack features are dynamically enhanced through a learnable frequency band selection matrix.

[0045] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the spatial-temporal feature extraction network comprises:

[0046] The attention weight generation module based on the gear stress distribution prior dynamically adjusts the feature map weight according to the position of the gear meshing point;

[0047] The cascaded Transformer encoder is used to capture the long-range spatial dependency of the crack features;

[0048] In the attention weight generation module, the stress distribution prior obtains the maximum principal stress distribution map during the gear meshing process through finite element simulation and encodes it as a spatial weight template; the module performs Hadamard product operation on the template and the real-time image feature map to generate attention weights.

[0049] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, in the frequency domain analysis branch of step S4, the frequency domain coefficients after Fourier transform are weighted through a learnable frequency band selection matrix to dynamically highlight the frequency domain components corresponding to the cracks, comprising:

[0050] The two-dimensional discrete Fourier transform is performed on the preprocessed artifact-free image X to obtain frequency domain coefficients:

[0051]

[0052] Where X(p,q) represents the pixel value at spatial coordinates (p,q), H and W represent the height and width of the image respectively, u=0,…,H-1, v=0,…,W-1 represent the frequency domain coordinates, j represents the imaginary unit, and F(u,v) represents the frequency domain coefficient.

[0053] The frequency domain plane is divided into N b non-overlapping frequency band sets And the average amplitude of each frequency band is calculated:

[0054]

[0055] Where Bl B = {B1, B2, …, B|B|} denotes the coordinate set of the lth frequency band, |B l | denotes the set size, |F(u, v)| denotes the amplitude of the frequency domain coefficient, f l denotes the feature quantity of the lth frequency band;

[0056] The frequency band feature vector is input, and the learnable matrix W m and the bias b m generate the frequency band weight vector:

[0057] w = softmax(W m f + b m ),

[0058] wherein, denotes the learnable frequency band selection matrix, denotes the learnable bias vector, N b denotes the preset number of frequency band divisions, and softmax(·) denotes a component normalization function, denotes the weight of each frequency band;

[0059] The elements of the matrix W m are denoted as:

[0060]

[0061] wherein, w ξ,ζ denotes the mapping coefficient of the zth frequency band feature to the ξth output weight;

[0062] For each frequency band, the weight w l is applied to the corresponding frequency domain coefficient to enhance the crack signal:

[0063]

[0064] wherein, F en (u, v) denotes the weighted frequency domain coefficient, and w l denotes the weighting factor of the lth frequency band;

[0065] The weighted frequency domain coefficient is restored to a spatial domain image for subsequent spatial branch processing:

[0066]

[0067] wherein, X en (p, q) denotes the enhanced spatial domain pixel value.

[0068] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the classification model fuses image features and vibration time sequence features, and the specific implementation manner is:

[0069] A double-channel input structure is constructed, a first channel receives image space-time features, and a second channel receives vibration signal spectrum features collected synchronously.

[0070] Cross-attention mechanism is used to realize cross-modal feature interaction, and a joint representation vector is generated.

[0071] As a preferred scheme of the offshore wind turbine gearbox fault diagnosis method based on image recognition, the generation of the maintenance decision instruction comprises:

[0072] According to the fault type, a preset maintenance priority rule is matched;

[0073] The maintenance time window is dynamically adjusted in combination with offshore weather data and the operation state of the unit;

[0074] A three-dimensional visualization report containing a fault position mark is output.

[0075] The present application has the following advantages: the present application significantly improves the reliability of offshore wind turbine gearbox crack diagnosis through multi-physical field coupling imaging optimization and deep learning architecture innovation; the cooperation of polarization filtering and dynamic frequency domain filtering effectively suppresses the optical artifact interference of salt crystal layer; the dual-generator adversarial network realizes artifact feature decoupling, avoiding crack information loss; the frequency domain branch captures crack spectrum energy distribution, and the spatial domain branch combines stress distribution prior to strengthen key area detection, solving the cross-scale identification problem of micro-pitting and macro-tooth breakage; adaptive frequency band selection and deformable convolution design enable the algorithm to cope with complex working conditions of different salt mist concentrations and crystal morphologies, reducing the dependence on hardware modification. BRIEF DESCRIPTION OF DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0077] Figure 1 The flowchart of the offshore wind turbine gearbox fault diagnosis method based on image recognition in embodiment 1 is shown. DETAILED DESCRIPTION

[0078] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail in conjunction with the drawings of the specification.

[0079] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be appreciated that the present application can be practiced in a variety of ways beyond the specific details set forth herein, having regard to the content of the following description, and thus the present application should not be limited to the details of the description.

[0080] Secondly, the "one embodiment" or "embodiment" referred to herein means a specific feature, structure, or characteristic that can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor does it mean an embodiment that is separate or alternative to other embodiments.

[0081] Embodiment 1, Reference Figure 1 The embodiment provides an offshore wind turbine gearbox fault diagnosis method based on image recognition, comprising the following steps:

[0082] Step S1, acquiring gear mesh surface image data through multi-band combined light source and industrial camera, the multi-band combined light source comprising visible light band and short-wave infrared band;

[0083] The visible light band is 450-680nm, and the short-wave infrared band is 900-1700nm. The light source adopts an alternating pulse trigger mode, and the irradiation time ratio of visible light to short-wave infrared is 1:3. This ratio can balance the requirements of visible light detail capture and infrared penetration, and the incidence angle of the short-wave infrared light source is controlled within the range of 30°±5°, which can suppress mirror reflection. Here, 450-680nm (visible light) and 900-1700nm (short-wave infrared) belong to industry general spectral segmentation.

[0084] Step S2, pre-processing the image data, including dynamic interference removal based on polarization filtering and noise suppression based on frequency domain filtering;

[0085] The pre-processing in step S2 specifically includes:

[0086] Obtain the linearly polarized reflection image of the gear surface through the polarization filtering device to suppress the mirror reflection interference of the salt crystal layer;

[0087] Perform frequency domain filtering on the linearly polarized reflection image, and use a bandpass filter with adaptive bandwidth to retain the frequency band related to crack features;

[0088] In step S2, the center frequency of the bandpass filter is dynamically adjusted according to the thickness of the salt crystal layer. The thickness detection is achieved by analyzing the absorption rate change of the short-wave infrared band image. The adjustment rule is that the center frequency of the bandpass filter shifts to high frequency as the thickness of the salt crystal layer increases;

[0089] Step S3, the pre-processed image is input into a crystallization artifact separation module, which includes a generative adversarial network and a deformable convolution layer, for extracting crack features from the salt-mist crystallization covered area;

[0090] The generative adversarial network of the crystallization artifact separation module adopts a double-generator architecture:

[0091] The first generator is used to generate a synthetic image with salt-mist crystallization artifacts, and the second generator is used to generate a tooth surface image without artifacts;

[0092] The deformable convolution layer dynamically adjusts the shape of the convolution kernel in the decoding stage to match the morphological changes of the crystallization artifacts;

[0093] In the double-generator architecture, the input of the first generator is a tooth surface image without crystallization artifacts and a random noise vector, and the output is a synthetic image superimposed with crystallization artifacts; the input of the second generator is a real image with crystallization artifacts, and the output is a tooth surface image after artifact separation; the two generators share a feature extraction layer and implement adversarial training through a gradient reversal layer;

[0094] In step S3, the pre-processed image is input into a crystallization artifact separation module, which is composed of a generative adversarial network and a deformable convolution layer, for extracting crack features from the salt-mist crystallization covered area, which is divided into the following two parts:

[0095] a) Double-generator image mapping:

[0096] The first generator fuses the artifact-free tooth surface image with random noise to generate a synthetic image with salt-mist crystallization artifacts:

[0097]

[0098] wherein, represents the synthetic image with artifacts, G1(·,·;θ G1 represents the first generator network mapping, x represents the input artifact-free real tooth surface image, represents a random noise vector subject to a standard multivariate normal distribution, I is a k×k identity matrix consistent with the dimension of the noise vector, θ G1 represents the network weight of the first generator, Enc(·) represents the shared encoder mapping, represents the matrix that projects the k-dimensional noise z into the d-dimensional encoded feature space, k represents the dimension of the random noise vector, d represents the number of encoded feature channels, R represents the real number field, and Dec1(·) represents the decoder mapping of the first generator;

[0099] wherein,

[0100]

[0101] where w r,s denotes the weight of mapping the s-th noise component to the r-th encoding feature dimension;

[0102] The second generator removes artifacts on the basis of shared encoder features and restores the real texture of the tooth surface:

[0103]

[0104] where, denotes the tooth surface image after removing artifacts, G2(·; θ G2 ) denotes the second generator network mapping, y denotes the input real artifact image, θ G2 denotes the network weight of the second generator, and Dec2(·) denotes the decoder mapping of the second generator.

[0105] b) Deformable convolution adaptive offset:

[0106] In the decoding stage, a deformable convolution layer is introduced to adaptively learn the sampling offset according to the intermediate feature to match the morphology of the crystallization artifact:

[0107] Δp(p n ) = H off (F mid (p0 + p n ); θ off ),

[0108]

[0109] where Δp(p n ) denotes the learned offset amount of the relative position p n , H off (·; θ off ) denotes the offset generation network mapping, F mid denotes the intermediate feature map of the decoder, p0 denotes the center position of the current output pixel, denotes the set of relative positions in the standard convolution sampling grid, θ off denotes the weight of the offset generation network, F out (p0) denotes the value of the output feature map at position p0, w(p n ) denotes the weight coefficient of the convolution kernel at the relative position p n , and F in denotes the input feature map.

[0110] where,

[0111] H off (F; θ off ) = σ(W off F + b off ),

[0112] where F is F mid (p0+p n ) extracted at the intermediate feature vector, denotes the full connection weight matrix of the offset network, denotes the bias vector of the offset network, σ denotes the activation function, ReLU and LeakyReLU are selected, C denotes the number of feature channels, N denotes the number of sampling points, R is the standard sampling grid, |R| = N;

[0113] Specifically, this step realizes accurate generation and removal of salt spray crystallization artifacts through the organic combination of the generative adversarial network and the deformable convolution layer. The double generator structure is in the same encoder feature space, and through noise projection and decoding mapping, it not only enhances the diversity of the artifact samples, but also realistically simulates the visual effect of the crystalline layer under different thicknesses and morphologies, thereby providing a rich positive and negative sample pair for adversarial training. The artifact removal generator focuses on restoring the tooth surface texture to ensure the detail restoration after artifact removal. The deformable convolution layer dynamically generates sampling offsets through the offset network and adaptively adjusts the convolution kernel position in the decoding stage, so that the network can flexibly cope with the edge and morphology changes of the artifacts, greatly improving the response ability to the characteristics of the micro cracks. The overall module not only retains the key information of the cracks, but also effectively suppresses the crystalline interference, laying a high-quality foundation for subsequent spatiotemporal feature extraction and significantly improving the accuracy and robustness of fault diagnosis.

[0114] Step S4, multi-scale fusion of the separated crack features through a spatiotemporal feature extraction network, the spatiotemporal feature extraction network comprising a frequency domain analysis branch and a spatial domain attention branch;

[0115] The frequency domain analysis branch of the spatiotemporal feature extraction network performs the following operations:

[0116] Performing curvelet transform on the input image to extract multi-directional and multi-scale frequency domain features;

[0117] Dynamically enhancing the frequency domain components corresponding to the crack features through a learnable frequency band selection matrix;

[0118] The spatial domain attention branch of the spatiotemporal feature extraction network comprises:

[0119] An attention weight generation module based on gear stress distribution prior, which dynamically adjusts the feature map weight according to the position of the gear meshing point;

[0120] A cascaded Transformer encoder for capturing long-range spatial dependencies of the crack features;

[0121] In the attention weight generation module, the stress distribution prior acquires the maximum principal stress distribution diagram in the gear meshing process through finite element simulation, and encodes it into a spatial weight template; the module performs Hadamard product operation on the template and the real-time image feature map to generate attention weights;

[0122] In the frequency domain analysis branch of step S4, the frequency domain coefficients after Fourier transform are weighted by a learnable frequency band selection matrix to dynamically highlight the frequency domain components corresponding to the crack, including:

[0123] A two-dimensional discrete Fourier transform is performed on the preprocessed artifact-free image X to obtain frequency domain coefficients:

[0124]

[0125] Where X(p,q) represents the pixel value at spatial coordinates (p,q), H and W represent the height and width of the image respectively, u=0,…,H-1, v=0,…,W-1 represent the frequency domain coordinates, j represents the imaginary unit, and F(u,v) represents the frequency domain coefficient.

[0126] The frequency domain plane is divided into N b non-overlapping frequency band sets The average amplitude of each frequency band is calculated:

[0127]

[0128] Where B l represents the coordinate set of the lth frequency band, |B l | represents the size of the set, |F(u,v)| represents the amplitude of the frequency domain coefficient, and f l represents the feature quantity of the lth frequency band.

[0129] The frequency band feature vector is input, and the learnable matrix W m and the bias b m are used to generate the frequency band weight vector:

[0130] w=softmax(W m f+b m ),

[0131] Where, represents the learnable frequency band selection matrix, represents the learnable bias vector, N b represents the preset number of frequency band divisions, and softmax(·) represents the component normalization function, represents the weight of each frequency band.

[0132] The elements of the matrix W m are represented as:

[0133]

[0134] wherein w ξ,ζ denotes the mapping coefficient of the z-th frequency band feature to the ξ-th output weight;

[0135] For each frequency band, the weight w l is applied to the corresponding frequency domain coefficient, reinforcing the crack signal:

[0136]

[0137] wherein F en (u,v) denotes the weighted frequency domain coefficient, w l denotes the weighting factor of the l-th frequency band;

[0138] The weighted frequency domain coefficient is restored to the spatial domain image for subsequent spatial branch processing:

[0139]

[0140] wherein X en (p,q) denotes the enhanced spatial domain pixel value;

[0141] Specifically, the present frequency domain branch dynamically weights different frequency band features through a learnable matrix, which can adaptively highlight the spectral energy distribution corresponding to the crack. The frequency band division and average amplitude extraction steps compress the two-dimensional Fourier coefficients into a lower-dimensional feature vector that is easier to learn. The learnable frequency band selection matrix W m and the bias b m During the training process, the spectral pattern specific to the crack is captured, so that the weight vector w can automatically adjust to different working conditions. The weighting operation of the frequency domain coefficient further enhances the signal-to-noise ratio, highlighting the performance of the crack texture in the spatial domain. Finally, the inverse transform ensures that the image passed to the spatial domain branch not only retains the original details, but also has higher contrast in the crack area;

[0142] Step S5, input the fused features into the classification model to output the fault type and severity;

[0143] The classification model fuses image features and vibration time series features, and the specific implementation manner is as follows:

[0144] A dual-channel input structure is constructed, the first channel receives image spatiotemporal features, and the second channel receives synchronous acquisition of vibration signal spectral features;

[0145] Cross-modal feature interaction is realized through a cross-attention mechanism to generate a joint representation vector;

[0146] Step S6, generating a maintenance decision instruction according to the classification result, and synchronously updating the gearbox health state database;

[0147] The generation of the maintenance decision instruction comprises:

[0148] According to the fault type, a preset maintenance priority rule is matched;

[0149] Combined with the offshore weather data and the unit operation state, a maintenance time window is dynamically adjusted;

[0150] A three-dimensional visualization report containing a fault position mark is output.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for fault diagnosis of offshore wind turbine gearbox based on image recognition, characterized in that: include, Step S1, collecting gear meshing surface image data using a multi-band combined light source and an industrial camera, wherein the multi-band combined light source includes a visible light band and a short-wave infrared band; Step S2, preprocessing the image data, including dynamic interference removal based on polarization filtering and noise suppression based on frequency domain filtering; Step S3, inputting the preprocessed image into a crystallization artifact separation module, which includes a generative adversarial network and a deformable convolutional layer to extract crack features from the salt spray crystallization covered area; Step S4, performing multi-scale fusion on the separated crack features through a spatiotemporal feature extraction network, where the spatiotemporal feature extraction network includes a frequency domain analysis branch and a spatial domain attention branch; Step S5: Input the fused features into the classification model and output the fault type and severity; Step S6: generating a maintenance decision instruction based on the classification result, and synchronously updating the gearbox health status database; The adversarial generative network of the crystallization artifact separation module adopts a dual generator architecture: The first generator is used to generate a synthetic image with salt spray crystallization artifacts, and the second generator is used to generate an artifact-free tooth surface image; The deformable convolution layer dynamically adjusts the shape of the convolution kernel during the decoding stage to match the morphological changes of the crystallization artifacts; In the dual-generator architecture, the first generator takes as input a tooth surface image without crystallization artifacts and a random noise vector, and outputs a composite image with crystallization artifacts superimposed on it. The second generator takes as input a real image with crystallization artifacts, and outputs a tooth surface image with the artifacts separated. The two generators share a feature extraction layer and implement adversarial training through a gradient reversal layer. Step S3 is divided into the following two parts: a) Dual Generator Image Mapping: The first generator fuses the artifact-free tooth surface image with random noise to generate a synthetic image with salt spray crystallization artifacts: in, represents the synthesized artifact image, G1(·,·; θ G1 ) represents the first generator network mapping, x represents the input artifact-free real tooth surface image, represents a random noise vector that obeys a standard multivariate normal distribution, I is a k×k unit matrix, which is consistent with the dimension of the noise vector, θ G1 represents the network weights of the first generator, Enc(·) represents the shared encoder mapping, represents the matrix that projects the k-dimensional noise z onto the d-dimensional encoding feature space, k represents the dimension of the random noise vector, d represents the number of encoder output feature channels, R represents the real number domain, and Dec1(·) represents the decoder mapping of the first generator; Where, Among them, w r,s represents the weight that maps the sth noise component to the rth encoding feature dimension; The second generator removes artifacts based on the shared encoder features and restores the true texture of the tooth surface: in, represents the tooth surface image after removing artifacts, G2(·; θ G2 ) represents the second generator network mapping, y represents the input real artifact image, θ G2 represents the network weight of the second generator, Dec2(·) represents the decoder mapping of the second generator; b) Deformable Convolution Adaptive Offset: A deformable convolutional layer is introduced in the decoding stage to adaptively learn sampling offsets based on intermediate features to match the morphology of crystallization artifacts: Δp(p n )=H off (F mid (p0+p n );θ off ), Among them, Δp(p n ) represents the relative position p n The learning offset, H off (·;θ off ) represents the offset generation network mapping, F mid represents the intermediate feature map of the decoder, p0 represents the center position of the current output pixel, represents the relative position set in the standard convolution sampling grid, θ off represents the weight of the offset generation network, F out (p0) represents the value of the output feature map at position p0, w(p n ) indicates the convolution kernel at relative position p n The weight coefficient, F in Represents the input feature map; Where, H off (F;θ off )=σ(W off F+b off ), Here, F is F mid (p0+p n ) is the intermediate feature vector extracted at represents the fully connected weight matrix of the offset network, Represents the bias vector of the offset network, σ represents the activation function, ReLU and LeakyReLU are selected, C represents the number of feature channels, N represents the number of sampling points, R is the standard sampling grid, |R| = N.

2. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 1, characterized in that: The visible light band is 450-680nm, the short-wave infrared band is 900-1700nm, the light source adopts an alternating pulse triggering mode, and the irradiation time ratio of visible light to short-wave infrared is 1:

3.

3. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 1, characterized in that: The preprocessing in step S2 specifically includes: The linear polarization reflection image of the gear surface is obtained through the polarization filter device to suppress the mirror reflection interference of the salt crystal layer; Performing frequency domain filtering on the linearly polarized reflection image, using a bandpass filter with adaptive bandwidth to retain frequency bands related to crack characteristics; In step S2, the center frequency of the bandpass filter is dynamically adjusted according to the thickness of the salt crystal layer. The thickness detection is achieved by analyzing the change in the absorbance of the short-wave infrared band image. The adjustment rule is: the center frequency of the bandpass filter shifts toward the high frequency direction as the thickness of the salt crystal layer increases.

4. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 1, characterized in that: The frequency domain analysis branch of the spatiotemporal feature extraction network performs the following operations: Perform curvelet transform on the input image to extract multi-directional and multi-scale frequency domain features; The frequency domain components corresponding to the crack features are dynamically enhanced through a learnable frequency band selection matrix.

5. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 4, characterized in that: The spatial attention branch of the spatiotemporal feature extraction network includes: An attention weight generation module based on the gear stress distribution prior dynamically adjusts the feature map weight according to the position of the gear meshing point; Cascaded Transformer encoders to capture long-range spatial dependencies of crack features; In the attention weight generation module, the stress distribution prior obtains the maximum principal stress distribution diagram during the gear meshing process through finite element simulation and encodes it into a spatial weight template; The module performs a Hadamard product operation on the template and the real-time image feature map to generate an attention weight.

6. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 5, characterized in that: In the frequency domain analysis branch of step S4, the frequency domain coefficients after Fourier transformation are weighted by a learnable frequency band selection matrix to dynamically highlight the frequency domain components corresponding to the crack, including: Perform a two-dimensional discrete Fourier transform on the artifact-free image X after preprocessing to obtain the frequency domain coefficients: Where X(p,q) represents the pixel value at the spatial coordinate (p,q), H, W represent the height and width of the image respectively, u=0,…,H-1,v=0,…,W-1 represent the frequency domain coordinates, j represents the imaginary unit, and F(u,v) represents the frequency domain coefficient; Divide the frequency domain plane into N b A set of non-overlapping frequency bands And calculate the average amplitude of each frequency band: in, Indicates the The coordinate set of the frequency bands, represents the size of the set, |F(u,v)| represents the amplitude of the frequency domain coefficient, f l Indicates the The characteristic quantity of each frequency band; Frequency band eigenvector As input, the learnable matrix W m With bias b m Generate a vector of band weights: w=softmax(W m f+b m ), in, represents the learnable band selection matrix, represents the learnable bias vector, N b represents the preset number of frequency band divisions, softmax(·) represents the component normalization function, w= Indicates the weight of each frequency band; Matrix W m The elements of are represented as: Among them, w ξ,ζ Represents the mapping coefficient of the ζ-th frequency band feature to the ξ-th output weight; For each frequency band, the weight Applied to the corresponding frequency domain coefficients to enhance the crack signal: Among them, F en (u,v) represents the weighted frequency domain coefficients, Indicates the The weighting factors for each frequency band; The weighted frequency domain coefficients are restored to spatial domain images for subsequent spatial domain branch processing: Among them, X en (p,q) represents the enhanced spatial domain pixel value.

7. The method for diagnosing offshore wind turbine gearbox faults based on image recognition according to claim 1, characterized in that: The classification model integrates image features and vibration time series features, and is specifically implemented as follows: Construct a dual-channel input structure, where the first channel receives the spatiotemporal features of the image, and the second channel receives the spectral features of the synchronously collected vibration signal; Cross-modal feature interaction is achieved through the cross-attention mechanism to generate a joint representation vector.

8. The offshore wind turbine gearbox fault diagnosis method based on image recognition according to claim 1, characterized in that: The generation of the maintenance decision instruction includes: Match preset maintenance priority rules based on fault type; Dynamically adjust the maintenance time window by combining offshore meteorological data with the unit's operating status; Output a 3D visualization report with fault location markers.

Images