Fan blade fault diagnosis method and device based on predictive coding

Abstract

The invention discloses a fan blade fault diagnosis method and device based on predictive coding, and the method comprises the steps: obtaining the high-dimensional image information of a fan blade, and converting the high-dimensional image information into a low-dimensional image feature vector; processing the low-dimensional image feature vector to obtain prediction representation information, and obtaining an error feature according to the prediction representation information and a preset reference tag feature; under the condition that the convergence condition is not met, a prediction error is determined according to the error characteristics, and the steps of processing the low-dimensional image characteristic vector and obtaining prediction representation information are executed again according to the prediction error until the convergence condition is met; and predicting the fault type of the fan blade according to the final error characteristics obtained when the convergence condition is satisfied.

Description

Technical Field

[0001] The present application relates to the technical field of wind power generation, and in particular to a method and device for diagnosing wind turbine blade faults based on predictive coding. Background Art

[0002] Wind turbine blades are crucial components of wind turbines, and their operating status directly affects the power generation efficiency and safety of the unit. During long-term operation, wind turbine blades are easily affected by external environmental factors such as wind, sand, rain, ice and snow, resulting in various faults such as surface wear, cracks, bubbles, etc. These faults will not only reduce the performance of the wind turbine, but may also cause serious safety hazards.

[0003] In practical applications, fault detection of wind turbine blades mainly relies on manual inspection and traditional monitoring methods. However, as time goes by, these methods have problems such as untimely detection, low efficiency and missed detection. Summary of the invention

[0004] To this end, this application discloses the following technical solutions:

[0005] The first aspect of the present application provides a fan blade fault diagnosis method based on predictive coding, comprising:

[0006] Obtaining high-dimensional image information of a fan blade, and converting the high-dimensional image information into a low-dimensional image feature vector;

[0007] Processing the low-dimensional image feature vector to obtain prediction representation information, and obtaining error features according to the prediction representation information and preset reference label features;

[0008] In the case where the convergence condition is not met, determining a prediction error according to the error feature, and performing the step of processing the low-dimensional image feature vector again according to the prediction error to obtain prediction representation information until the convergence condition is met;

[0009] The fault type of the wind turbine blade is predicted according to the final error characteristics obtained when the convergence condition is met.

[0010] Optionally, the convergence condition includes that the number of iterations is greater than or equal to a preset upper limit N of the number of iterations, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain image condensation information.

[0011] Optionally, converting the high-dimensional image information into a low-dimensional image feature vector includes:

[0012] The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each of the residual blocks includes a two-layer residual convolutional network.

[0013] Optionally, the processing the low-dimensional image feature vector to obtain prediction representation information includes:

[0014] Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector;

[0015] Forward prediction is performed according to the transformed image feature vector to obtain prediction representation information.

[0016] Optionally, performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector includes:

[0017] The low-dimensional image feature vector is sequentially subjected to two-dimensional convolution processing, batch regularization processing, rectification processing, two-dimensional convolution processing and batch regularization processing to obtain a transformed image feature vector.

[0018] Optionally, the prediction error includes a direct prediction error and a high-level prediction error;

[0019] The determining the prediction error according to the error feature comprises:

[0020] Directly learning the error feature according to the two-dimensional convolutional layer to obtain a direct prediction error;

[0021] The error features are processed according to the second predictor and the transformed error module to obtain a high-level prediction error.

[0022] A second aspect of the present application provides a fan blade fault diagnosis device based on predictive coding, comprising:

[0023] An obtaining unit, used for obtaining high-dimensional image information of the fan blades, and converting the high-dimensional image information into a low-dimensional image feature vector;

[0024] A processing unit, used to process the low-dimensional image feature vector to obtain prediction representation information, and obtain error features according to the prediction representation information and preset reference label features;

[0025] a determining unit, configured to determine a prediction error according to the error feature when the convergence condition is not satisfied, and to perform the step of processing the low-dimensional image feature vector again according to the prediction error to obtain prediction representation information until the convergence condition is satisfied;

[0026] The prediction unit is used to predict the fault type of the wind turbine blade according to the final error characteristics obtained when the convergence condition is met.

[0027] Optionally, the convergence condition includes that the number of iterations is greater than or equal to a preset upper limit N of the number of iterations, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain image condensation information.

[0028] Optionally, when the obtaining unit converts the high-dimensional image information into a low-dimensional image feature vector, it is specifically used to:

[0029] The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each of the residual blocks includes a two-layer residual convolutional network.

[0030] Optionally, when the processing unit processes the low-dimensional image feature vector to obtain the prediction representation information, it is specifically used to:

[0031] Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector;

[0032] Forward prediction is performed according to the transformed image feature vector to obtain prediction representation information.

[0033] The beneficial effects of the present application are that, on the one hand, it can perform real-time diagnosis based on the high-dimensional image information of the wind turbine blades when the wind turbine is operating normally, without shutting down the machine, thus avoiding production losses caused by equipment downtime; on the other hand, the acquisition of image data is relatively simple, and existing camera equipment (such as high-definition cameras, drones, etc.) can be used for regular shooting to achieve fast and extensive data collection; on the other hand, image processing and computer vision technology can perform detailed analysis of wind turbine blades, identify various potential faults (such as cracks, corrosion, wear, etc.), and improve the accuracy of fault identification; finally, this solution combines advanced end-to-end algorithms such as deep learning, and does not require too much empirical design. This technology can realize automatic analysis of collected images, identify faults in real time, reduce manual intervention, and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 It is a flow chart of a fan blade fault diagnosis method based on predictive coding provided in an embodiment of the present application;

[0036] Figure 2 It is a structural schematic diagram of a diagnostic model provided in an embodiment of the present application;

[0037] Figure 3 It is a structural schematic diagram of a fan blade fault diagnosis device based on predictive coding provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] Image technology can comprehensively monitor the surface of wind turbine blades through high-resolution image acquisition equipment. By analyzing and processing images, defects, cracks and other faults on the blade surface can be effectively identified. In addition, using deep learning algorithms, the collected images can be automatically classified and identified, thereby achieving real-time monitoring of the status of wind turbine blades.

[0040] In practical applications, by regularly collecting images of wind turbine blades and combining them with computer vision and machine learning technologies, fault detection and identification can be achieved, that is, different types of faults can be accurately identified, such as surface cracks, corrosion, wear, etc.; health assessment, that is, through image analysis, the health of the blades can be evaluated, potential failure risks can be predicted, and maintenance plans can be formulated; automated monitoring, that is, the monitoring system based on image technology can realize automated inspections, reduce labor costs, and improve detection efficiency; data analysis and decision support, that is, the large amount of image data collected can provide important basis for fault analysis and decision-making, and help optimize operation and maintenance strategies.

[0041] Wind turbine blade fault detection and monitoring based on image technology can not only improve the accuracy and timeliness of fault identification, but also effectively improve the operating stability and safety of wind turbines and ensure the efficient use of renewable energy.

[0042] In view of the shortcomings of the existing manual detection of fan blade faults and the advantages of the above-mentioned image technology, this application provides a fan blade fault diagnosis method based on predictive coding, see Figure 1 , is a flow chart of the method, and the method may include the following steps.

[0043] S101, obtaining high-dimensional image information of a fan blade, and converting the high-dimensional image information into a low-dimensional image feature vector.

[0044] The method provided in this embodiment can be Figure 2 The diagnostic model implementation shown.

[0045] The high-dimensional image information of the wind turbine blades may be images obtained by photographing the blades of the wind turbine using various image acquisition devices. The image acquisition device may be an unmanned aerial vehicle device with a camera, or a ground-based camera device with a telephoto lens, without limitation.

[0046] like Figure 2 As shown, after obtaining the high-dimensional image information, the image information processing module of the diagnostic model can be used to process the high-dimensional image information to obtain the corresponding low-dimensional image feature vector, wherein each frame of high-dimensional image information can obtain a corresponding low-dimensional image feature vector.

[0047] S102, processing the low-dimensional image feature vector to obtain prediction representation information, and obtaining error features according to the prediction representation information and preset reference label features.

[0048] Step S102 can be performed by Figure 2 The forward propagation module implementation of the diagnostic model shown.

[0049] The forward propagation module can be used to process the low-dimensional feature vector generated by the image information processing module, further process the low-dimensional feature vector to condense the information carried by the low-dimensional image feature vector, and obtain the predicted representation information corresponding to the low-dimensional image feature vector. The predicted representation information obtained through condensation can then be compared with the correct reference label feature to output the error feature.

[0050] S103, when the convergence condition is not met, determining the prediction error according to the error characteristics, and performing the step of processing the low-dimensional image feature vector again according to the prediction error to obtain the prediction representation information, until the convergence condition is met.

[0051] Step S103 can be performed by Figure 2 The error correction module shown is implemented. In this embodiment, the error correction module is designed inspired by the predictive coding principle in neuroscience and can re-learn the error in a high-level representation space. Specifically, the error correction module receives the error features output by the forward propagation module, learns the error features to obtain the prediction error, and then the prediction error can be passed from top to bottom to the forward propagation module, so that the forward propagation module processes the low-dimensional image feature vector again according to the obtained prediction error.

[0052] The information interaction between the forward propagation module and the error correction module can be repeated multiple times until the error feature output by the forward propagation module meets the convergence condition. When the convergence condition is met, the error feature output by the forward propagation module can be used as the final error feature to predict the fault type in step S104.

[0053] The convergence conditions can be set as needed without limitation.

[0054] As some examples, the convergence condition may be that the difference between the error feature obtained this time and the error feature obtained last time is less than or equal to a set convergence threshold. In other words, if after multiple iterative processes, it is found that the error feature after each processing has remained basically unchanged, it can be determined that the error feature has converged, and the error feature at this time can be used as the final error feature for predicting the fault type.

[0055] As some other examples, the convergence condition may include that the number of iterations is greater than or equal to a preset upper limit of the number of iterations N, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain the image condensation information. That is, the low-dimensional image feature vector may be processed N times continuously by the forward propagation module, wherein during the second to Nth processing, the forward propagation module processes the low-dimensional image feature vector according to the prediction error output by the error correction module, and then the error feature output by the forward propagation module after the Nth processing is used as the final error feature for predicting the fault type.

[0056] S104, predicting the fault type of the wind turbine blade according to the final error feature obtained when the convergence condition is met.

[0057] Step S104 can be performed by Figure 2 The fault identification module shown is implemented. The fault identification module can receive the final error feature output by the forward propagation module, and process the final error feature through the multi-layer perceptron contained in the fault identification module to obtain the fault type corresponding to the high-dimensional image information, and the fault type can indicate whether the wind turbine blade that obtains the high-dimensional image information has a corresponding type of fault.

[0058] In this embodiment, the fault type output by the fault identification module may include any one of no fault, crack, corrosion, and wear. If no fault is output, it means that the wind blade whose high-dimensional image information is captured has no fault at present. If crack is output, it means that the wind blade whose high-dimensional image information is captured has a crack fault. If corrosion is output, it means that the wind blade whose high-dimensional image information is captured has a corrosion fault. If wear is output, it means that the wind blade whose high-dimensional image information is captured has a wear fault.

[0059] The diagnostic method provided in this embodiment can be implemented through an end-to-end learning method. The diagnostic module used in this method can use back-propagation technology to optimize all learnable parameters in the model during training.

[0060] The beneficial effects of the present application are that, on the one hand, it can perform real-time diagnosis based on the high-dimensional image information of the wind turbine blades when the wind turbine is operating normally, without shutting down the machine, thus avoiding production losses caused by equipment downtime; on the other hand, the acquisition of image data is relatively simple, and existing camera equipment (such as high-definition cameras, drones, etc.) can be used for regular shooting to achieve fast and extensive data collection; on the other hand, image processing and computer vision technology can perform detailed analysis of wind turbine blades, identify various potential faults (such as cracks, corrosion, wear, etc.), and improve the accuracy of fault identification; finally, this solution combines advanced end-to-end algorithms such as deep learning, and does not require too much empirical design. This technology can realize automatic analysis of collected images, identify faults in real time, reduce manual intervention, and improve work efficiency.

[0061] In some optional embodiments, converting high-dimensional image information into a low-dimensional image feature vector may include:

[0062] The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each residual block includes two layers of residual convolutional networks.

[0063] The low-dimensional image feature vector of this embodiment may be a low-dimensional image feature vector matrix with a certain width and height.

[0064] As mentioned before, the low-dimensional image feature vector can be obtained by Figure 2 The image information processing module shown is obtained. The image information processing module may include a 4-layer residual convolution network, which can be divided into two residual blocks, wherein the first residual block (denoted as residual block 1) includes 2 layers of residual convolution networks, and the second residual block (denoted as residual block 2) includes another 2 layers of residual convolution networks, the input of the first residual block is high-dimensional image information, and the output is used as the input of the second residual block, and the output of the second residual block is a low-dimensional image feature vector.

[0065] In each residual block, the input information is processed by two layers of residual convolutional networks and then undergoes a maximum pooling operation. The result of the maximum pooling operation is then added to the output of the two layers of residual convolutional networks to obtain the output of the residual block.

[0066] Specifically, for any residual block, its processing process can be expressed by the following formulas (1) and (2).

[0067]

[0068]

[0069] Among them, x0 represents the high-dimensional image information of the input image information processing module, L represents the subscript of the convolution layer contained in the residual block, L can be equal to 1 or 2, and the two residual convolution layers contained in each residual block can be connected in series, respectively recorded as residual convolution layer 1 and residual convolution layer 2. The input of residual convolution layer 1 is the input of the residual block, and the output of residual convolution layer 1 is the input of residual convolution layer 2. in，L represents the input of the residual convolution layer L, x L Represents the output of the residual convolution layer L, that is, x1 is the output of the residual convolution layer 1 in the residual block, x2 is the output of the residual convolution layer 2 in the residual block, Maxpool2d represents the two-dimensional maximum pooling operation, and x' represents the output of the residual block.

[0070] The convolution kernels of the four layers of convolutional networks contained in the above two residual blocks can be set to 64, 64, 64 and 16 respectively. After each residual block, that is, after processing two convolutional layers, the width and height of the output data can be reduced to 1 / 2 of the width and height of the input image, that is, the low-dimensional image feature vector matrix obtained after processing by two residual blocks. Its width can be 1 / 4 of the width of the high-dimensional image information, and its height can be 1 / 4 of the height of the high-dimensional image information.

[0071] In some optional embodiments, processing the low-dimensional image feature vector to obtain the prediction representation information may include:

[0072] Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector;

[0073] Forward prediction is performed based on the transformed image feature vector to obtain prediction representation information.

[0074] like Figure 2 As shown, the forward propagation module may include a first feature converter, a separation information representation module, and a first predictor.

[0075] When the low-dimensional image feature vector is processed by the forward propagation module, the first feature converter can perform feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector, and then the transformed image feature vector is processed by the first predictor for a forward prediction to obtain prediction representation information. The separation information representation module can separate the low-dimensional image feature vector used for forward prediction from the low-dimensional image feature vector of the labeled image during the training phase of the model.

[0076] In this embodiment, the reference label features may be obtained by extracting corresponding low-dimensional image feature vectors from multiple frames of images of wind blades that have been pre-labeled with fault types (referred to as label images), and then processing these low-dimensional image feature vectors using the first feature converter and the first predictor of the forward propagation module to obtain predicted representation information of these label images, and using the predicted representation information of these label images as reference label features.

[0077] After obtaining the predicted representation information, the predicted representation information can be subtracted from the pre-obtained reference label feature, and the result obtained is used as the error feature.

[0078] Optionally, the first feature converter may include, in order: a first two-dimensional convolution layer, a first batch normalization layer, a leaky rectified linear unit layer, a second two-dimensional convolution layer, and a second batch normalization layer.

[0079] Among them, two two-dimensional convolution layers are used to perform two-dimensional convolution processing on the input data, two batch normalization layers are used to perform batch normalization processing on the input data, and the rectified linear unit layer with leakage is used to rectify the output of the first batch normalization layer.

[0080] The relevant parameters of the first two-dimensional convolutional layer can be: the number of input channels is 8, the number of output channels is 1, the convolution kernel size is set to 3x3, the convolution step is 1, and the padding is 1.

[0081] The relevant parameters of the second two-dimensional convolutional layer can be: the number of input channels is 8, the number of output channels is 1, the convolution kernel size is set to 3x3, the convolution step is 1, and the padding is 1.

[0082] When the first feature converter performs feature space transformation on the low-dimensional image feature vector, the low-dimensional image features sequentially pass through the first two-dimensional convolution layer, the first batch normalization layer, the rectified linear unit layer with leakage, the second two-dimensional convolution layer, the second batch normalization layer, and finally obtain the transformed image feature vector output by the second batch normalization layer. In other words, the process of obtaining the transformed image feature vector is equivalent to:

[0083] The low-dimensional image feature vector is sequentially subjected to two-dimensional convolution processing, batch regularization processing, rectification processing, two-dimensional convolution processing and batch regularization processing to obtain a transformed image feature vector.

[0084] Optionally, the first predictor may include a two-dimensional convolution layer, the number of input channels of the two-dimensional convolution layer is 8, the number of output channels is 1, the convolution kernel size is 3x3, the stride is 1, and 1 is added to the padding around.

[0085] In some embodiments, the separated information representation module can separate the original unclassified information from the information with label features.

[0086] That is to say, Figure 2 In the diagnostic model shown, the separated information representation module can be used only in the training stage of the diagnostic model to separate the information used for forward prediction and the information used as reference label features from the input information. In the reasoning stage, that is, when using the model to process the high-dimensional image information of the wind blades actually taken to predict the fault type, this module can be not used. In this case, the transformed image feature vector can be directly output to the first predictor to obtain the corresponding prediction representation information.

[0087] In some embodiments, the separation information representation module may also be used to:

[0088] Determine whether the transformed image feature vector output by the first feature converter corresponds to an image of a fault type that needs to be diagnosed in the inference stage, or corresponds to an image whose fault type has been labeled in the training stage. If it belongs to the former, the transformed image feature vector is input into the first predictor; if it belongs to the latter, the transformed image feature vector is stored as a reference label feature so that it can be used to determine the error feature in the inference stage.

[0089] Optionally, the prediction error includes a direct prediction error and a high-level prediction error;

[0090] Determine the forecast error based on the error characteristics, including:

[0091] Directly learn the error features based on the two-dimensional convolutional layer to obtain the direct prediction error;

[0092] The error features are processed according to the second predictor and the transformed error module to obtain a high-level prediction error.

[0093] In some optional embodiments, see Figure 2 , the error correction module may include a direct feedback path and an indirect feedback path.

[0094] like Figure 2 As shown, the direct feedback path may include a two-dimensional convolution layer, and the parameters of the two-dimensional convolution layer may be: the number of input channels is 16, the number of output channels is 8, the convolution kernel size is 3x3, the step size is 1, and 1 is added to the surrounding padding. After obtaining the error feature, the two-dimensional convolution layer of the direct feedback path can directly learn the error feature, obtain the corresponding direct prediction error, and input the direct prediction error to the first feature converter.

[0095] The indirect feedback path may include a conversion error module and a second predictor, the conversion error module and the second predictor being jump-connected.

[0096] The conversion error module may sequentially include a 2D convolution layer and a 2D maximum pooling layer for keeping the input and output sizes consistent. The parameters of the 2D convolution layer may be: the number of input channels is 16, the number of output channels is 16, the convolution kernel size is 3x3, the stride is 1, and 1 padding is added around, and the parameters of the 2D maximum pooling layer may be: the pooling kernel size is 2x2, and the stride is 2. The 2D maximum pooling layer can be used to halve the width and height of the input, thereby extracting the main information of the feature and reducing the size of the feature map.

[0097] The indirect feedback path uses a jump connection to subtract the features learned by the conversion error module from the error features (i.e., the output of the conversion error module) from the features predicted by the second predictor (i.e., the output of the second predictor), and the result is used as the high-level prediction error. Among them, the input of the second predictor includes the output of the conversion error module. Before the high-level prediction error is passed down to the first feature converter of the forward propagation module, a layer of two-dimensional convolution layer and a layer of batch normalization layer can be used to reduce the dimension of the high-level prediction error to obtain a simplified high-level prediction error, and then the simplified high-level prediction error is input to the first feature converter.

[0098] At this point, the first feature converter can process the low-dimensional image feature vector again based on the input direct prediction error and the simplified high-level prediction error, and then use the first predictor to predict again to obtain new prediction representation information and new error features, and so on, until the error features obtained at a certain time meet the convergence conditions.

[0099] Among them, the parameters of the two-dimensional convolution layer used to obtain the simplified high-level prediction error can be: the number of input channels is 16, the number of output channels is 8, the convolution kernel size is 3x3, the step size is 1, and 1 is added to the surrounding padding to keep the size of the output feature map consistent with the input.

[0100] The batch normalization layer used to obtain the simplified high-level prediction error can batch normalize the feature map with 8 channels output by the 2D convolution layer to improve the convergence speed and stability of the model. Finally, an upsampling operation is performed to increase the size of the feature map by 2 times.

[0101] In some optional embodiments, the fault identification module mainly includes a multilayer perceptron, which includes two residual blocks, an adaptive average pooling layer and an output head in sequence. To ensure that the input and output dimensions match, the first residual block includes a downsampling module, which may include a convolution layer with a stride of 2 and a convolution kernel size of 1x1 and a batch normalization layer. The downsampling module can convert the number of channels of the input final error feature from 8 to 16, while reducing the spatial size of the final error feature.

[0102] The second residual block further processes the output of the first residual block. Specifically, the second residual block may also include a downsampling module, which may be composed of a convolution layer with a stride of 2 and a convolution kernel size of 1x1 and a batch normalization layer. The downsampling module may convert the number of channels of the input data from 16 to 8 and compress the size of the output data.

[0103] The adaptive average pooling layer pools the data output by the second residual block to a fixed 1x1 size to obtain the global feature representation corresponding to the high-dimensional image information, providing a more compact feature expression for subsequent classification or regression tasks.

[0104] The output head processes the above global feature representation and outputs label information characterizing the fault type. For example, no fault, crack, corrosion, and wear correspond to labels 1 to 4 respectively. If label 1 is output, it is determined that the predicted fault type is no fault. If label 2 is output, it is determined that the output fault label is crack, and so on.

[0105] The diagnostic model of this embodiment can be trained as follows.

[0106] In the first step, drones and other equipment are used to collect wind turbine blade data sets, and some samples in the data sets are manually marked for fault types. According to the fault type, they are divided into 8 types: common damage, leading edge corrosion, leading edge cracking, trailing edge damage, blade root fracture, blade tip cracking and breaking, surface cracks, and lightning damage. After that, the image information processing module combines 8 unlabeled continuous original images with 8 types of images, one image for each type, into a training sample, that is, a training sample contains 16 images, including 8 original images without manual fault type marking, and 8 labeled images with manual fault type marking, and the fault types of the 8 labeled images are different. The construction method of the test sample is the same as that of the training sample, so it will not be repeated.

[0107] The image information processing module uniformly scales all image widths and heights to 80x80 for subsequent batch processing. Finally, for each training sample, the image processing module uses a 4-layer residual convolutional network to batch process all images and process each image of the training sample into a 20x20 feature map, which is the low-dimensional image feature vector of S101.

[0108] In the second step, the forward propagation module is used to perform refined learning of the feature map. After the feature map is processed by the first feature converter, the size of the feature map is still 20x20. After that, the feature map of the 16 images contained in a training sample is divided by the separation information representation module, and the 8 low-dimensional image feature vectors corresponding to the 8 original images are input into the first predictor for forward prediction. The size of the predicted feature map (that is, the aforementioned prediction representation information) is 20x20. The low-dimensional image feature vectors of the last 8 label images are used as reference label features to perform a difference operation with the prediction representation information output by the first predictor, once in the forward direction and once in the reverse direction, and finally spliced ​​together to form an error feature E0 with a shape of [16, 20, 20]. The forward direction here refers to the subtraction of the reference label feature from the prediction representation information of the original image, and the reverse direction refers to the subtraction of the prediction representation information from the reference label feature.

[0109] The third step is to determine whether the number of iterations n is greater than or equal to the upper limit of the number of iterations N. If it is greater than or equal to N, the error feature E0 at this time is sent to the fault identification module as the final error feature. If it is less than N, the error feature E0 at this time is sent to the error correction module.

[0110] In the fourth step, the error correction module learns the error feature E0 through the direct feedback path and the indirect feedback path respectively. The direct feedback path directly learns the error feature E0, and reduces the dimension of the error feature E0 data to the size of [8, 20, 20] through a layer of two-dimensional convolution, and obtains a direct prediction error of size [8, 20, 20]. The indirect feedback path learns the error feature E0 as a higher-level representation through the conversion error module, and the data size is [8, 10, 10]. After forward learning by the second predictor, the data size is still [8, 10, 10]. The jump connection performs a difference operation between the features learned by the conversion error module and the features predicted by the second predictor to obtain a high-level prediction error. The difference operation still uses two methods, forward and reverse, and the data size of the obtained high-level prediction error is [16, 10, 10].

[0111] During the propagation of the high-level prediction error to the forward propagation module, a two-dimensional convolution layer and a batch normalization layer are used to reduce the dimension of the high-level prediction error. The data finally input to the forward propagation module includes the direct prediction error and the simplified high-level prediction error obtained by dimensionality reduction. Among them, the batch normalization layer performs batch normalization on the high-level prediction error with 8 channels, and the resulting feature map size is [8, 10, 10]. Finally, an upsampling operation is performed to scale the size of the feature map to [8, 20, 20]. In other words, the simplified high-level prediction error input to the forward propagation module can be a feature map of size [8, 20, 20].

[0112] In the fifth step, the two types of feature maps of the direct feedback path and the indirect feedback path are summed with the first 8-dimensional data in the feature map of [16, 20, 20] output by the first feature converter, and then the result after the summation operation is processed again in the second step.

[0113] That is to say, a direct prediction error of size [8, 20, 20] is added to the first 8 feature maps of [20, 20] outputted by the first feature converter (that is, the feature maps corresponding to the 8 original images), and a simplified high-level prediction error of size [8, 20, 20] is also added to the first 8 feature maps of [20, 20] outputted by the first feature converter. Finally, the first 8 feature maps of [20, 20] to which the above prediction error is added and the last 8 feature maps of [20, 20] to which the above error is not added (that is, the feature maps corresponding to the 8 label images in the training sample) are again sent to the forward propagation module as input to the forward propagation module for processing.

[0114] In the sixth step, the final error feature E0 is sent to the fault recognition module for learning. The fault recognition module contains two residual blocks and an adaptive average pooling layer. The final learned feature dimension is [8, 1, 1], that is, a vector containing 8 data. Each data of the vector corresponds to an original image, and each data is equivalent to the fault label of the corresponding original image. After obtaining these fault labels, the fault labels can be compared with the true labels to realize supervised learning. The true labels can be obtained by manually annotating the original images.

[0115] In addition, alternative solutions of specific embodiments are also considered, and the number of images included in the training and test samples can be flexibly allocated, but the number of original images must be the same as the number of labeled images (fault categories).

[0116] Through practice, the accuracy of this method in identifying 8 types of faults increases with the increase of the upper limit N of the number of iterations. For example, when N is equal to 4, the highest recognition accuracy can reach 85.3%.

[0117] The method of this embodiment can also be used to diagnose faults of other structures besides wind blades, such as wind turbine gearbox faults, and is applicable to a wide range of applications.

[0118] The present application also provides a fan blade fault diagnosis device based on predictive coding, see Figure 3 , is a schematic diagram of the structure of the device, and the device may include the following units.

[0119] An obtaining unit 301 is used to obtain high-dimensional image information of the fan blades and convert the high-dimensional image information into a low-dimensional image feature vector;

[0120] The processing unit 302 is used to process the low-dimensional image feature vector to obtain prediction representation information, and obtain error features according to the prediction representation information and preset reference label features;

[0121] The determining unit 303 is used to determine the prediction error according to the error characteristics when the convergence condition is not met, and to perform the step of processing the low-dimensional image feature vector again according to the prediction error to obtain the prediction representation information until the convergence condition is met;

[0122] The prediction unit 304 is used to predict the fault type of the wind turbine blade according to the final error feature obtained when the convergence condition is met.

[0123] Optionally, the convergence condition includes that the number of iterations is greater than or equal to a preset upper limit N of the number of iterations, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain image condensation information.

[0124] Optionally, when the obtaining unit 301 converts the high-dimensional image information into a low-dimensional image feature vector, it is specifically used to:

[0125] The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each residual block includes two layers of residual convolutional networks.

[0126] Optionally, when the processing unit 302 processes the low-dimensional image feature vector to obtain the prediction representation information, it is specifically used to:

[0127] Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector;

[0128] Forward prediction is performed based on the transformed image feature vector to obtain prediction representation information.

[0129] Optionally, the processing unit 302 performs feature space transformation on the low-dimensional image feature vector to obtain the transformed image feature vector, which is specifically used for:

[0130] The low-dimensional image feature vector is sequentially subjected to two-dimensional convolution processing, batch regularization processing, rectification processing, two-dimensional convolution processing and batch regularization processing to obtain a transformed image feature vector.

[0131] Optionally, the prediction error includes a direct prediction error and a high-level prediction error;

[0132] When the determination unit 303 determines the prediction error according to the error feature, it is specifically used to:

[0133] Directly learn the error features based on the two-dimensional convolutional layer to obtain the direct prediction error;

[0134] The error features are processed according to the second predictor and the transformed error module to obtain a high-level prediction error.

[0135] The specific working principle of the fan blade fault diagnosis device based on predictive coding provided in this embodiment can be found in the relevant steps of the fan blade fault diagnosis method based on predictive coding provided in any embodiment of the present application, and will not be repeated here.

[0136] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0137] For the convenience of description, the above system or device is described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0138] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0139] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0140] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A fan blade fault diagnosis method based on predictive coding, characterized in that: include: Obtaining high-dimensional image information of a fan blade, and converting the high-dimensional image information into a low-dimensional image feature vector; Processing the low-dimensional image feature vector to obtain prediction representation information, and obtaining error features according to the prediction representation information and preset reference label features; In the case where the convergence condition is not met, determining a prediction error according to the error feature, and performing the step of processing the low-dimensional image feature vector again according to the prediction error to obtain prediction representation information until the convergence condition is met; The fault type of the wind turbine blade is predicted according to the final error characteristics obtained when the convergence condition is met.

2. The method according to claim 1, characterized in that The convergence condition includes that the number of iterations is greater than or equal to a preset upper limit N of the number of iterations, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain image condensation information.

3. The method according to claim 1, characterized in that The converting the high-dimensional image information into a low-dimensional image feature vector comprises: The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each of the residual blocks includes a two-layer residual convolutional network.

4. The method according to claim 1, characterized in that: The processing of the low-dimensional image feature vector to obtain prediction representation information includes: Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector; Forward prediction is performed according to the transformed image feature vector to obtain prediction representation information.

5. The method according to claim 4, characterized in that The step of performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector includes: The low-dimensional image feature vector is sequentially subjected to two-dimensional convolution processing, batch regularization processing, rectification processing, two-dimensional convolution processing and batch regularization processing to obtain a transformed image feature vector.

6. The method according to claim 1, characterized in that The prediction error includes a direct prediction error and a high-level prediction error; The determining the prediction error according to the error feature comprises: Directly learning the error feature according to the two-dimensional convolutional layer to obtain a direct prediction error; The error features are processed according to the second predictor and the transformed error module to obtain a high-level prediction error.

7. A fan blade fault diagnosis device based on predictive coding, characterized in that: include: An obtaining unit, used for obtaining high-dimensional image information of the fan blades, and converting the high-dimensional image information into a low-dimensional image feature vector; A processing unit, used to process the low-dimensional image feature vector to obtain prediction representation information, and obtain error features according to the prediction representation information and preset reference label features; a determining unit, configured to determine a prediction error according to the error feature when the convergence condition is not satisfied, and to perform the step of processing the low-dimensional image feature vector again according to the prediction error to obtain prediction representation information until the convergence condition is satisfied; The prediction unit is used to predict the fault type of the wind turbine blade according to the final error characteristics obtained when the convergence condition is met.

8. The device according to claim 7, characterized in that The convergence condition includes that the number of iterations is greater than or equal to a preset upper limit N of the number of iterations, wherein the number of iterations represents the number of times the low-dimensional image feature vector is processed to obtain image condensation information.

9. The device according to claim 7, characterized in that When the obtaining unit converts the high-dimensional image information into a low-dimensional image feature vector, it is specifically used to: The high-dimensional image information is processed in sequence according to two residual blocks to obtain a low-dimensional image feature vector, and each of the residual blocks includes a two-layer residual convolutional network.

10. The device according to claim 7, characterized in that When the processing unit processes the low-dimensional image feature vector to obtain the prediction representation information, it is specifically used to: Performing feature space transformation on the low-dimensional image feature vector to obtain a transformed image feature vector; Forward prediction is performed according to the transformed image feature vector to obtain prediction representation information.

Images