Fan blade state diagnosis method and device, electronic equipment and medium

Abstract

The invention discloses a fan blade state diagnosis method and device, electronic equipment and a medium. The method comprises the steps that a data set needed by blade state diagnosis is obtained, and each sample in the data set at least comprises dynamic image data of a blade and blade state data; training a constructed blade state diagnosis model based on the data set, and when model parameters of the blade state diagnosis model converge, obtaining the trained blade state diagnosis model; and at least inputting dynamic image data of a to-be-detected blade into the trained blade state diagnosis model to obtain a diagnosis result of the to-be-detected blade. According to the invention, the state of the fan blade can be quickly identified and the diagnosis accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power, and in particular to a method and device for diagnosing the state of a wind turbine blade, an electronic device, and a medium. Background Art

[0002] While wind power generation has developed rapidly, the overhaul and maintenance of wind turbine units are gradually becoming a bottleneck restricting the rapid development of the wind power industry. Since most wind turbine units work in complex and harsh environments and are subjected to alternating loads for a long time, various faults occur in the wind turbines, and maintaining and repairing these faults result in a relatively large operating cost during the life cycle of the wind turbines. According to statistics, the maintenance and repair costs account for about one-fifth of the total cost of wind power generation. Detecting faults as early as possible and dealing with them in a timely manner are beneficial to improving the power generation efficiency and reducing the operating cost of wind turbine units. Therefore, more effective monitoring of the state of wind turbine units has become a research hotspot in the field of wind power technology.

[0003] Currently, common non-destructive diagnostic methods for wind turbine blades at home and abroad include visual inspection, ultrasonic diagnosis, etc.

[0004] Among them, the visual inspection method observes the surface of the blade and the accessible area inside through the naked eye and with the aid of a magnifying glass and an endoscope, and can check obvious defects such as surface scratches, blisters, wrinkles, dents, lack of glue, dry fibers, cracks, and interface delamination. Especially after the blade is infused and cured and before the mold is bonded, problems can be well detected through visual diagnosis, and corresponding remedial measures can be taken in a timely manner. However, when the blade is mold-bonded, the visual inspection method can only diagnose the areas that can be reached by people. During the operation of the wind turbine, the visual inspection method cannot judge the fracture situation of the blade, resulting in inaccurate diagnosis of the blade state.

[0005] The ultrasonic diagnosis method uses the influence of the acoustic properties of the composite material itself or defects on the ultrasonic propagation path to diagnose defects inside or on the surface of the material. Ultrasonic waves can diagnose defects such as delamination, debonding, pores, lack of glue, bubbles, cracks, inclusions, and impacts in composite material components. Ultrasonic diagnosis has the advantages of high sensitivity and can accurately determine the position and distribution of defects. The mobile ultrasonic scanners AMS-46 and AMS-57 developed by FORCE Technology in Denmark, and the phased array ultrasonic flaw detector OmniScan_MX2 developed by Olympus Corporation all use the ultrasonic principle to diagnose the defects of the blade. The ultrasonic diagnosis method can only perform static diagnosis on the blade and cannot perform dynamic diagnosis during the operation of the blade. Different specifications of probes need to be replaced for different types of defects. Therefore, this diagnosis method cannot accurately diagnose the state of the blade. Summary of the Invention

[0006] The present invention provides a method, apparatus, electronic device and medium for diagnosing the state of a fan blade, which are used to solve the technical problem that the prior art cannot accurately diagnose the state of the blade.

[0007] According to one aspect of the present invention, there is provided a method for diagnosing the state of a fan blade, including:

[0008] Obtaining a data set required for diagnosing the state of the blade, where each sample in the data set includes at least dynamic image data of the blade and the state of the blade;

[0009] Training a pre-constructed blade state diagnosis model based on the data set, and obtaining the trained blade state diagnosis model when the model parameters of the blade state diagnosis model converge;

[0010] Inputting at least the dynamic image data of the blade to be measured into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be measured.

[0011] Optionally, the blade state diagnosis model is a bilinear Transformer interaction network, and the bilinear Transformer interaction network includes at least two feature extraction networks, a feature information interaction network, and a fusion module corresponding to the feature extraction network;

[0012] The feature extraction network is used to extract key features of the input image;

[0013] The feature information interaction network is used to perform self-interaction on the key features extracted by the feature extraction network, and perform homogeneous interaction on the key features extracted by at least two feature extraction networks respectively; the input images of at least two feature extraction networks are homogeneous images;

[0014] The fusion module is used to fuse the features output by the self-interaction with the features of the homogeneous interaction.

[0015] Optionally, the feature extraction network obtains a weight matrix during fast operation based on the multi-head self-attention mechanism, and selects the feature corresponding to the maximum value in the weight matrix as the extracted image feature.

[0016] Optionally, the homogeneous interaction is:

[0017] F 1 ' = W 12 × F 1

[0018] F 2 ' = W 21 × F 2

[0019] Wherein, F 1 and F2 are the features generated after two sets of similar images pass through the feature extraction network; F 1 ' represents the feature F 1 the features obtained through similar interaction; F 2 ' represents the feature F 2 the features obtained through similar interaction; W 12 and W 21 represent the weight matrix;

[0020] The solution formula for the weight matrix is:

[0021]

[0022] F P and F q are the features generated after two sets of similar images pass through the feature extraction network; W ij represents the calculated interaction weight, where W 12 is calculated from F p F q T is calculated, and W 21 is calculated from F q F p T is calculated; exp is the exponential function, i and j represent the channel indices in the neural network, and k represents the number of heads of the attention mechanism;

[0023] The fused feature obtained by fusing the features output from self-interaction with the features of the similar interaction is:

[0024] F final1 = cat(F 1 '+ F 1 ”)

[0025] F final2 = cat(F 2 '+ F 2 ”)

[0026] In the formula, F 1 ” represents the feature F 1 obtained through self-interaction, F final1 represents the feature F 1 ” output from self-interaction and the fused feature obtained by fusing with the feature of the similar interaction F 1 '; F 2 ” represents the feature F 2 obtained through self-interaction, F final2 represents the feature F 2 ” output from self-interaction and the fused feature obtained by fusing with the feature of the similar interaction F 2 '.

[0027] Optionally, the loss function of the blade state diagnosis model is a cross-entropy loss, including:

[0028]

[0029] where x represents a sample, y represents the actual label, a is the output predicted by the model, and n represents the total number of samples.

[0030] Optionally, the sample also includes weather data; the blade state data at least includes non-destructive, crack, split, defect, and lightning strike.

[0031] Optionally, before obtaining the dataset required for blade state diagnosis, where each sample in the dataset at least includes the dynamic image data of the blade and the blade state, further includes:

[0032] Collect the dynamic image data of the wind turbine blade during operation using a high-speed photography system, and label the blade state of the dynamic image data.

[0033] According to another aspect of the present invention, there is provided a wind turbine blade state diagnosis device, including:

[0034] A data acquisition unit for acquiring a dataset required for blade state diagnosis, where each sample in the dataset at least includes the dynamic image data of the blade and the blade state;

[0035] A training unit for training the constructed blade state diagnosis model based on the dataset, and obtaining the trained blade state diagnosis model when the model parameters of the blade state diagnosis model converge;

[0036] A diagnosis unit for inputting at least the dynamic image data of the blade to be tested into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be tested.

[0037] According to another aspect of the present invention, there is provided an electronic device, the electronic device includes:

[0038] At least one processor; and

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the wind turbine blade state diagnosis method according to any embodiment of the present invention.

[0041] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the fan blade state diagnosis method according to any embodiment of the present invention when executed.

[0042] In the technical solution of the embodiment of the present invention, by obtaining a data set required for blade state diagnosis, each sample in the data set includes at least dynamic image data of the blade and the blade state, performing supervised learning on the blade state diagnosis model, and inputting the dynamic image data of the blade to be measured into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be measured. The present application can diagnose various states of the fan blade and can quickly and accurately identify the state of the fan blade.

[0043] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0045] Figure 1 is a flowchart of a fan blade state diagnosis method according to Embodiment 1 of the present invention;

[0046] Figure 2 is a schematic structural diagram of a high-speed photography system installed in a wind turbine in the embodiment of the present invention;

[0047] Figure 3 is a schematic structural diagram of a high-speed photography system provided in the embodiment of the present invention;

[0048] Figure 4 is a schematic structural diagram of a bilinear Transformer interaction network model in the embodiment of the present invention;

[0049] Figure 5 is a schematic structural diagram of the encoder part in the embodiment of the present invention;

[0050] Figure 6 is a schematic structural diagram of a fan blade state diagnosis device according to Embodiment 2 of the present invention;

[0051] Figure 7It is a schematic structural diagram of an electronic device for implementing the fan blade state diagnosis method according to an embodiment of the present invention. Detailed implementation manners

[0052] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0054] Embodiment 1

[0055] Figure 1 A flowchart of a fan blade state diagnosis method is provided for Embodiment 1 of the present invention. As Figure 1 shown, the method includes:

[0056] S101. Obtain a data set required for blade state diagnosis, where each sample in the data set includes at least dynamic image data of the blade and the blade state.

[0057] It should be noted that the dynamic image data in each sample can be collected by a high-speed photography system provided on the fan. When the fan is working, the high-speed photography system can be turned on to collect the dynamic image data of the blade in real time, and then the pictures in the dynamic image data are marked with the blade state. The dynamic image data of the blade and the blade state can be used as a sample data to construct a data set. Of course, the sample data can also include other data, such as weather data, blade rotation speed, and the number of the high-speed photography system, etc.

[0058] The dynamic image data refers to the image data of the fan blades during operation. The high-speed photography system can use high-frequency cameras to collect high-quality blade images. The blade states can include various states such as non-damaged blades, cracked blades, split blades, missing blade parts, and lightning-struck blades. The blades can be labeled by manual labeling or other labeling methods.

[0059] S102. Based on the dataset, train and construct a well-built blade state diagnosis model. When the model parameters of the blade state diagnosis model converge, obtain the trained blade state diagnosis model.

[0060] It should be noted that in this embodiment, the blade state diagnosis model is a neural network model, which can adopt a relatively conventional neural network architecture or a required neural network architecture constructed according to requirements. At the beginning of training, the dataset can be divided into a training set, a test set, and a validation set in proportion to train the blade state diagnosis model. This embodiment adopts a supervised training method, enabling the blade state diagnosis model to learn the characteristics under various blade states, so as to output a matching blade state according to the input sample data.

[0061] During the training process, an appropriate loss function can be selected to perform iterative training on the neural network. In this embodiment, a fixed number of iterations can be set or a parameter threshold can be set. When the parameters of the neural network model reach the set number of iterations, stop training, or when the parameters of the neural network model converge, stop training, and obtain the trained blade state diagnosis model.

[0062] S103. Input at least the dynamic image data of the blade to be measured into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be measured.

[0063] When it is necessary to predict the fan blade to be measured, the target high-speed photography system can be controlled to collect the dynamic image data of the target blade during operation, and the dynamic image data is input into the trained blade state diagnosis model. The blade state diagnosis model can quickly predict the state of the fan blade, so that the staff can understand the current state of the fans in the wind farm, and then quickly respond to repair and maintain the fan blades to ensure the normal operation of the fans.

[0064] It should be noted that when the training samples also include data such as weather data, blade rotation speed, and the number of the high-speed photography system, etc., when diagnosing the blade to be measured, it is also necessary to collect the current weather data, blade rotation speed, and the number of the high-speed photography system, etc., and input the dynamic image data of the target blade during operation, the current weather data, blade rotation speed, and the number of the high-speed photography system, etc. into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be measured.

[0065] In the technical solution of the embodiment of the present invention, by obtaining a data set required for blade state diagnosis, each sample in the data set includes at least dynamic image data of the blade and the blade state, performing supervised learning on the blade state diagnosis model, and inputting the dynamic image data of the blade to be measured into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be measured. This application can diagnose various states of the fan blade and can quickly and accurately identify the state of the fan blade.

[0066] In one embodiment, a plurality of high-speed photography systems can be arranged on the fixed bracket of the fan tower to collect blade images in multiple directions. Specifically, it can be as Figure 2 and Figure 3 shown in the structural schematic diagram of the high-speed photography system installed on the fan. Figure 2 and Figure 3 include a high-speed photography system 1, a fixed bracket 2, a steel belt 3, and a fan tower 4.

[0067] Among them, the steel belt 3 is arranged at the middle position of the height of the fan tower 4 and is arranged around the fan tower 7 in a circle. Two steel belts 3 can be arranged, and the two steel belts 3 are abutted up and down. A plurality of fixed brackets 2 can be included, for example, three. The plurality of fixed brackets 2 are evenly arranged around the tower 4 in a circle. The fixed bracket 2 can be set as a right-angled triangular structure, and one side of the right angle is fixed on the steel belt 3, that is, the right-angled triangular structure is fixed on the side wall of the fan tower 4 through the steel belt 3. Among them, the other right-angled surface of the right-angled triangular structure faces upward and the inclined surface faces downward. The high-speed photography system 1 can be arranged on the upper surface of the other right-angled surface of the fixed bracket 2. A high-speed photography system 1 is arranged above each triangular structure so that when the fan blade adjusts its direction, other high-speed photography systems can also be used to collect blade dynamic image signals. During use, control one of the high-speed photography systems 1 to collect the real-time dynamic image signals of the blade.

[0068] In one embodiment, the collected dynamic image data can be preprocessed. The preprocessing can at least include denoising and analog-to-digital conversion to improve the quality of the data set for model training, thereby improving the accuracy of fan blade state diagnosis.

[0069] In one embodiment, each sample in the data set further includes at least one of weather data, blade rotation speed, and high-speed photography system number.

[0070] Among them, the weather data is the weather condition of the collection address, and the collection address is the address where the wind turbine is located. The relevant information on the weather condition can be obtained from the weather forecast information of the collection address. Among them, the weather condition can include meteorological parameters such as temperature, wind force, and light. In addition, for the same collection address, if the collection time is different, the weather condition is also different. Therefore, the weather condition can be regarded as a diagnostic parameter jointly affected by the collection address and the collection time. For example, generally, the temperature is higher during the day, lower at night, lower in winter, and higher in summer. When the wind turbine is working, the relevant electrical equipment will also generate heat, resulting in a higher temperature of the entire wind turbine. A higher temperature will affect the operating state of the wind turbine, and further affect the real-time dynamic image signal of the blade. For example, generally, the greater the wind force, the more flying debris in the environment, which will also affect the real-time dynamic image signal of the blade.

[0071] Among them, the high-speed photography system number is the identification code determined in advance for each high-speed photography system 1, which is used to uniquely identify a high-speed photography system 1 and can also reflect the type of the high-speed photography system. The performance of the high-speed photography system 1 will affect the quality of the real-time dynamic image of the blade collected. For example, when the performance of the high-speed photography system 1 is poor, there are more system noises in the real-time dynamic image signal of the blade collected, resulting in a poor quality of the real-time dynamic image signal of the blade, thereby reducing the accuracy of the blade state diagnosis result; on the contrary, when the performance of the high-speed photography system 1 is good, it helps to improve the accuracy of the blade state diagnosis result.

[0072] Furthermore, the sample can also include other information, such as the type of the wind turbine, the number of the wind turbine, the blade information, etc. The present application does not limit this.

[0073] In one embodiment, the blade state diagnosis model is a bilinear Transformer interaction network, and the bilinear Transformer interaction network includes at least two feature extraction networks, a feature information interaction network, and a fusion module corresponding to the feature extraction network;

[0074] The feature extraction network is used to extract the key features of the input image;

[0075] The feature information interaction network is used to perform self-interaction on the key features extracted by the feature extraction network, and perform homogeneous interaction on the key features extracted by at least two of the feature extraction networks respectively; the input images of at least two of the feature extraction networks are homogeneous images;

[0076] The fusion module is used to fuse the features output by the self-interaction with the features of the homogeneous interaction.

[0077] It should be noted that the bilinear Transformer interaction network includes at least two feature extraction networks. For example, Figure 4 the architecture diagram of the bilinear Transformer interaction network model shown in Figure 4 includes two TransFG modules, and the TransFG module is used as a feature extraction module to extract features from the input image.

[0078] The TransFG module may include an encoder part, and the encoder part includes a multi-head self-attention mechanism and a multi-layer perceptron. Specifically, as shown in Figure 5 the encoder structure diagram shown in Figure 5 the feature corresponding to the maximum value in the weight matrix obtained after calculating the parameters of the multi-head self-attention mechanism in the encoder structure diagram is used as the image feature.

[0079] Among them, the feature information interaction network can complete self-interaction on the features selected by the feature extraction network in a channel interaction manner to strengthen the discriminability of the features themselves and obtain more expressive features; by constructing a bilinear structure to provide a homogeneous interaction process to judge the differences between images. In this embodiment, the interaction between homogeneous images is completed with the help of the bilinear structure, and then the common features of the sample images of the same type are obtained. The common features can be used as the key to distinguish from other categories, highlighting the differences between different category images.

[0080] Specifically, the self-interaction process of the feature information interaction network can be expressed as:

[0081] F 1 ” = F 1 W 1 + F 1

[0082] F 2 ” = F 2 W 2 + F 2

[0083] Among them, F 1 ” represents the feature obtained after the self-interaction of feature F 1 , F 2 ” represents the feature obtained after the self-interaction of feature F 2 , W 1 and W 2 are self-interaction enhancement matrices; F 1 and F 2 are the features generated after two groups of homogeneous images pass through the feature extraction network.

[0084] In one embodiment, the homogeneous interaction can be expressed as:

[0085] F 1 ' = W 12 × F 1

[0086] F 2 ' = W 21 × F 2

[0087] Among them, F 1 and F 2 are the features generated after two sets of similar images pass through the feature extraction network; F 1 ' represents the feature obtained after the feature F 1 goes through homogeneous interaction; F 2 ' represents the feature obtained after the feature F 2 goes through homogeneous interaction; W 12 and W 21 represent weight matrices;

[0088] The solution formula for the weight matrix is:

[0089]

[0090] F P and F q are the features generated after two sets of similar images pass through the feature extraction network; W ij represents the calculated interaction weight, where W 12 is calculated from F p F q T 21 is calculated from F q p T final1 1 exp is the exponential function, i and j represent the channel indices in the neural network, and k represents the number of heads of the attention mechanism.

[0091] The fusion module is used to fuse the features output by self-interaction with the features of homogeneous interaction, specifically as follows:

[0092] The fusion feature obtained by fusing the features output by self-interaction with the features of homogeneous interaction is:

[0093] F final1 = cat(F 1 '+ F 1 ”)

[0094] F final2 = cat(F 2 '+ F 2 ”)

[0095] In the formula, F 1 ” represents the feature F 1Features obtained through self-interaction, F final1 Indicates the feature F output by self-interaction 1 "The feature F interacting with the same type 1 ' is fused to obtain the fused feature; F 2 "Indicates the feature F 2 Features obtained through self-interaction, F final2 Indicates the feature F output by self-interaction 2 "The feature F interacting with the same type 2 ' is fused to obtain the fused feature.

[0096] In one embodiment, cross-entropy loss can be used as the loss function of the blade state diagnosis model, and the model parameters are adjusted according to the calculation result of the loss function. The loss function can be expressed as:

[0097]

[0098] In the formula, x represents the sample, y represents the actual label, a is the output predicted by the model, and n represents the total number of samples.

[0099] In one embodiment, the sample further includes weather data; the blade state data at least includes non-destructive, crack, split, defect, and lightning strike.

[0100] The technical solution of the embodiment of the present invention obtains the dataset required for blade state diagnosis. Each sample in the dataset includes at least the dynamic image data of the blade and the blade state, and performs supervised learning on the blade state diagnosis model. The dynamic image data of the blade to be tested is input into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be tested, so that the present application can diagnose various states of the fan blade and can quickly and accurately identify the state of the fan blade. In addition, by preprocessing the collected dynamic image data, the quality of the dataset used for model training is improved, and further the accuracy of fan blade state diagnosis is improved; by using the bilinear Transformer interaction network, while strengthening the discriminability of the features themselves, the common features of the sample images of the same type are obtained as the key points for distinguishing the differences between different category images, so that the solution of this embodiment can further improve the accuracy of the model in identifying the blade state.

[0101] Embodiment Two

[0102] Figure 6 It is a schematic structural diagram of a fan blade state diagnosis device provided in Embodiment Three of the present invention.

[0103] As Figure 6 shown, the device includes:

[0104] A data acquisition unit 601 is configured to acquire a data set required for diagnosing the state of a blade. Each sample in the data set includes at least dynamic image data of the blade and the state of the blade.

[0105] A training unit 602 is configured to train a pre-constructed blade state diagnosis model based on the data set. When the model parameters of the blade state diagnosis model converge, the trained blade state diagnosis model is obtained.

[0106] A diagnosis unit 603 is configured to input at least the dynamic image data of a to-be-diagnosed blade into the trained blade state diagnosis model to obtain a diagnosis result of the to-be-diagnosed blade.

[0107] In one embodiment, a preprocessing unit is further included. The preprocessing unit is configured to preprocess the acquired dynamic image data. The preprocessing may at least include denoising and analog-to-digital conversion.

[0108] The fan blade state diagnosis device provided by the embodiment of the present invention can execute the fan blade state diagnosis method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0109] Embodiment III

[0110] Figure 7 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0111] As Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0112] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for diagnosing the state of a wind turbine blade.

[0114] In some embodiments, a method for diagnosing the state of a wind turbine blade can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for diagnosing the state of a wind turbine blade described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute a method for diagnosing the state of a wind turbine blade in any other appropriate manner (e.g., by means of firmware).

[0115] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0116] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0117] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0120] The computing system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0121] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0122] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for diagnosing the status of a fan blade, characterized in that: include: Acquire a data set required for blade status diagnosis, wherein each sample in the data set includes at least dynamic image data of the blade and the blade status; The blade state diagnosis model constructed by training based on the data set is obtained when the model parameters of the blade state diagnosis model converge. At least the dynamic image data of the blade to be tested is input into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be tested.

2. The method for diagnosing the status of a fan blade according to claim 1, characterized in that: The blade state diagnosis model is a bilinear Transformer interaction network, which includes at least two feature extraction networks, a feature information interaction network, and a fusion module corresponding to the feature extraction network; The feature extraction network is used to extract key features of the input image; The feature information interaction network is used to perform self-interaction on the key features extracted by the feature extraction network, and to perform similar interaction on the key features extracted by at least two feature extraction networks; the input images of at least two feature extraction networks are similar images; The fusion module is used to fuse the features output from the self-interaction with the features of the same type of interaction.

3. The fan blade status diagnosis method according to claim 2, characterized in that: The feature extraction network obtains a weight matrix during fast operation based on a multi-head self-attention mechanism, and selects the feature corresponding to the maximum value in the weight matrix as the extracted image feature.

4. The method for diagnosing the status of a fan blade according to claim 2, characterized in that: The same type of interactions are: F1'=W 12 ×F1 <h2 style=";text-align:left;direction:ltr">F2' = W<h2 style=";text-align:left;direction:ltr"> 21 <h2 style=";text-align:left;direction:ltr"> ×F2 Among them, F1 and F2 are the features generated after two groups of similar images pass through the feature extraction network; F1' represents the feature obtained by the same type of interaction of feature F1; F2' represents the feature obtained by the same type of interaction of feature F2; W 12 and W 21 represents the weight matrix; The solution formula for the weight matrix is: F P and F q is the feature generated after two groups of similar images pass through the feature extraction network; W ij Represents the calculation of interaction weight, where W 12 By F p F q T Calculated, and W 21 By F q F p T Calculated; exp is an exponential function, i and j represent the channel indexes in the neural network, and k represents the number of heads of the attention mechanism; The fusion feature obtained by fusing the features output from the self-interaction with the features of the same type of interaction is: F final1 =cat(F1'+F1”) F final2 =cat(F2'+F2”) In the formula, F1” represents the feature obtained by self-interaction of feature F1, final1 represents the fusion feature obtained by fusing the feature F1" output from the self-interaction with the feature F1' of the same type of interaction; F2" represents the feature obtained by the self-interaction of feature F2, final2 It represents the fusion feature obtained by fusing the feature F2" output from the interaction with the feature F2' of the same type of interaction.

5. The method for diagnosing the status of a fan blade according to claim 1, characterized in that: The loss function of the blade status diagnosis model is a cross entropy loss, including: In the formula, x represents the sample, y represents the actual label, a is the output predicted by the model, and n represents the total number of samples.

6. The method for diagnosing the status of a fan blade according to claim 1, characterized in that: The sample also includes weather data; the blade status data at least includes intact, cracked, split, defective and lightning-struck.

7. The method for diagnosing the status of a fan blade according to claim 2, characterized in that: In the step of obtaining a data set required for blade status diagnosis, each sample in the data set includes at least dynamic image data of the blade and the blade status, and also includes: A high-speed photography system is used to collect the dynamic image data of the wind turbine blades when they are working, and the blade status of the dynamic image data is marked.

8. A fan blade status diagnosis device, characterized in that: include: A data acquisition unit, used for acquiring a data set required for blade status diagnosis, wherein each sample in the data set includes at least dynamic image data of the blade and the blade status; A training unit, used for training the constructed blade state diagnosis model based on the data set, and obtaining the trained blade state diagnosis model when the model parameters of the blade state diagnosis model converge; The diagnosis unit is used to input at least the dynamic image data of the blade to be tested into the trained blade state diagnosis model to obtain the diagnosis result of the blade to be tested.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the wind turbine blade status diagnosis method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the wind turbine blade status diagnosis method according to any one of claims 1 to 7 when executed.

Images