Fan deformation identification method, device and system
By acquiring the 3D point map and vibration signal map of the wind turbine, and enriching the model feature vector set with shape and appearance features, the problem of low accuracy in wind turbine deformation recognition is solved, and efficient deformation recognition is achieved.
Patent Information
- Application Number
- CN202411073080.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-06
AI Technical Summary
In existing technologies, there is a lack of deformation and vibration data for wind turbine towers and blades, making it difficult to train deep neural networks and resulting in low accuracy in identifying wind turbine deformation.
By acquiring the 3D point map and vibration signal map of the wind turbine, and inputting them into the pre-trained deformation recognition model, the model uses the shape and appearance features in the feature vector set for recognition, enriching the feature vector set of the prototype network model and improving the recognition accuracy.
It improves the accuracy of wind turbine deformation identification, reduces computing resources and data collection difficulty, and enhances the model's generalization ability.
Smart Images

Figure CN118982824A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image recognition technology, and in particular to a method, device and system for recognizing fan deformation. Background Art
[0002] In related technologies, in order to accurately detect the deformation of wind turbine towers and blades to ensure the safe operation of wind turbines, radar and other means are usually used to obtain deformation data and vibration data of wind turbine towers and blades. Based on the deformation data and vibration data, a deep neural network is used to determine whether deformation has occurred. However, due to the small amount of deformation data and vibration data of the above-mentioned wind turbine towers and blades in a deformed state, it is difficult to complete the preliminary training of the deep neural network, resulting in low accuracy in identifying wind turbine deformation using a neural network model. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, device and system for identifying wind turbine deformation.
[0004] According to a first aspect of an embodiment of the present disclosure, a method for identifying deformation of a wind turbine is provided, comprising:
[0005] Acquire a measured image of the fan; the types of the measured image include a three-dimensional point image and a vibration signal image;
[0006] Inputting the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model;
[0007] Obtaining the recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to a preset feature vector set; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images;
[0008] Determine whether the fan is deformed according to the identification result.
[0009] According to a second aspect of an embodiment of the present disclosure, a wind turbine deformation recognition device is provided, comprising:
[0010] A first acquisition unit is used to acquire a measured image of the wind turbine; the types of the measured image include a three-dimensional point image and a vibration signal image;
[0011] An input unit, used for inputting the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model;
[0012] A second acquisition unit is used to obtain a recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to a preset feature vector set; the feature vectors in the feature vector set are obtained based on shape features and appearance features of different original sample images;
[0013] A determination unit is used to determine whether the fan is deformed according to the identification result.
[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspects is implemented.
[0015] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method according to any one of the first aspects is implemented.
[0016] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method as described in any one of the first aspects when executed by a processor.
[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by obtaining measured images of the wind turbine; the types of measured images include three-dimensional dot graphs and vibration signal graphs; the three-dimensional dot graphs and vibration signal graphs are input into the pre-trained deformation recognition model; the deformation recognition model is obtained according to a preset feature vector set, and the output recognition result after the deformation state of the wind turbine is recognized based on the three-dimensional dot graph and the vibration signal graph; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of the different original sample images; and whether the wind turbine is deformed is determined according to the recognition result. More images of wind turbines with deformation are generated by using the shape features and appearance features in different images, thereby enriching the feature vector set of the prototype network model, thereby improving the accuracy of the prototype network model in recognizing the deformation state of the wind turbine.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0020] Figure 1 The figure is a flow chart of a method for identifying deformation of a wind turbine according to an exemplary embodiment.
[0021] Figure 2 The figure is a flow chart of another method for identifying deformation of a wind turbine according to an exemplary embodiment.
[0022] Figure 3 The figure is a block diagram of a wind turbine deformation identification device according to an exemplary embodiment.
[0023] Figure 4 It is a block diagram of a device for wind turbine deformation identification according to an exemplary embodiment. DETAILED DESCRIPTION
[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0025] The terms used in the disclosed embodiments are only for the purpose of describing specific embodiments, and are not intended to limit the disclosed embodiments. The singular forms "a", "an" and "the" used in the disclosed embodiments and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0026] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the disclosed embodiments, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the disclosed embodiments, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at" or "when" or "in response to determination".
[0027] In addition, various forms of processes shown in the embodiments of the present disclosure may be used to reorder, add or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and this document does not limit this.
[0028] In related technologies, in order to accurately detect the deformation of wind turbine towers and blades to ensure the safe operation of wind turbines, radar and other means are usually used to obtain deformation data and vibration data of wind turbine towers and blades. Based on the deformation data and vibration data, a deep neural network is used to determine whether deformation has occurred. However, due to the small amount of deformation data and vibration data of the above-mentioned wind turbine towers and blades in a deformed state, it is difficult to complete the preliminary training of the deep neural network, resulting in low accuracy in identifying wind turbine deformation using a neural network model.
[0029] One of the commonly used methods for wind turbine deformation detection is to use deep neural networks, but deep neural networks (DNNs) require a large amount of data, and the high cost of label collection remains a pain point. In addition, some types of data may be rare and difficult to collect, and deep neural networks do not learn well. FSL (Few-Shot Learning) solves the classification problem of limited test class labels. It is generally believed that when the support set contains only a small number of labeled samples as references, the FSL model must classify new class images. However, the trained FSL usually has poor generalization, and this generalization error mainly involves the generalization gap between training data and test data, which is aggravated by the unconventionality of test data samples. At present, various algorithms have been studied to address the problem of high generalization error, most of which involve training enhancement to improve the generalization ability of FSL models. However, this enhancement involves training the model on an increased amount of data, and additional computing resources are spent on each additionally trained FSL model. In addition, since the gap between test data and enhanced training data is still large, such enhancement means still have limited performance improvements.
[0030] In order to solve the above problems, the present disclosure provides a method, device and system for identifying deformation of a fan, which obtains measured images of the fan; the types of measured images include three-dimensional dot graphs and vibration signal graphs; the three-dimensional dot graphs and vibration signal graphs are input into a pre-trained deformation recognition model; the deformation recognition model obtains the recognition result output after identifying the deformation state of the fan based on the three-dimensional dot graph and the vibration signal graph according to a preset feature vector set; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images; and whether the fan is deformed is determined according to the recognition result. More images of fan with deformation are generated by using the shape features and appearance features in different images, thereby enriching the feature vector set of the prototype network model, thereby improving the accuracy of the prototype network model in identifying the deformation state of the fan.
[0031] Figure 1 is a flow chart of a method for identifying deformation of a wind turbine according to an exemplary embodiment. Figure 1As shown, it should be noted that the fan deformation recognition method of the embodiment of the present disclosure is applied to a fan deformation recognition device. Figure 1 As shown, the method may include the following steps:
[0032] Step 101, obtaining a measured image of the wind turbine.
[0033] Among them, the types of measured images include three-dimensional point graphs and vibration signal graphs.
[0034] In some embodiments of the present disclosure, the measured image may be any one or more of a wind turbine tower and a wind turbine blade.
[0035] In one embodiment, the three-dimensional point map may be obtained by scanning the wind turbine using a radar device.
[0036] Step 102: input the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model.
[0037] It can be understood that there is a correlation between the three-dimensional point diagram and the vibration signal diagram and the deformation state of the fan, so the deformation state of the fan can be determined based on the three-dimensional point diagram and the vibration signal diagram.
[0038] In one embodiment, after obtaining the overall deformation image (i.e., three-dimensional point diagram) and vibration signal of the wind turbine measured on site, a support data set and a test data set of the deformation recognition model can be established based on these data sets according to the requirements of few-sample learning. The established data set can change the number of data set types and the corresponding number of samples according to actual needs. For example, data can be randomly extracted based on the requirements and the number of samples that can actually be obtained to establish a 4-way 3-shot support data set (i.e., containing 4 different categories, each with 3 samples). The deformation recognition model is trained based on the support data set and the test data set.
[0039] In some embodiments of the present disclosure, the deformation recognition model may be a model that can directly obtain sample embedding representation information, that is, a model that can directly obtain sample feature vectors, for example, it may be a prototype network ProtoNet model or a matching network Matching Networks model.
[0040] Step 103 , obtaining a recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to a preset feature vector set.
[0041] The feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images.
[0042] It should be noted that, when the original sample image is a three-dimensional point graph, the above-mentioned shape features may include any one or more of contour edges and corner points, and the above-mentioned appearance features may include any one or more of texture and reflectivity; when the original sample image is a vibration signal graph, the above-mentioned shape features may include any one or more of waveform and spectrum features, and the above-mentioned appearance features may include any one or more of distribution and noise characteristics.
[0043] It is understandable that when the number of original samples is small, the number of samples needs to be increased to improve the accuracy of the deformation recognition model in identifying the deformation state of the fan. Therefore, the shape features and appearance features can be extracted from the original sample images respectively, and the shape features and appearance features of different original sample images can be fused to obtain more fan sample images with deformation. The fan sample images obtained based on the above fusion are used to enrich the feature vector set of the deformation recognition model, so that the deformation recognition model can more accurately identify the deformation state of the fan, thereby improving the generalization of the deformation recognition model without the need for more computing resources and data requirements.
[0044] Step 104: determine whether the fan is deformed according to the identification result.
[0045] In one embodiment, when the identification result is a deformation parameter of the fan, the deformation parameter is compared with a preset threshold value, and when the deformation parameter is greater than the preset threshold value, it is determined that the fan is deformed.
[0046] In some embodiments of the present disclosure, Figure 2 As shown in Figure 1, the feature vector set of the deformation recognition model is obtained by the following steps:
[0047] Step 201: Acquire an original sample image of a wind turbine.
[0048] The original sample image includes a first sample measured image and a second sample measured image.
[0049] It should be noted that the first sample measured image and the second sample measured image are of the same type. For example, the first sample measured image and the second sample measured image are both three-dimensional point images, or the first sample measured image and the second sample measured image are both vibration signal images, and the first sample measured image and the second sample measured image are both wind turbine blade images, or the first sample measured image and the second sample measured image are both wind turbine tower images, or the first sample measured image and the second sample measured image are both wind turbine overall images.
[0050] Step 202: Use a pre-trained data merging generator to extract shape features and appearance features of the first sample measured image and the second sample measured image.
[0051] In some embodiments of the present disclosure, the data merging generator can be trained by the following steps:
[0052] Step a1: input the original sample image of the wind turbine to the generator to be trained, and obtain a fused image output by the generator after fusion processing of the original sample image.
[0053] In one embodiment, the AAE (Adaptive Arithmetic Encoder) algorithm trains the data merging generator based on the train-test split data set. The method used by the generator is to extract different dimensional features (shape features and style features) of two images, and then merge the two dimensions from different images to obtain a new image, where image x provides the shape features of the generated image and image y provides the style features of the generated image.
[0054] Step a2, calculate the loss value between the fused image and the original sample image through the following comprehensive loss function:
[0055]
[0056] Among them, L total is the comprehensive loss function, D is the discriminator of GANs, G is the generator of GANs, k R and k FM is a hyperparameter, is the loss function of GANs, which is used to drive the competition between the generator and the discriminator. is the loss function of the shape feature extractor, which is used to calculate the loss value between the shape feature and its corresponding original sample image. is the loss function of the appearance feature extractor, which is used to determine the loss value between the appearance feature and its corresponding original sample image.
[0057] It should be noted that in the embodiment of the present disclosure, when determining the loss value between the fused image and the original sample image, not only the competition between the GANs algorithm generator and the discriminator is taken into account, that is, not only the loss function of GANs is calculated, but also the loss between the appearance features in the fused image and the corresponding original sample image, as well as the loss between the shape features and the corresponding original sample image are considered, which effectively improves the accuracy of the loss value and thereby improves the training efficiency of the data merging generator.
[0058] In some embodiments of the present disclosure, Calculated by the following formula:
[0059]
[0060]
[0061]
[0062] Among them, Φ e is the feature extractor, x is the original sample image corresponding to the extracted shape feature, and y is the original sample image corresponding to the extracted appearance feature.
[0063] Step a3, when the loss value is greater than the preset threshold, the generator is adjusted according to the loss value, and the step of inputting the original sample image of the wind turbine into the generator to be trained is returned to execute until the loss value is less than the preset threshold.
[0064] Step 203 , using a data merging generator, performs M fusion processing on the shape features and appearance features of different sample measured images to obtain M first wind turbine images.
[0065] Wherein, M is an integer greater than 1.
[0066] It can be understood that the data merging generator can obtain a different wind turbine image each time it performs fusion processing, and can obtain M first wind turbine images by performing M fusion processing.
[0067] Step 204 , selecting the first fan image with the smallest difference from the original sample image from the M first fan images to obtain a target fan image.
[0068] It can be understood that in order to ensure that the difference between the final generated sample and the original sample image is as small as possible, the first fan image with the smallest difference from the original sample image can be selected from the M first fan images to obtain the target fan image, which can be used for further fusion operation with the original sample image to obtain a fan image with a smaller difference from the original sample image.
[0069] In some embodiments of the present disclosure, step 204 may specifically include the following steps:
[0070] For each of the M first wind turbine images, the divergence score of the first wind turbine image is calculated by the following formula:
[0071]
[0072] in, is the divergence score, KL is the relative entropy, also known as Kullback-Leibler divergence, which is used to measure the difference between a probability distribution and another reference probability distribution, x is the original sample image to which the shape feature corresponding to the first wind turbine image belongs, y is the original sample image to which the appearance feature belongs, G is the generator of the generative adversarial network GANs, and h dis the function that computes the hidden representation of the generated image, Φ f is a feature extractor;
[0073] A first wind turbine image with the smallest divergence score among the M first wind turbine images is selected to obtain a target wind turbine image.
[0074] In one embodiment, a neighbor selector may be used to select samples with the smallest class difference (ie, lowest divergence score) between the generated and original sample images from the M first wind turbine images in the candidate sample pool (ie, support set) for subsequent generation.
[0075] In some other embodiments of the present disclosure, JS (Jensen-Shannon) divergence may also be used to calculate the above divergence score.
[0076] Step 205 , using a data merging generator to perform N fusion processes on the target wind turbine image and the original sample image to obtain N second wind turbine images.
[0077] Wherein, N is an integer greater than 1.
[0078] In one embodiment, a data merging generator is used to fuse the target wind turbine image having a relatively low degree of difference with the original sample image with the original sample image for N times to obtain N second wind turbine images.
[0079] It can be understood that the degree of similarity between the second wind turbine image obtained by the fusion processing in step 205 and the original sample image is lower than that of the target wind turbine image.
[0080] In some embodiments of the present disclosure, step 205 may specifically include the following steps:
[0081] Obtaining shape features of the target wind turbine image and appearance features in the original sample image;
[0082] The data merging generator is used to perform N fusion processing on the shape features of the target wind turbine image and the appearance features in the original sample image to obtain N second wind turbine images.
[0083] It is understandable that, since the deformation recognition model needs to be able to accurately identify the deformation state of the fan, the second fan image can be made more different from the original sample image in terms of shape features. That is, the shape features are enhanced in the newly generated second fan image, which improves the robustness of the second fan image and thus improves the performance of the prototype network model.
[0084] Step 206 : determining an average feature vector according to the N second wind turbine images, and storing the average feature vector in a feature vector set of the deformation recognition model.
[0085] It should be noted that since the deformation recognition model can be updated by storing the above-mentioned average feature vector in the feature vector set of the deformation recognition model, this solution can be used as a plug-and-play model optimization method and can be adapted to all trained few-sample learning models that can directly obtain sample embedding representation information (that is, feature vectors), and has good portability and scalability.
[0086] In some embodiments of the present disclosure, determining the average feature vector according to the N second wind turbine images may specifically include the following steps:
[0087] Generate feature vectors of N second wind turbine images respectively to obtain N feature vectors;
[0088] Calculate the average of N eigenvectors to get the average eigenvector.
[0089] It can be understood that by calculating the average of the N second fan image feature vectors to obtain the average feature vector, the deviation of the few-sample learning FSL model can be corrected, that is, the deviation between the second fan image and the expected fan image can be corrected, thereby reducing the misjudgment rate of the deformation recognition model.
[0090] According to the fan deformation recognition method proposed in the embodiment of the present disclosure, the measured image of the fan is obtained; the types of measured images include three-dimensional point graphs and vibration signal graphs; the three-dimensional point graphs and vibration signal graphs are input into the pre-trained deformation recognition model; the deformation recognition model is obtained according to a preset feature vector set, and the recognition result output after the deformation state of the fan is recognized based on the three-dimensional point graph and the vibration signal graph; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images; and whether the fan is deformed is determined according to the recognition result. More fan images with deformation are generated by using the shape features and appearance features in different images, thereby enriching the feature vector set of the prototype network model, thereby improving the accuracy of the prototype network model in recognizing the deformation state of the fan. In addition, since the above scheme only needs to operate on the original sample images in the test phase, no additional data set is required during training, which not only reduces the training calculation cost, but also reduces the difficulty of collecting sample library data.
[0091] Figure 3 FIG. 1 is a block diagram of a wind turbine deformation identification device according to an exemplary embodiment. Figure 3 The device includes a first acquisition unit 301, an input unit 302, a second acquisition unit 303 and a determination unit 304.
[0092] The first acquisition unit 301 is used to acquire a measured image of the wind turbine; the types of the measured image include a three-dimensional point image and a vibration signal image;
[0093] An input unit 302, used to input the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model;
[0094] The second acquisition unit 303 is used to obtain the recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to the preset feature vector set; the feature vector in the feature vector set is obtained based on the shape features and appearance features of different original sample images;
[0095] The determination unit 304 is used to determine whether the fan is deformed according to the recognition result.
[0096] In some embodiments of the present disclosure, the apparatus may further include a first training unit, which is used to train the deformation recognition model. The first training unit may be specifically used to:
[0097] Acquire an original sample image of the fan; the original sample image includes a first sample measured image and a second sample measured image;
[0098] A pre-trained data merging generator is used to extract shape features and appearance features of the first sample measured image and the second sample measured image;
[0099] The shape features and appearance features of different sample measured images are fused M times by using a data merging generator to obtain M first wind turbine images; wherein M is an integer greater than 1;
[0100] Selecting the first wind turbine image with the smallest difference from the original sample image from the M first wind turbine images to obtain a target wind turbine image;
[0101] The target wind turbine image and the original sample image are fused N times by using a data merging generator to obtain N second wind turbine images, wherein N is an integer greater than 1;
[0102] An average feature vector is determined according to the N second wind turbine images, and the average feature vector is stored in a feature vector set of the deformation recognition model.
[0103] In some embodiments of the present disclosure, the first training unit may also be used to:
[0104] For each of the M first wind turbine images, the divergence score of the first wind turbine image is calculated by the following formula:
[0105]
[0106] in, is the divergence score, KL is the relative entropy, x is the original sample image to which the shape feature corresponding to the first wind turbine image belongs, y is the original sample image to which the appearance feature belongs, G is the generator of the Generative Adversarial Network GANs, and h d is the function that computes the hidden representation of the generated image, Φ f is a feature extractor;
[0107] A first wind turbine image with the smallest divergence score among the M first wind turbine images is selected to obtain a target wind turbine image.
[0108] In some embodiments of the present disclosure, the first training unit may also be used to:
[0109] Obtaining shape features of the target wind turbine image and appearance features in the original sample image;
[0110] The shape features of the target wind turbine image and the appearance features in the original sample image are fused N times to obtain N second wind turbine images.
[0111] In some embodiments of the present disclosure, the first training unit may also be used to:
[0112] Generate feature vectors of N second wind turbine images respectively to obtain N feature vectors;
[0113] Calculate the average of N eigenvectors to get the average eigenvector.
[0114] In some embodiments of the present disclosure, the apparatus further includes a second training unit, which may be specifically configured to:
[0115] The original sample image of the wind turbine is input into the generator to be trained, and a fused image is output after the generator performs fusion processing on the original sample image;
[0116] The loss value between the fused image and the original sample image is calculated by the following comprehensive loss function:
[0117]
[0118]
[0119] Among them, L total is the comprehensive loss function, D is the discriminator of GANs, G is the generator of GANs, k E and k FM is a hyperparameter, is the loss function of GANs, is the loss function of the shape feature extractor, which is used to calculate the loss value between the shape feature and its corresponding original sample image. is the loss function of the appearance feature extractor, which is used to determine the loss value between the appearance feature and its corresponding original sample image;
[0120] When the loss value is greater than the preset threshold, the generator is adjusted according to the loss value, and the step of inputting the original sample image of the wind turbine into the generator to be trained is returned to execute until the loss value is less than the preset threshold.
[0121] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0122] According to the fan deformation recognition device proposed in the embodiment of the present disclosure, the measured image of the fan is obtained; the types of measured images include three-dimensional point graphs and vibration signal graphs; the three-dimensional point graphs and vibration signal graphs are input into the pre-trained deformation recognition model; the deformation recognition model is obtained according to a preset feature vector set, and the recognition result outputted after the deformation state of the fan is recognized based on the three-dimensional point graph and the vibration signal graph; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images; and whether the fan is deformed is determined according to the recognition result. More fan images with deformation are generated by using the shape features and appearance features in different images, thereby enriching the feature vector set of the prototype network model, thereby improving the accuracy of the prototype network model in recognizing the deformation state of the fan.
[0123] Figure 4 4 is a block diagram of a device for wind turbine deformation identification method according to an exemplary embodiment. For example, the device 400 may be an electronic device, such as a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0124] Reference Figure 4 , the device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .
[0125] The processing component 402 generally controls the overall operation of the device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.
[0126] The memory 404 is configured to store various types of data to support operations on the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0127] The power component 406 provides power to the various components of the device 400. The power component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 400.
[0128] The multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0129] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the device 400 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.
[0130] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0131] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the device 400. For example, the sensor assembly 414 can detect the open / closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor assembly 414 can also detect the position change of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration of the device 400, and the temperature change of the device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0132] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0133] In an exemplary embodiment, the apparatus 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.
[0134] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the instructions can be executed by a processor 420 of the device 400 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0135] In an exemplary embodiment, a computer program product is also provided, comprising a computer program, which implements the above method when executed by the processor 420 of the device 400 .
[0136] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0137] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for identifying fan deformation, characterized in that: include: Acquire a measured image of the fan; the types of the measured image include a three-dimensional point image and a vibration signal image; Inputting the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model; Obtaining the recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to a preset feature vector set; the feature vectors in the feature vector set are obtained based on the shape features and appearance features of different original sample images; Determine whether the fan is deformed according to the identification result.
2. The fan deformation identification method according to claim 1, characterized in that: The feature vector set of the deformation recognition model is obtained by the following steps: Acquire an original sample image of the wind turbine; the original sample image includes a first sample measured image and a second sample measured image; Using a pre-trained data merging generator to extract shape features and appearance features of the first sample measured image and the second sample measured image; The shape features and appearance features of different sample measured images are fused M times by using the data merging generator to obtain M first wind turbine images; wherein M is an integer greater than 1; Selecting a first fan image having the smallest difference from the original sample image from the M first fan images to obtain a target fan image; The data merging generator is used to perform N fusion processing on the target wind turbine image and the original sample image to obtain N second wind turbine images; wherein N is an integer greater than 1; An average feature vector is determined according to the N second wind turbine images, and the average feature vector is stored in a feature vector set of the deformation recognition model.
3. The fan deformation identification method according to claim 2, characterized in that: Selecting the first wind turbine image with the smallest difference from the original sample image from the M first wind turbine images to obtain a target wind turbine image includes: For each first wind turbine image in the M first wind turbine images, the divergence score of the first wind turbine image is calculated by the following formula: in, is the divergence score, KL is the relative entropy, x is the original sample image to which the shape feature corresponding to the first wind turbine image belongs, y is the original sample image to which the appearance feature belongs, G is the generator of the generative adversarial network GANs, and h d is the function that computes the hidden representation of the generated image, Φ f is a feature extractor; A first wind turbine image with the smallest divergence score among the M first wind turbine images is selected to obtain the target wind turbine image.
4. The fan deformation identification method according to claim 2, characterized in that: The step of using the data merging generator to perform N fusion processes on the target wind turbine image and the original sample image to obtain N second wind turbine images includes: Acquire shape features of the target wind turbine image and appearance features in the original sample image; The shape features of the target wind turbine image and the appearance features in the original sample image are fused N times to obtain the N second wind turbine images.
5. The fan deformation identification method according to claim 2, characterized in that: The determining the average feature vector according to the N second wind turbine images comprises: Generating feature vectors of the N second wind turbine images respectively to obtain N feature vectors; An average value of the N eigenvectors is calculated to obtain the average eigenvector.
6. The fan deformation identification method according to claim 2, characterized in that: The data merging generator is trained by the following steps: Inputting the original sample image of the wind turbine into the generator to be trained, and obtaining a fused image output by the generator after fusion processing of the original sample image; The loss value between the fused image and the original sample image is calculated by the following comprehensive loss function: Among them, L total is the comprehensive loss function, D is the discriminator of GANs, G is the generator of GANs, k R and k FM is a hyperparameter, is the loss function of GANs, is the loss function of the shape feature extractor, which is used to calculate the loss value between the shape feature and its corresponding original sample image. is the loss function of the appearance feature extractor, which is used to determine the loss value between the appearance feature and its corresponding original sample image; When the loss value is greater than a preset threshold, the generator is adjusted according to the loss value, and the step of inputting the original sample image of the wind turbine to the generator to be trained is returned to be executed until the loss value is less than the preset threshold.
7. A fan deformation recognition device, characterized in that: include: A first acquisition unit is used to acquire a measured image of the wind turbine; the types of the measured image include a three-dimensional point image and a vibration signal image; An input unit, used for inputting the three-dimensional point graph and the vibration signal graph into a pre-trained deformation recognition model; A second acquisition unit is used to obtain a recognition result output by the deformation recognition model after recognizing the deformation state of the fan based on the three-dimensional point graph and the vibration signal graph according to a preset feature vector set; the feature vectors in the feature vector set are obtained based on shape features and appearance features of different original sample images; A determination unit is used to determine whether the fan is deformed according to the identification result.
8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 6 when executed by a processor.
Citation Information
Patent Citations
Image processing method and device, computer equipment and readable storage medium
CN113705597A
Detection model training method and device and image detection method and device
CN114266308A
Multi-modal deep learning-based power generation equipment abnormity integrated identification method and equipment
CN114463594A
Fan blade defect detection method, system, equipment and device based on multiple modes
CN114549436A
Multi-branch deep learning-based 3D face reconstruction model training method and system, and medium
CN114926591A
