Blade icing identification method and device of wind generating set

Abstract

The invention discloses a blade icing identification method and device for a wind generating set. The method comprises the following steps: acquiring video data during operation of the wind generating set; acquiring a first image including a predetermined portion exceeding the leaf from the video data; based on the first images, a plurality of image sets are determined, and each image set comprises images which are continuous in time in all the first images; extracting a predetermined number of second images from any one image set in the plurality of image sets according to a predetermined interval; and the preset number of second images are input into the blade icing classification model to obtain an identification result, and the identification result is used for indicating that the blades of the wind generating set are iced or not iced.

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of wind power generation, and more specifically, to a method and device for identifying icing on the blades of a wind turbine generator. Background Art

[0002] At the current stage, the utilization rate of wind power generation resources is relatively high, and the technology is relatively mature. With the development of technology, the cost is also decreasing, so the installed capacity has increased significantly, and wind energy has gradually become one of the indispensable energy sources in China. However, wind turbine generators are very sensitive to changes in meteorological conditions and have a certain degree of instability. For example, in areas with humid climates in winter and spring, icing on the fan blades becomes the main factor affecting the normal operation and power generation of wind farms. After the blades are iced, the aerodynamic characteristics of the blades change, and the dynamic load of the wind turbine generator increases, which will have a negative impact on the safe operation and efficiency of the wind farm. Affected by different climate environments and terrain conditions, the ice formed on the fan blades during operation has different textures and shapes, and there are significant differences in the icing processes and characteristics of the fan blades in different regions. Accordingly, the de-icing technical measures are also different.

[0003] Regarding the identification of blade icing, at present, in addition to many traditional technical means such as power monitoring, sensors, and acoustic wave detection, many algorithms based on machine learning or deep learning in the field of computer vision have emerged. However, the vast majority of current icing identification algorithms are based on single images. Since the characteristics of the icing part changing over time are lost during the processing, it is difficult to exclude the interference of the surrounding environment, thus reducing the accuracy of the identification. Summary of the Invention

[0004] Embodiments of the present disclosure provide a method and device for identifying icing on the blades of a wind turbine generator, which can effectively solve the problem of low accuracy of identification in the prior art.

[0005] In one general aspect, a method for identifying icing on blades is provided, including: obtaining video data during the operation of a wind turbine generator; obtaining a first image including a predetermined part exceeding the blade from the video data; based on the first image, determining a plurality of image sets, where each image set includes images that are temporally continuous in all the first images; extracting a predetermined number of second images from any one of the plurality of image sets at a predetermined interval; inputting the predetermined number of second images into a blade icing classification model to obtain an identification result, where the identification result is used to indicate whether the blades of the wind turbine generator are iced or not.

[0006] Optionally, input a predetermined number of second images into the blade icing classification model to obtain an identification result, including: removing information outside the blade area from each second image and retaining the information of the blade area in each second image, so as to obtain a predetermined number of third images; removing information outside the blade icing area from each second image and retaining the information of the blade icing area in each second image, so as to obtain a predetermined number of fourth images; fusing the third image and the fourth image corresponding to the same second image to obtain a predetermined number of fusion vectors; inputting the predetermined number of fusion vectors into the blade icing classification model to obtain an identification result.

[0007] Optionally, fusing the third image and the fourth image corresponding to the same second image to obtain a predetermined number of fusion vectors, including: performing downsampling processing and compression on each third image respectively to obtain a first vector of each third image; performing downsampling processing and compression on each fourth image respectively to obtain a second vector of each fourth image; combining the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors.

[0008] Optionally, combining the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors, including: obtaining predetermined information corresponding to each second image, where the predetermined information includes blade icing information in the corresponding second image; adding the predetermined information to the combination result of the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors.

[0009] Optionally, the predetermined information corresponding to each second image is determined in the following manner: in the case where there is no blade icing area in the second image, the predetermined information of the second image is a first preset vector only including the same preset value; in the case where there is a blade icing area in the second image, the predetermined information of the second image is a second preset vector including multiple values, where the multiple values are determined based on one or more of the coordinates of the blade icing area in the second image, the first direction metric, the second direction metric, and the size of the second image.

[0010] Optionally, in the case where the first preset vector and the second preset vector are one-dimensional vectors including 5 columns, the first preset vector only includes the preset value; the second preset vector includes a first value, a second value, a third value, a fourth value, and an icing confidence level, where the first value and the second value are determined based on the coordinates of the blade icing area in the second image and the size of the second image, and the third value and the fourth value are determined based on the first direction metric and the second direction metric of the blade icing area in the second image and the size of the second image respectively.

[0011] Optionally, when the recognition result indicates icing on the blades of the wind turbine generator, determine the icing area and the blade area of a predetermined number of second images; when the icing area is less than a first predetermined ratio of the blade area, determine that the wind turbine generator is in a first icing state; when the icing area is greater than the first predetermined ratio of the blade area and less than a second predetermined ratio of the blade area, determine that the wind turbine generator is in a second icing state, and reduce the load of the wind turbine generator by controlling the pitch angle of the wind turbine generator; when the icing area is greater than the second predetermined ratio of the blade area, determine that the wind turbine generator is in a third icing state, and control the wind turbine generator to stop.

[0012] Optionally, after controlling the wind turbine generator to stop, monitor the ambient temperature of the wind turbine generator; when the ambient temperature is higher than a preset temperature and lasts for a preset duration, determine whether the icing condition of the wind turbine generator meets a predetermined condition; in response to the icing condition of the wind turbine generator meeting the predetermined condition, restart the wind turbine generator, where the predetermined condition includes at least one of the following: determining that the blades of the wind turbine generator are not iced through a blade icing classification model; determining that the blades of the wind turbine generator are iced and the icing area of all predetermined images is less than the second predetermined ratio of the blade area through a blade icing classification model, where the predetermined images are the images used to determine whether the blades of the wind turbine generator are iced this time.

[0013] Optionally, in the case of multiple wind turbine generators, when the recognition result indicates icing on the blades of a wind turbine generator, determine the wind turbine generator with icing blades among the multiple wind turbine generators through the positioning information included in a predetermined number of second images.

[0014] Optionally, remove the information outside the blade area from each second image and retain the information of the blade area in each second image, so as to obtain a predetermined number of third images, including: respectively segmenting each second image through a pre-trained blade segmentation model to obtain a first mask of each second image; respectively segmenting each second image through a pre-trained blade icing segmentation model to obtain a second mask of each second image; multiplying each second image by the corresponding first mask and second mask respectively to obtain a predetermined number of third images and fourth images.

[0015] Optionally, the blade segmentation model is trained in the following manner: obtain a first training dataset, where the first training dataset includes multiple images containing a predetermined part exceeding the blade and the identification of the blade area in each image; train an initial blade segmentation model through the multiple images containing a predetermined part exceeding the blade and the identification of the blade area in each image to obtain the blade segmentation model.

[0016] Optionally, the blade icing segmentation model is trained as follows: Obtain a second training dataset, where the second training dataset includes a plurality of images with blade icing regions and the identification of the blade icing regions in each image; Train an initial blade icing segmentation model with the plurality of images with blade icing regions and the identification of the blade icing regions in each image to obtain the blade icing segmentation model.

[0017] Optionally, the blade icing classification model is trained as follows: Obtain a third training dataset, where the third training dataset includes a plurality of sequence data and the identification of each sequence data, the sequence data includes information of a predetermined number of second images, and each identification is used to indicate whether the blade of the wind turbine generator is iced or not; Train an initial blade icing classification model with the plurality of sequence data and the identification of each sequence data to obtain the blade icing classification model.

[0018] In another general aspect, there is provided a computer-readable storage medium storing instructions, where when the instructions are run by at least one computing device, the at least one computing device is caused to execute any one of the above-mentioned blade icing recognition methods of a wind turbine generator.

[0019] In another general aspect, there is provided a system including at least one computing device and at least one storage device storing instructions, where when the instructions are run by at least one computing device, the at least one computing device is caused to execute any one of the above-mentioned blade icing recognition methods of a wind turbine generator.

[0020] According to the blade icing recognition method and device of a wind turbine generator according to an embodiment of the present disclosure, a first image including more than a predetermined part of the blade can be obtained from the video data during the operation of the wind turbine generator, which can exclude the interference of images without sufficient blade information. Moreover, by extracting second images from the temporally continuous first images, the characteristics of the icing part changing with time can be accurately reflected. Then, through multiple second images including sufficient blade information and in combination with the blade icing classification model, it can be identified whether the blade is iced. In this process, not only the information of a single image is considered, but also the characteristics of the icing part changing with time are considered, reducing the possibility of misjudgment of some similar characteristics such as reflection and snow accumulation when only relying on a single image for blade icing recognition, so as to achieve an accurate judgment of blade icing. Thus, through the embodiments of the present disclosure, the blade icing situation can be timely and accurately detected, various risks caused by blade icing can be effectively avoided, economic losses can be reduced, and the operation stability and safety of the wind turbine can be improved. Therefore, through the present disclosure, the problem of low accuracy of existing technology recognition can be effectively solved.

[0021] Additional aspects and / or advantages of the present disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects and features of the embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the drawings showing embodiments thereof, in which:

[0023] Figure 1 is a flowchart showing a method for identifying icing on a blade of a wind turbine according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram showing blade identification according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram showing frame extraction of a blade according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram showing an initial blade segmentation model according to an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram showing a wind turbine blade photographed by a pan-tilt according to an embodiment of the present disclosure;

[0028] Figure 6 is a schematic diagram showing the segmentation effect of a blade segmentation model according to an embodiment of the present disclosure;

[0029] Figure 7 is a schematic diagram showing training of an initial blade segmentation model and an initial blade icing segmentation model according to an embodiment of the present disclosure;

[0030] Figure 8 is a schematic diagram showing the segmentation effect of a blade icing segmentation model according to an embodiment of the present disclosure;

[0031] Figure 9 is a schematic diagram showing icing identification using a blade icing classification model according to an embodiment of the present disclosure;

[0032] Figure 10 is a block diagram showing a device for identifying icing on a blade of a wind turbine according to the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The following specific embodiments are provided to assist the reader in obtaining a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will be apparent. For example, the order of operations described herein is merely exemplary and is not limited to those set forth herein, but may be changed as will be apparent after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, descriptions of features known in the art may be omitted for greater clarity and conciseness.

[0034] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Instead, the examples described herein are provided only to illustrate some of the many possible ways of implementing the methods, apparatuses, and / or systems described herein, which will be apparent after understanding the disclosure of the present application.

[0035] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more of them.

[0036] Although terms such as "first", "second", and "third" may be used herein to describe various components, elements, regions, layers, or parts, these components, elements, regions, layers, or parts should not be limited by these terms. Instead, these terms are only used to distinguish one component, element, region, layer, or part from another. Thus, a first component, first element, first region, first layer, or first part referred to in the examples described herein may also be referred to as a second component, second element, second region, second layer, or second part without departing from the teachings of the examples.

[0037] In the specification, when an element (such as a layer, region, or substrate) is described as "on", "connected to", or "coupled to" another element, the element may be directly "on", directly "connected to", or "coupled to" the other element, or there may be one or more other elements therebetween. In contrast, when an element is described as "directly on", "directly connected to", or "directly coupled to" another element, there may be no other elements therebetween.

[0038] The terms used herein are only for describing various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. The terms "comprising", "including", and "having" specify the presence of the recited features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains after understanding this disclosure. Unless explicitly defined herein, terms (such as those defined in a general dictionary) shall be interpreted to have a meaning consistent with their meaning in the context of the relevant art and this disclosure, and shall not be interpreted in an idealized or overly formal manner.

[0040] In addition, in the description of the examples, when it is considered that a detailed description of related structures or functions that are well-known will cause an ambiguous interpretation of this disclosure, such detailed descriptions will be omitted.

[0041] To solve the problem that in the prior art, the time factor cannot be introduced, and when predicting through a single image, it is greatly affected by environmental factors and is prone to losing a large number of characteristics of the icing area changing over time, this disclosure provides a method and device for identifying icing on the blades of a wind turbine, which can solve the above problems. The method for identifying icing on the blades of the wind turbine in this disclosure can be applied to a server, or to the controller of a single wind turbine, or to the general controller of a wind farm. This disclosure does not limit this. The server, the controller, and the wind turbine can be connected wirelessly or wiredly, and this is not limited here. The above server can be a single server, or a server cluster composed of several servers, or a cloud computing platform or a virtualization center. Hereinafter, the server will be used as an example for illustration.

[0042] The server acquires video data during the operation of the wind turbine, obtains a first image from the video data that includes a predetermined portion exceeding the blade, and then, based on the first image, determines a plurality of image sets, where each image set includes images that are temporally continuous among all the first images. Then, a predetermined number of second images are extracted from any one of the plurality of image sets at a predetermined interval, and then the predetermined number of second images are input into the blade icing classification model to obtain an identification result, where the identification result is used to indicate whether the blades of the wind turbine are iced or not.

[0043] The following will describe in detail the method and device for identifying icing on the blades of the wind turbine in this disclosure with reference to the accompanying drawings.

[0044] This disclosure proposes a method for identifying icing on the blades of a wind turbine, Figure 1 which is a flowchart showing the method for identifying icing on the blades of the wind turbine according to an embodiment of this disclosure. Referring to Figure 1 , the method for identifying icing on the blades of the wind turbine includes the following steps:

[0045] In step S101, video data during the operation of the wind turbine is acquired.

[0046] As an example, video data during the operation of a wind turbine can be captured by a pan-tilt camera with a fixed angle mounted on the wind turbine, or video data during the operation of the wind turbine can be captured by other cameras, such as manually holding a camera to capture video data during the operation of the wind turbine. The present disclosure does not limit this.

[0047] In step S102, a first image including a predetermined portion of the blade exceeding the blade is obtained from the video data.

[0048] Specifically, the video data obtained in step S101 is the video data during the operation of the wind turbine. When the wind turbine is operating, the blades are constantly rotating. Therefore, there may be images in the video data captured by the camera that do not include the blades. In order to reduce the amount of calculation and exclude the interference of blade-less images during subsequent model training and implementation of inference, images including the blades can be extracted from the video data. It should be noted that if the portion of the blade included in the image is relatively small, it is not very meaningful to use. Therefore, the first image extracted in this embodiment is an image including a predetermined portion of the blade exceeding the blade, which can further reduce the amount of calculation. Among them, the predetermined portion of the blade can be a portion of a predetermined ratio of the blade or a portion of a predetermined range of the blade. The present disclosure does not limit this.

[0049] As an example, image recognition can be performed on each frame of the video data by a pre-trained blade segmentation model to obtain all the first images in the video data. By using the pre-trained blade segmentation model, the first images including the blades can be conveniently and accurately recognized.

[0050] Specifically, as Figure 2 shown, after obtaining the video data captured by the pan-tilt camera, the pre-trained blade segmentation model is used to perform frame-by-frame recognition on the video data to obtain all the frames including the blades, or all the frames including a predetermined ratio portion of the complete blade exceeding the blade, that is, the first images. The present disclosure does not limit this.

[0051] In step S103, based on the first image, a plurality of image sets are determined, where each image set includes images that are temporally continuous among all the first images.

[0052] Specifically, after the first image is extracted from the video data, the obtained first image must present the following form, that is, the time of a part of the first images is continuous, and the time of a part of the first images is discontinuous. Therefore, the first images with continuous time in each segment can be grouped into an image set, and thus a plurality of image sets can be obtained, that is, each image set includes a segment of continuous first images.

[0053] In step S104, a predetermined number of second images are extracted from any one of the multiple image sets at a predetermined interval.

[0054] Specifically, after obtaining the multiple image sets, the second images can be extracted based on any one of the image sets, which is equivalent to extracting the second images from the first images with continuous time, so that the characteristics of the icing part changing with time can be reflected more accurately. It should be noted that any of the above image sets can be the image set with the time closest to the current time, or an image set arbitrarily selected by the user from the multiple image sets, and the present disclosure does not limit this.

[0055] As an example, as Figure 3 shown, from a continuous frame containing the blade, that is, from any one of the image sets, 10 frames are extracted at a fixed time interval t (for example, if the blade appears in a period of 91 frames, then the 1st, 11th, 21st, 31st,... 81st, 91st frames are extracted), and 10 second images are obtained, so that the calculation amount can be further reduced. The above fixed time interval can be set as needed, and the present disclosure does not limit this.

[0056] In step S105, a predetermined number of second images are input into the blade icing classification model to obtain an identification result, where the identification result is used to indicate whether the blade of the wind turbine is iced or not.

[0057] According to an embodiment of the present disclosure, inputting a predetermined number of second images into the blade icing classification model in step S105 to obtain an identification result may include: removing the information outside the blade area from each second image and retaining the information of the blade area in each second image, so as to obtain a predetermined number of third images; removing the information outside the blade icing area from each second image and retaining the information of the blade icing area in each second image, so as to obtain a predetermined number of fourth images; fusing the third image and the fourth image corresponding to the same second image to obtain a predetermined number of fusion vectors; inputting the predetermined number of fusion vectors into the blade icing classification model to obtain an identification result.

[0058] Through this embodiment, before inputting into the blade icing classification model, the blade information and icing information in the second image can be extracted and fused, and then a fusion vector is obtained. The fusion vector is used as the input of the blade icing classification model, so that the input of the blade icing classification model can contain more blade information and icing information. Moreover, after removing the information outside the blade area and the blade icing area, unnecessary information is further reduced, so that the accuracy of identification can be improved while the calculation amount of the model is reduced.

[0059] According to an embodiment of the present disclosure, in the above embodiment, removing information other than the leaf area from each second image and retaining the information of the leaf area in each second image to obtain a predetermined number of third images may include: segmenting each second image respectively by a pre-trained leaf segmentation model to obtain a first mask of each second image; multiplying each second image by the corresponding first mask respectively to obtain a predetermined number of third images. Through this embodiment, by using the pre-trained leaf segmentation model, the mask of the corresponding image can be obtained conveniently and quickly, and then the third image that only retains the information of the leaf area in the second image can be obtained.

[0060] As an example, still taking Figure 3 as an example, after extracting 10 frames from any image set at a fixed time interval t, the leaf segmentation model can be used to segment these 10 second images respectively to obtain the corresponding mask of each second image, which can also be called a mask image. This mask image is a binary image that labels the leaf area in the second image. At this time, the 10 second images can be grayscale-converted to obtain the corresponding grayscale images, and then multiplied by the corresponding mask images. This step will set the grayscale values of the non-leaf areas in each second image to 0, that is, an image that retains the grayscale values of the leaf area is obtained.

[0061] According to an embodiment of the present disclosure, in the above embodiment, removing information other than the leaf icing area from each second image and retaining the information of the leaf icing area in each second image to obtain a predetermined number of fourth images may include: segmenting each second image respectively by a pre-trained leaf icing segmentation model to obtain a second mask of each second image; multiplying each second image by the corresponding second mask respectively to obtain a predetermined number of fourth images. Through this embodiment, by using the pre-trained leaf icing segmentation model, the mask of the corresponding image can be obtained conveniently and quickly, and then the fourth image that only retains the information of the icing area in the second image can be obtained.

[0062] As an example, still taking Figure 3 as an example, after extracting 10 frames from any image set at a fixed time interval t, the leaf icing segmentation model can be used to segment these 10 second images respectively to obtain the corresponding mask of each second image, which can also be called a mask image. This mask image is a binary image that labels the icing area in the second image. At this time, the 10 second images can be grayscale-converted to obtain the corresponding grayscale images, and then multiplied by the corresponding mask images. This step will set the grayscale values of the non-icing areas in each second image to 0, that is, an image that retains the grayscale values of the leaf icing area is obtained.

[0063] According to an embodiment of the present disclosure, fusing the third image and the fourth image corresponding to the same second image to obtain a predetermined number of fusion vectors may include: performing downsampling processing and compression on each third image respectively to obtain a first vector of each third image; performing downsampling processing and compression on each fourth image respectively to obtain a second vector of each fourth image; combining the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors. Through this embodiment, by performing downsampling and compression on the images and then fusing them, the difficulty of fusing the third image and the fourth image can be simplified.

[0064] The above-mentioned third image and fourth image are obtained by multiplying the grayscale image of the second image with the first mask and the second mask respectively. Therefore, the third image and the fourth image are also grayscale images, that is, both are grayscale pictures.

[0065] As an example, after obtaining the third image and the fourth image, the obtained images can be downsampled to the required dimension, such as a dimension of 20*20 or a dimension of 60*60. The present disclosure does not limit this. Assuming that the downsampling is performed to a dimension of 20*20, then, the downsampled result is compressed into a one-dimensional vector, and the compressed one-dimensional vector is normalized (that is, the grayscale value is divided by 255), and a one-dimensional vector with a length of 400 corresponding to each image can be obtained. Then, the one-dimensional vectors of the third image and the fourth image corresponding to the same second image are combined to obtain the fusion vector of each second image.

[0066] It should be noted that the above compression form can adopt any one of the following two methods: assuming there is a square matrix [1, 2; 3, 4], the compressed one-dimensional vector can be [1, 2, 3, 4], or it can be [1, 3, 2, 4]. The above combination processing can be addition processing or other processing methods. The present disclosure does not limit this.

[0067] According to an embodiment of the present disclosure, combining the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors may include: obtaining predetermined information corresponding to each second image, where the predetermined information includes the ice formation information of the blades in the corresponding second image; adding the predetermined information to the combination processing result of the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors. Through this embodiment, after combining a vector and the second vector, predetermined information including the ice formation information of the blades in the corresponding second image can also be added, so that the ice formation information contained in the fusion vector, which is the input of the ice formation classification model of the blades, is richer, and the recognition accuracy is further improved.

[0068] As an example, taking the merging process as an addition process, the one-dimensional vectors of the third image and the fourth image corresponding to the same second image are added to obtain a vector after the addition process. At this time, the sampling parameters and compression method in the above embodiment are still used, and the vector after the addition process is still a one-dimensional vector with a length of 400. At this time, the ice formation information of the blades in the second image can be further obtained, and the ice formation information of the blades is added to the vector after the addition process, so as to enrich the ice formation information of the blades included in the fusion vector.

[0069] According to an embodiment of the present disclosure, the predetermined information corresponding to each second image is determined as follows: in the case where there is no ice formation area of the blades in the second image, the predetermined information of the second image is a first predetermined vector only including the same preset value; in the case where there is an ice formation area of the blades in the second image, the predetermined information of the second image is a second predetermined vector including a plurality of values, where the plurality of values are determined based on one or more of the coordinates of the ice formation area of the blades in the second image, the first direction measurement, the second direction measurement, and the size of the second image. Through this embodiment, the predetermined information is determined in different cases, and relatively accurate predetermined information can be given.

[0070] Specifically, whether there is an ice formation area of the blades in the second image can be determined by an ice formation segmentation model, that is, the output of the ice formation segmentation model includes a mask of the second image, and whether there is an ice formation area of the blades in the second image can be known according to the mask. In addition, when training the ice formation segmentation model, it can be ensured that the output of the trained ice formation segmentation model not only includes the mask of the second image, but also includes the information on whether there is an ice formation area of the blades. The present disclosure does not limit this.

[0071] The preset value in the above first predetermined vector can be set as needed, such as 0 or 1, etc., and the present disclosure does not limit this. The dimension of the above second predetermined vector is the same as that of the first predetermined vector, that is, the number of preset values in the first predetermined vector is the same as the number of values in the second predetermined vector.

[0072] The coordinates of the ice - covered area of the blade in the second image, the first direction measurement, and the second direction measurement can adopt corresponding parameters in different coordinate systems. For example, when the coordinate system is a rectangular coordinate system, the coordinates of the ice - covered area of the blade in the second image can be the upper - left - corner coordinates, the first direction measurement can be the width, and the second direction measurement can be the height. Moreover, the upper - left - corner coordinates, width, and height can also be determined by the blade ice - covered segmentation model, that is, the output of the blade ice - covered segmentation model includes the mask of the second image. According to this mask, the upper - left - corner coordinates, width, and height of the ice - covered area of the blade in the second image can be known. Additionally, when training the blade ice - covered segmentation model, it can be ensured that the output of the trained blade ice - covered segmentation model not only includes the mask of the second image but also the upper - left - corner coordinates, width, and height of the ice - covered area of the blade. The present disclosure does not limit this. Correspondingly, the size of the second image can include the total width and total height of the second image, and the present disclosure does not limit this.

[0073] In the case of obtaining the corresponding predetermined information by using the model, it is equivalent to obtaining the characteristics of the ice - covered part considered by the model and incorporating them into the vector after the addition process, which is equivalent to introducing richer blade ice - covered information.

[0074] According to an embodiment of the present disclosure, when the first preset vector and the second preset vector are one - dimensional vectors with 5 columns, the first preset vector only includes preset values; the second preset vector includes the first value, the second value, the third value, the fourth value, and the ice - covered confidence level. Among them, the first value and the second value are determined based on the coordinates of the ice - covered area of the blade in the second image and the size of the second image, and the third value and the fourth value are respectively determined based on the first direction measurement, the second direction measurement of the ice - covered area of the blade in the second image, and the size of the second image. Through this embodiment, the first preset vector can be determined relatively conveniently, and at the same time, the values in the second preset vector can also be conveniently determined.

[0075] Specifically, the above - mentioned first preset vector and second preset vector are not limited to including 5 columns, and can also include 4 columns or 6 columns, and the present disclosure does not limit this. The first value can be determined based on the abscissa of the upper - left corner of the ice - covered area of the blade in the second image and the total width of the second image, the second value can be determined based on the ordinate of the upper - left corner of the ice - covered area of the blade in the second image and the total height of the second image, the third value can be determined based on the width of the ice - covered area of the blade in the second image and the total width of the second image, and the fourth value can be determined based on the height of the ice - covered area of the blade in the second image and the total height of the second image.

[0076] As an example, assuming that the preset value is 0, the first preset vector can be [0, 0, 0, 0, 0], and the second preset vector can be [x, y, w, h, icing confidence], where x can be determined based on the abscissa of the upper left corner of the blade icing area in the second image and the total width of the second image, for example, it can be the ratio of the abscissa of the upper left corner to the total width of the second image. y can be determined based on the ordinate of the upper left corner of the blade icing area in the second image and the total height of the second image, for example, it can be the ratio of the ordinate of the upper left corner to the total height of the second image. w can be determined based on the width of the blade icing area in the second image and the total width of the second image, for example, it can be the ratio of the width of the blade icing area to the total width of the second image. h can be determined based on the height of the blade icing area in the second image and the total height of the second image, for example, it can be the ratio of the height of the blade icing area to the total height of the second image.

[0077] As an example, after obtaining the ratios corresponding to the 5 values, these 5 ratios can be normalized to be used as the final x, y, w, h, and icing confidence, because the dimension gap can be eliminated and the recognition error can be reduced after normalization.

[0078] As an example, in the above embodiment, the one-dimensional vectors of the third image and the fourth image corresponding to the same second image are added. After obtaining the added vector, if the added vector is still a one-dimensional vector with a length of 400, at this time, after adding the above-mentioned predetermined information to the added vector, they together form a one-dimensional vector with a length of 405, that is, the fused vector after adding the predetermined information.

[0079] It should be noted that the above icing confidence can be determined by the blade icing segmentation model, that is, when training the blade icing segmentation model, ensure that the output of the trained blade icing segmentation model not only includes the mask of the second image, but also includes the icing confidence of the second image. The present disclosure does not limit this.

[0080] According to the embodiments of the present disclosure, the blade segmentation model is trained in the following manner: obtaining a first training dataset, where the first training dataset includes multiple images containing a predetermined part exceeding the blade and the identification of the blade area in each image; training the initial blade segmentation model through multiple images containing a predetermined part exceeding the blade and the identification of the blade area in each image to obtain the blade segmentation model. Through this embodiment, the initial blade segmentation model can be effectively trained, so as to obtain a blade segmentation model with excellent segmentation effect.

[0081] As an example, first, images containing more than a predetermined portion of the blade can be collected, and the blade regions in these images can be labeled using annotation software (such as labelme), or manually labeled, to obtain the identification of the blade regions in each image. A dataset for training the blade segmentation model, i.e., the above-mentioned first training dataset, can be obtained through the images and the corresponding identifications. Then, using the above-mentioned images and the identifications of the corresponding blade regions, the initial blade segmentation model is trained to obtain the blade segmentation model. The trained blade segmentation model can be used to judge each input image to determine whether the input image contains a blade, that is, to determine whether the input image contains a blade according to the output estimated recognition result. Alternatively, when training the initial blade segmentation model, the trained blade segmentation model can be made to directly output information on whether the image contains a blade, and the present disclosure does not limit this.

[0082] As an example, the above-mentioned initial blade segmentation model can adopt the YOLOv8-seg algorithm, such as Figure 4 shown. After training the YOLOv8-seg algorithm with the above-mentioned first training dataset, a blade segmentation model can be obtained. Assuming that the fan blade captured by the pan-tilt is as Figure 5 shown, then the segmentation effect of this blade segmentation model is as Figure 6 shown. It should be noted that the present disclosure does not limit the algorithm adopted by the initial blade segmentation model, and it can also be other image segmentation algorithms, such as MaskR-CNN, SOLO model, etc.

[0083] It should be noted that YOLO (You Only Look Once) is an efficient object detection algorithm. It first proposed a real-time end-to-end object detection method. Compared with previous methods, YOLO can complete the detection task only through one network. It only predicts the detection output based on regression, greatly improving the detection speed. And YOLOv8 is the latest YOLO-based object detection model series from Ultralytics, providing state-of-the-art performance. Compared with previous versions of YOLO, the YOLOv8 model is faster and more accurate, and at the same time provides a unified framework for training models to perform tasks such as object detection, instance segmentation, and image classification models.

[0084] As an example, the process of training the initial blade segmentation model using the images and the identifications of the corresponding blade regions in the above-mentioned embodiment can be as Figure 7As shown, that is, after collecting images containing a predetermined proportion of parts exceeding the complete leaves and performing annotation, the above-mentioned images are preprocessed, such as operations like data augmentation and resizing. Then, the preprocessed images are respectively input into the initial leaf segmentation model to obtain the estimated identification results. Based on the loss between the estimated recognition results and the pre-obtained annotations, the parameters of the initial leaf segmentation model are adjusted until the training conditions are met, thereby obtaining the final parameters in the leaf segmentation model, that is, the weight file.

[0085] According to an embodiment of the present disclosure, the leaf icing segmentation model is trained in the following manner: obtaining a second training dataset, wherein the second training dataset includes a plurality of images with leaf icing regions and the identification of the leaf icing regions in each image; training the initial leaf icing segmentation model through the plurality of images with leaf icing regions and the identification of the leaf icing regions in each image to obtain the leaf icing segmentation model. Through this embodiment, the initial leaf icing segmentation model can be effectively trained, thereby obtaining a leaf icing segmentation model with excellent segmentation effect.

[0086] As an example, first, images containing icing features can be collected, and annotation software (such as labelme) can be used to annotate the leaf icing regions in these images, or the leaf icing regions in these images can be manually annotated to obtain the identification of the leaf icing regions in each image. The dataset for training the leaf icing segmentation model is obtained through the images and the corresponding identifications, that is, the above-mentioned second training dataset. Then, the initial leaf icing segmentation model is trained using the above-mentioned images and the corresponding identifications of the leaf icing regions to obtain the leaf icing segmentation model. The trained leaf icing segmentation model can be used to judge each input image to determine whether the input image contains a leaf icing region, that is, to determine whether the input image contains a leaf icing region according to the output estimated recognition result. Or, when training the initial leaf icing segmentation model, the trained leaf icing segmentation model can directly output information on whether the image contains a leaf icing region, and the present disclosure does not limit this.

[0087] As an example, the above-mentioned initial leaf icing segmentation model can also adopt the YOLOv8-seg algorithm, such as Figure 4 shown. After training the YOLOv8-seg algorithm through the above-mentioned second training dataset, a leaf icing segmentation model can be obtained, and the segmentation effect of this leaf icing segmentation model is as Figure 8 shown. It should be noted that the present disclosure does not limit the algorithm adopted by the initial leaf icing segmentation model, and it can also be other image segmentation algorithms, such as Mask R-CNN, SOLO model, etc.

[0088] As an example, in the above embodiments, by using the above-mentioned images and the identifications of the corresponding blade icing areas, the process of training the initial blade icing segmentation model can be as follows Figure 7 shown. That is, after collecting the images containing icing features and annotating them, preprocess the above-mentioned images, such as data augmentation, resizing, etc. Then, input the preprocessed images into the initial blade icing segmentation model respectively to obtain the predicted identification results. Based on the loss between the predicted recognition results and the pre-obtained annotations, adjust the parameters of the initial blade icing segmentation model until the training conditions are met, so as to obtain the final parameters in the blade icing segmentation model, that is, the weight file.

[0089] According to the embodiments of the present disclosure, the blade icing classification model is trained in the following manner: Obtain a third training dataset, wherein the third training dataset includes a plurality of sequence data and the identifications of each sequence data. The sequence data includes information of a predetermined number of second images, and each identification is used to indicate whether the blade of the wind turbine is iced or not; Through the plurality of sequence data and the identifications of each sequence data, train the initial blade icing classification model to obtain the blade icing classification model. Through this embodiment, the initial blade icing classification model can be effectively trained, so as to obtain a blade icing classification model with excellent classification effect.

[0090] The above-mentioned initial blade icing classification model can adopt an LSTM model, that is, construct an LSTM model. This model accepts a sequence of vectors (i.e., the above-mentioned sequence data) as input, uses an embedding layer to map the above-mentioned sequence data to a continuous vector representation, and at the same time adds one or more LSTM layers to capture the temporal correlations in the sequence data, and then adds an output layer and a classification network (such as a perceptron) for icing classification (binary classification, output 1 or 0). The input data form and model structure of the blade icing classification model can be as follows Figure 9 shown. This model is just an example. Figure 9 The shown blade icing classification model constructs an LSTM model with 5 hidden layers and 405 nodes (neurons) in each layer. This model structure is adjusted according to the dataset situation and can achieve a balance between efficiency and accuracy (but its parameters can also be adjusted further). The number of input sequence of this model is fixed at 10 (because 10 frames are extracted in the above embodiments), the length of each sequence is 405, and the output layer is used for icing classification (binary classification, output 1 or 0).

[0091] It should be noted that the Long Short-Term Memory (LSTM) network is a variant of the Recurrent Neural Network (RNN) in deep learning, which is particularly suitable for processing and predicting time series data, natural language processing, and other sequence modeling tasks. LSTM solves the long sequence dependence problem existing in traditional RNNs, enabling it to better capture and remember long-term dependencies in sequences, and thus is very useful in many applications.

[0092] As an example, first, video data of a wind turbine generator can be collected. Still taking the extraction of 10 frames from consecutive frames as an example, as Figure 9 shown, 10 fused vectors with predetermined information can be obtained from it. For example, each vector after the addition process is still a one-dimensional vector with a length of 400. After adding the predetermined information in the above embodiment, 10 one-dimensional vectors with a length of 405 (fused vectors) are obtained. The specific acquisition process has been described in detail above and will not be elaborated here. At this time, the 10 one-dimensional vectors with a length of 405 obtained can be labeled according to prior knowledge as the blades of the wind turbine generator being iced or not iced, and a labeled sequence data is obtained through this step. For a large amount of video data, repeating the above process can obtain multiple groups of data with 10 images in each group. After corresponding labeling operations, multiple labeled sequence data can be obtained, that is, the third training dataset is obtained. This training dataset contains multiple samples, and each sample is a sequence data. Using the above sequence data and corresponding identifiers, an initial blade icing classification model is trained to obtain a trained blade icing classification model.

[0093] After obtaining the trained blade icing classification model, icing detection can be performed on the real-time video data captured by the wind turbine pan-tilt, that is, through the above embodiment, 10 one-dimensional vectors with a length of 405 are obtained from the video data to be detected. The 10 one-dimensional vectors with a length of 405 obtained are input into the trained blade icing classification model in sequence, and the final classification result of whether the blade is iced is obtained based on the output result of the model.

[0094] In summary, the above embodiments of the present disclosure involve three models, namely a blade segmentation model based on semantic segmentation, a blade icing segmentation model based on semantic segmentation, and a blade icing classification model based on a recurrent neural network. The first two models both adopt the yolov8-seg structure, and the blade icing classification model adopts a self-built LSTM network. Among them, the training of the blade segmentation model and the blade icing segmentation model is used to extract the image features of the wind turbine blades, which is very important in the present disclosure. Both of them are trained respectively using the labeled images containing blades, blade icing, etc. based on the pre-trained weights.

[0095] According to the embodiments of the present disclosure, inFigure 1 When the recognition result obtained by the blade icing recognition method shown indicates that the blade of the wind turbine is iced, determine the area of the blade icing region and the area of the blade region of a predetermined number of second images; when the area of the blade icing region is less than the first predetermined ratio of the area of the blade region, determine that the wind turbine is in the first icing state; when the area of the blade icing region is greater than the first predetermined ratio of the area of the blade region and less than the second predetermined ratio of the area of the blade region, determine that the wind turbine is in the second icing state; when the area of the blade icing region is greater than the second predetermined ratio of the area of the blade region, determine that the wind turbine is in the third icing state. Through this embodiment, when the blade is iced, the icing state of the wind turbine can be determined by the area of the blade icing region and the area of the blade region, so that corresponding processing can be performed based on the icing state subsequently, reducing the power generation loss.

[0096] The above-mentioned first predetermined ratio can be set according to actual needs, such as it can be set to 10%, 20%, etc., and the above-mentioned second predetermined ratio can also be set according to actual needs, such as it can be set to 30%, 40%, etc., and the present disclosure does not limit this.

[0097] As an example, taking the first predetermined ratio as 10% and the second predetermined ratio as 30% as an example, when the recognition result indicates that the blade of the wind turbine is iced, the area of the blade icing region and the area of the blade region of each second image can be obtained through the blade segmentation model, and the corresponding areas of each second image in the predetermined number of second images are added together to obtain the area of the blade icing region and the area of the blade region of the predetermined number of second images. When the area of the blade icing region is less than 10% of the area of the blade region, the wind turbine can be regarded as being in the first icing state (for example, a light icing state); when the area of the blade icing region is greater than 10% of the area of the blade region and less than 30% of the area of the blade region, the wind turbine can be regarded as being in the second icing state (for example, a moderate icing state); when the area of the blade icing region is greater than 30% of the area of the blade region, the wind turbine can be regarded as being in the third icing state (for example, a severe icing state).

[0098] It should be noted that it is also possible to determine the icing state of the wind turbine based on the area of the blade icing region and the area of the blade region of each second image after obtaining the area of the blade icing region and the area of the blade region of each second image through the blade segmentation model, and the present disclosure does not limit this.

[0099] As an example, still taking the first predetermined ratio as 10% and the second predetermined ratio as 30% as an example, when the recognition result indicates that the blades of the wind turbine are iced, the ice-covered area and the blade area of each second image can be obtained through the blade segmentation model. When the ice-covered area of all second images is less than 10% of the blade area, the wind turbine can be regarded as being in the first icing state (e.g., a light icing state). When the ice-covered area of any one second image is greater than 10% and less than 30% of the blade area, and the ice-covered area of none of the second images is greater than 30% of the blade area, the wind turbine can be regarded as being in the second icing state (e.g., a moderate icing state). When the ice-covered area of any one second image is greater than 30% of the blade area, the wind turbine can be regarded as being in the third icing state (e.g., a severe icing state).

[0100] According to an embodiment of the present disclosure, when the wind turbine is in the second icing state, the load of the wind turbine is reduced by controlling the pitch angle of the wind turbine; when the wind turbine is in the third icing state, the wind turbine is controlled to stop. Through this embodiment, different control strategies are adopted according to different icing states, so as to maintain the maximum power generation without damaging the wind turbine.

[0101] As an example, when the wind turbine is in the second icing state, the load can be reduced through a pitch control strategy, such as adjusting the pitch angle of the wind turbine to reduce the load of the wind turbine, so as to enable the wind turbine to operate with ice; when the wind turbine is in the third icing state, protective shutdown measures can be taken to avoid losses to the wind turbine.

[0102] According to an embodiment of the present disclosure, after controlling the wind turbine to stop, the ambient temperature of the wind turbine is monitored; when the ambient temperature is higher than a preset temperature and lasts for a preset duration, it is determined whether the icing condition of the wind turbine meets a predetermined condition; in response to the icing condition of the wind turbine meeting the predetermined condition, the wind turbine is restarted, where the predetermined condition includes at least one of the following: it is determined through the blade icing classification model that the blades of the wind turbine are not iced; it is determined through the blade icing classification model that the blades of the wind turbine are iced and the ice-covered area of all predetermined images is less than a second predetermined ratio of the blade area, where the predetermined images are the images used this time to determine whether the blades of the wind turbine are iced. Through this embodiment, after the wind turbine stops, the ambient temperature is detected in real time. When the ambient temperature remains at a relatively high temperature for a long time, the icing condition of the blades can be judged again. If the icing condition of the blades meets the preset conditions, the wind turbine can be restarted to avoid long-term shutdown of the wind turbine and reduce power generation.

[0103] The above preset temperature and preset duration can be set as needed, and the present disclosure does not limit this.

[0104] As an example, taking the preset temperature as T degrees Celsius and the preset duration as H hours, after the wind turbine is shut down, the ambient temperature of the wind turbine is monitored in real time; when the ambient temperature is higher than T degrees Celsius for H consecutive hours, further determine the icing condition of the wind turbine, such as determining whether the blades of the wind turbine are iced through a blade icing classification model, and determining the relationship between the area of the blade icing region of all predetermined images and the area of the blade region; if the blades of the wind turbine are not iced, or the blades of the wind turbine are iced and the area of the blade icing region of all predetermined images is less than a second predetermined ratio of the blade region area, it means that the wind turbine no longer belongs to the severe icing state, and at this time, the shut-down wind turbine can be restarted.

[0105] According to an embodiment of the present disclosure, when there are multiple wind turbines in a wind farm, if the recognition result indicates that the blades of a wind turbine are iced, the wind turbine with iced blades among the multiple wind turbines can be determined through the positioning information included in a predetermined number of second images. Through this embodiment, the wind turbine with iced blades can be located, so that the staff can take targeted measures.

[0106] The above positioning information may include watermarks, time, shooting location, etc., and the present disclosure does not limit this.

[0107] As an example, after the blade icing classification model determines that a wind turbine is iced through a predetermined number of second images, the wind turbine with iced blades can be located by intercepting information such as the watermark, time, and shooting location of the second image.

[0108] Therefore, the present disclosure uses the video data of the operation of the fan captured by the pan-tilt camera, extracts the frames containing the blades in the video through image segmentation, further uses an image segmentation algorithm to extract the characteristics of the blade icing part, and combines a recurrent neural network to perform icing recognition on the monitored fan blades, that is, through model training, feature extraction and segmentation processing are performed on the blades and the icing region, and a blade icing classification model based on LSTM is constructed, reducing the possibility of misjudgment of some similar features, such as reflection and snow accumulation, when only a single image is used for blade icing recognition, so as to achieve accurate judgment of blade icing. Through the embodiments of the present disclosure, the icing condition of the blades can be discovered in a timely and accurate manner, various risks caused by blade icing can be effectively avoided, economic losses can be reduced, and the operation stability and safety of the fan can be improved.

[0109] Based on technologies such as computer vision and deep learning, the present disclosure combines image information of blades in multiple consecutive time periods and characteristics changing over time, and realizes automatic detection and alarm of blade icing conditions through steps such as image acquisition, preprocessing, feature extraction, classification, and recognition, thereby improving the efficiency and safety of wind turbine operation.

[0110] Figure 10 is a block diagram showing a blade icing recognition device of a wind power generation unit according to the present disclosure. As Figure 10 shown, the device includes a video data acquisition unit 100, an image acquisition unit 102, an image set determination unit 104, an extraction unit 106, and a recognition unit 108.

[0111] The video data acquisition unit 100 is configured to acquire video data during the operation of the wind power generation unit; the image acquisition unit 102 is configured to acquire a first image including a predetermined portion exceeding the blade from the video data; the image set determination unit 104 is configured to determine a plurality of image sets based on the first image, where each image set includes images that are temporally continuous among all the first images; the extraction unit 106 is configured to extract a predetermined number of second images from any one of the plurality of image sets at a predetermined interval; the recognition unit 108 is configured to input the predetermined number of second images into a blade icing classification model to obtain a recognition result, where the recognition result is used to indicate whether the blade of the wind power generation unit is iced or not.

[0112] According to an embodiment of the present disclosure, the recognition unit 108 is further configured to remove information outside the blade area from each second image and retain the information of the blade area in each second image, so as to obtain a predetermined number of third images; remove information outside the blade icing area from each second image and retain the information of the blade icing area in each second image, so as to obtain a predetermined number of fourth images; fuse the third image and the fourth image corresponding to the same second image to obtain a predetermined number of fusion vectors; input the predetermined number of fusion vectors into the blade icing classification model to obtain a recognition result.

[0113] According to an embodiment of the present disclosure, the recognition unit 108 is further configured to perform downsampling processing and compression on each third image respectively to obtain a first vector of each third image; perform downsampling processing and compression on each fourth image respectively to obtain a second vector of each fourth image; perform a merging process on the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors.

[0114] According to an embodiment of the present disclosure, the recognition unit 108 is further configured to obtain predetermined information corresponding to each second image, where the predetermined information includes icing information of the blades in the corresponding second image; add the predetermined information to the combined processing result of the first vector and the second vector corresponding to the same second image to obtain a predetermined number of fusion vectors.

[0115] According to an embodiment of the present disclosure, the predetermined information corresponding to each second image is determined as follows: when there is no blade icing area in the second image, the predetermined information of the second image is a first preset vector only including the same preset value; when there is a blade icing area in the second image, the predetermined information of the second image is a second preset vector including multiple values, where the multiple values are determined based on one or more of the coordinates of the blade icing area in the second image, the first direction metric, the second direction metric, and the size of the second image.

[0116] According to an embodiment of the present disclosure, when the first preset vector and the second preset vector are one-dimensional vectors including 5 columns, the first preset vector only includes the preset value, the preset value, the preset value, the preset value; the second preset vector includes a first value, a second value, a third value, a fourth value, and an icing confidence level, where the first value and the second value are determined based on the coordinates of the blade icing area in the second image and the size of the second image, and the third value and the fourth value are determined based on the first direction metric and the second direction metric of the blade icing area in the second image and the size of the second image, respectively.

[0117] According to an embodiment of the present disclosure, the above device further includes: a state determination unit configured to, when the recognition result indicates that the blades of the wind turbine are iced, determine the area of the blade icing area and the area of the blade area of a predetermined number of second images; when the area of the blade icing area is less than a first predetermined ratio of the area of the blade area, determine that the wind turbine is in a first icing state; when the area of the blade icing area is greater than the first predetermined ratio of the area of the blade area and less than a second predetermined ratio of the area of the blade area, determine that the wind turbine is in a second icing state; when the area of the blade icing area is greater than the second predetermined ratio of the area of the blade area, determine that the wind turbine is in a third icing state.

[0118] According to an embodiment of the present disclosure, the state determination unit is further configured to, when the wind turbine is in the second icing state, reduce the load of the wind turbine by controlling the pitch angle of the wind turbine; when the wind turbine is in the third icing state, control the wind turbine to stop.

[0119] According to an embodiment of the present disclosure, the status determination unit is further configured to monitor the ambient temperature of the wind turbine after controlling the wind turbine to stop; determine whether the icing condition of the wind turbine meets a predetermined condition when the ambient temperature is higher than a preset temperature and lasts for a predetermined duration; and restart the wind turbine in response to the icing condition of the wind turbine meeting the predetermined condition, where the predetermined condition includes at least one of the following: determining that the blades of the wind turbine are not iced through a blade icing classification model; determining that the blades of the wind turbine are iced and the area of the icing region of all predetermined images is less than a second predetermined ratio of the blade area through the blade icing classification model, where the predetermined images are the images used for determining the icing of the blades of the wind turbine this time.

[0120] According to an embodiment of the present disclosure, in the case of multiple wind turbines, the above device further includes: a positioning unit configured to determine the wind turbine with iced blades among the multiple wind turbines through the positioning information included in a predetermined number of second images when the recognition result indicates that the blades of the wind turbine are iced.

[0121] According to an embodiment of the present disclosure, the recognition unit 108 is further configured to separately segment each second image through a pre-trained blade segmentation model to obtain a first mask of each second image; and multiply each second image by the corresponding first mask to obtain a predetermined number of third images.

[0122] According to an embodiment of the present disclosure, the recognition unit 108 is further configured to separately segment each second image through a pre-trained blade icing segmentation model to obtain a second mask of each second image; and multiply each second image by the corresponding second mask to obtain a predetermined number of fourth images.

[0123] According to an embodiment of the present disclosure, the blade segmentation model is trained in the following manner: obtaining a first training dataset, where the first training dataset includes multiple images each containing a predetermined portion exceeding the blade and the identification of the blade area in each image; training an initial blade segmentation model through the multiple images each containing a predetermined portion exceeding the blade and the identification of the blade area in each image to obtain the blade segmentation model.

[0124] According to an embodiment of the present disclosure, the blade icing segmentation model is trained in the following manner: obtaining a second training dataset, where the second training dataset contains multiple images with blade icing regions and the identification of the blade icing region in each image; training an initial blade icing segmentation model through the multiple images with blade icing regions and the identification of the blade icing region in each image to obtain the blade icing segmentation model.

[0125] According to an embodiment of the present disclosure, the blade icing classification model is trained in the following manner: obtaining a third training data set, where the third training data set includes a plurality of sequence data and an identifier for each sequence data, the sequence data includes a predetermined number of fusion vectors, and each identifier is used to indicate whether the blade of the wind turbine is iced or not; training an initial blade icing classification model with the plurality of sequence data and the identifier for each sequence data to obtain the blade icing classification model.

[0126] According to an embodiment of the present disclosure, there is provided a computer-readable storage medium storing instructions, wherein when the instructions are run by at least one computing device, the at least one computing device is caused to execute the method for identifying blade icing of a wind turbine as described in any one of the above embodiments.

[0127] According to an embodiment of the present disclosure, there is provided a system including at least one computing device and at least one storage device storing instructions, wherein when the instructions are run by at least one computing device, the at least one computing device is caused to execute the method for identifying blade icing of a wind turbine as described in any one of the above embodiments.

[0128] In another general aspect, there is provided a wind turbine including the device for identifying blade icing of a wind turbine as described above.

[0129] Although some embodiments of the present disclosure have been shown and described, those skilled in the art should understand that these embodiments can be modified without departing from the principles and spirit of the present disclosure defined by the claims and their equivalents.

Claims

1. A method for identifying icing on the blades of a wind turbine, characterized in that, it includes: Obtaining video data during the operation of the wind turbine; Obtaining a first image from the video data that includes a predetermined portion exceeding the blade; Based on the first image, determining a plurality of image sets, where each image set includes images that are temporally continuous in all the first images; Extracting a predetermined number of second images from any one of the plurality of image sets at a predetermined interval; Inputting the predetermined number of second images into a blade icing classification model to obtain an identification result, where the identification result is used to indicate whether the blades of the wind turbine are iced or not.

2. The method for identifying icing on the blades according to claim 1, characterized in that, The step of inputting the predetermined number of second images into a blade icing classification model to obtain an identification result includes: Removing information outside the blade area from each second image and retaining the information of the blade area in each second image, thereby obtaining the predetermined number of third images; Removing information outside the blade icing area from each second image and retaining the information of the blade icing area in each second image, thereby obtaining the predetermined number of fourth images; Fusing the third image and the fourth image corresponding to the same second image to obtain the predetermined number of fusion vectors; Inputting the predetermined number of fusion vectors into the blade icing classification model to obtain the identification result.

3. The method for identifying icing on the blades according to claim 2, characterized in that, The step of fusing the third image and the fourth image corresponding to the same second image to obtain the predetermined number of fusion vectors includes: Performing downsampling processing and compression on each third image respectively to obtain a first vector of each third image; Performing downsampling processing and compression on each fourth image respectively to obtain a second vector of each fourth image; Performing a merging process on the first vector and the second vector corresponding to the same second image to obtain the predetermined number of fusion vectors.

4. The method for identifying icing on the blades according to claim 3, characterized in that, The step of performing a merging process on the first vector and the second vector corresponding to the same second image to obtain the predetermined number of fusion vectors includes: Obtaining predetermined information corresponding to each second image, where the predetermined information includes blade icing information in the corresponding second image; Adding the predetermined information to the merging result of the first vector and the second vector corresponding to the same second image to obtain the predetermined number of fusion vectors.

5. The method for identifying icing on the blades according to claim 4, characterized in that, The predetermined information corresponding to each second image is determined by the following method: In the case where there is no blade icing area in the second image, the predetermined information of the second image is a first preset vector that only includes the same preset value; In the case where there is a blade icing area in the second image, the predetermined information of the second image is a second preset vector including a plurality of values, where the plurality of values are determined based on one or more of the coordinates of the blade icing area in the second image, the first direction measurement, the second direction measurement, and the size of the second image.

6. The method for identifying blade icing according to claim 1, wherein, it further includes: when the recognition result indicates that the blade of the wind turbine is iced, determining the area of the blade icing area and the area of the blade area of the predetermined number of second images; when the area of the blade icing area is less than a first predetermined ratio of the area of the blade area, determining that the wind turbine is in a first icing state; when the area of the blade icing area is greater than the first predetermined ratio of the area of the blade area and less than a second predetermined ratio of the area of the blade area, determining that the wind turbine is in a second icing state, and reducing the load of the wind turbine by controlling the pitch angle of the wind turbine; when the area of the blade icing area is greater than the second predetermined ratio of the area of the blade area, determining that the wind turbine is in a third icing state, and controlling the wind turbine to stop.

7. The method for identifying blade icing according to claim 6, wherein, after controlling the wind turbine to stop, it further includes: monitoring the ambient temperature of the wind turbine; when the ambient temperature is higher than a preset temperature and lasts for a preset duration, determining whether the icing condition of the wind turbine meets a predetermined condition; in response to the icing condition of the wind turbine meeting the predetermined condition, restarting the wind turbine, where the predetermined condition includes at least one of the following: determining that the blade of the wind turbine is not iced through the blade icing classification model; determining that the blade of the wind turbine is iced and the area of the blade icing area of all predetermined images is less than the second predetermined ratio of the area of the blade area through the blade icing classification model, where the predetermined images are the images used to determine whether the blade of the wind turbine is iced this time.

8. The method for identifying blade icing according to claim 1, wherein, in the case where there are multiple wind turbines, it further includes: when the recognition result indicates that the blade of the wind turbine is iced, determining the wind turbine with the iced blade among the multiple wind turbines through the positioning information included in the predetermined number of second images.

9. The method for identifying blade icing according to claim 2, wherein, removing the information outside the blade area from each second image and retaining the information of the blade area in each second image, so as to obtain the predetermined number of third images, includes: respectively segmenting each second image through a pre-trained blade segmentation model to obtain a first mask of each second image; respectively segmenting each second image through a pre-trained blade icing segmentation model to obtain a second mask of each second image; Multiply each second image with the corresponding first mask and second mask respectively to obtain the predetermined number of third images and fourth images.

10. A computer-readable storage medium storing instructions, wherein, when the instructions are run by at least one computing device, the at least one computing device is caused to execute the method for identifying icing on blades of a wind turbine according to any one of claims 1 to 9.

11. A system comprising at least one computing device and at least one storage device storing instructions, wherein, when the instructions are run by the at least one computing device, the at least one computing device is caused to execute the method for identifying icing on blades of a wind turbine according to any one of claims 1 to 9.

Images