A method, device and terminal for collecting blade images under fan operation state

Abstract

The present invention discloses a method, device and terminal for collecting blade images in a wind turbine operation state. The method comprises: controlling a drone to fly above a target wind turbine and vertically shoot downward to obtain at least one upper image, adjusting the position of the drone according to at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image; when the tower center is at the center position of the upper image shot by the drone, determining the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image shot by the drone; determining a collection point according to the three-dimensional spatial position of the blade working plane, controlling the drone to move to each collection point to shoot the blade working plane, and obtaining multiple target images; and classifying the multiple target images into each blade according to the blade pixel ratio in the multiple target images, and obtaining the blade image of the target wind turbine. The present invention can collect blade images in a wind turbine operation state without generating power generation loss.

Description

Technical Field

[0001] The present invention relates to the field of wind power generation, and in particular to a method, device and terminal for collecting blade images in a wind turbine operating state. Background Art

[0002] Wind energy is an important sustainable energy source. Wind power generation has become one of the most widely used new energy technologies. Wind turbine blades are key components for wind turbines to achieve energy conversion. Due to long-term exposure to harsh open-air working environments, scratches, cracks, corrosion and other defects will inevitably occur on their surfaces. Timely detection and repair of these surface defects is an effective way to ensure the normal operation and power generation efficiency of wind turbines and avoid major safety accidents such as blade breakage.

[0003] When inspecting a wind turbine, it is necessary to obtain images of the surface of the wind turbine blades. The existing method of acquiring images of wind turbine blades is to manually control a drone to take images of the wind turbine blades after the wind turbine is shut down. This method is difficult to control manually, and wind turbine shutdown will also result in power generation loss.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, the present invention provides a method, device and terminal for collecting blade images of a wind turbine in operation, aiming to solve the problem in the prior art that the wind turbine needs to be shut down for collecting blade images, resulting in loss of power generation.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A first aspect of the present invention provides a method for collecting blade images in a wind turbine operating state, the method comprising:

[0008] Controlling the drone to fly above the target wind turbine and photographing vertically downward to obtain at least one upper image, and adjusting the position of the drone according to the at least one upper image so that the center of the tower of the target wind turbine is at the center of the upper image;

[0009] When the center of the tower is at the center position of the upper image taken by the drone, the three-dimensional spatial position of the blade working plane of the target wind turbine is determined according to the heading of the drone and the upper image taken by the drone;

[0010] Determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to photograph the blade working plane, and obtain multiple target images;

[0011] The multiple target images are classified into respective blades according to the proportion of blade pixels in the multiple target images to obtain blade images of the target wind turbine.

[0012] The method for collecting blade images in a wind turbine operating state, wherein the step of adjusting the position of the drone according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image, comprises:

[0013] Identify key points of the wind turbine in at least one continuously shot upper image according to a pre-trained first neural network, wherein the key points of the wind turbine include a center point of a nacelle tail, a center point of a hub, and a blade tip point;

[0014] Determine the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine;

[0015] When the center of the tower is not at the center of the upper image, controlling the drone to move toward the center of the tower and acquiring at least one new continuously shot upper image;

[0016] The step of identifying the key points of the wind turbine in at least one continuously shot upper image is repeatedly performed until the center of the tower is at the center position of the upper image.

[0017] The method for collecting blade images in a wind turbine operating state, wherein determining the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine, comprises:

[0018] Acquire an average position of the center point of the tail of the nacelle and an average position of the center point of the hub in the at least one continuously shot upper image;

[0019] Determine the position of the tower center in the upper image according to the ratio of the first physical parameter to the second physical parameter of the target wind turbine and the distance between the average position of the center point of the nacelle tail and the average position of the center point of the hub in the at least one continuously shot upper image;

[0020] The first physical parameter is the actual horizontal distance between the center point of the hub and the center of the tower, and the second physical parameter is the actual horizontal distance between the center of the tower and the center of the tail of the nacelle.

[0021] The method for collecting blade images in a wind turbine operating state, wherein determining the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone, comprises:

[0022] A first straight line is obtained according to all the blade tip points in at least one latest continuously taken upper image;

[0023] A second straight line is obtained according to all the center points of the tail of the nacelle and the center points of the wheel hub in at least one latest continuously shot upper image;

[0024] Determining the orientation of the nacelle of the target wind turbine in the upper image according to the angle between the first straight line and the second straight line;

[0025] Determine the orientation of the cabin of the target wind turbine in the geographic coordinate system according to the heading of the UAV and the orientation of the cabin of the target wind turbine in the upper image;

[0026] The three-dimensional spatial position of the blade working plane is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine.

[0027] The method for acquiring blade images in a wind turbine operating state, wherein determining the orientation of the nacelle of the target wind turbine in the image according to the angle between the first straight line and the second straight line, comprises:

[0028] If the angle between the first straight line and the second straight line is within a preset range, determining the direction of the second straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0029] If the angle between the first straight line and the second straight line is not within the preset range, determining the vertical direction of the first straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0030] The preset range is (90°-X°, 90°+X°), X is a constant, and X>0.

[0031] The method for collecting blade images in a wind turbine operating state, wherein determining the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in a geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine, comprises:

[0032] Determining the position of the center point of the blade working plane in the geographic coordinate system according to a third physical parameter of the target wind turbine and the geographic location of the target wind turbine, wherein the third physical parameter is the hub center height;

[0033] determining the area of ​​the blade working plane according to a fourth physical parameter of the target wind turbine, wherein the fourth physical parameter is an impeller radius;

[0034] The orientation of the blade working plane in the geographic coordinate system is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system.

[0035] The method for acquiring blade images when the wind turbine is in operation, wherein the height of at least one acquisition point is consistent with the center height of the blade working plane; the acquisition points satisfy that the overlapping area between the image taken by the drone at each acquisition point and the image taken at an adjacent acquisition point is greater than a first preset threshold; and the distance from each acquisition point to the blade working plane is greater than a second preset threshold.

[0036] The blade image acquisition method in the wind turbine running state, wherein the controlling the drone to move to each acquisition point to photograph the blade working plane, comprises:

[0037] The drone is controlled to photograph the blade working plane at each collection point by respectively adopting a preset angle upward shooting and a preset angle downward shooting method.

[0038] The method for collecting blade images in a wind turbine operating state, wherein the multiple target images are classified into respective blades according to the percentage of blade pixels in the multiple target images, comprises:

[0039] Extracting the leaf portion in each of the target images through the trained second neural network to obtain the leaf pixel ratio in each of the target images;

[0040] The multiple target images are classified into respective blades according to a change period of the blade pixel ratio in the multiple target images and the number of blades of the target wind turbine.

[0041] A second aspect of the present invention provides a blade image acquisition device in a wind turbine operating state, comprising:

[0042] An upper image shooting control module, the upper image shooting control module is used to control the UAV to fly above the target wind turbine and shoot vertically downward to obtain at least one upper image, and adjust the position of the UAV according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image;

[0043] A blade working plane determination module, the blade working plane determination module is used to determine the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone after the center of the tower is at the center position of the upper image taken by the drone;

[0044] A target image shooting control module, the target image shooting control module is used to determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to shoot the blade working plane, and obtain multiple target images;

[0045] A blade image classification module is used to classify the multiple target images into individual blades according to the proportion of blade pixels in the multiple target images, so as to obtain the blade image of the target wind turbine.

[0046] According to a third aspect of the present invention, a terminal is provided, comprising a processor and a computer-readable storage medium communicatively connected to the processor, wherein the computer-readable storage medium is suitable for storing a plurality of instructions, and the processor is suitable for calling the instructions in the computer-readable storage medium to execute the steps of the method for acquiring blade images when the wind turbine is in operation as described above.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the blade image acquisition method when the wind turbine is in operation as described in any of the above items.

[0048] Compared with the prior art, the present invention provides a method, device and terminal for acquiring blade images in a wind turbine operating state. The method for acquiring blade images in a wind turbine operating state first controls a drone to fly above a target wind turbine and shoots an image of the target wind turbine vertically downward, and adjusts the position of the drone according to the captured image so that the center of the tower of the target wind turbine is at the center of the image captured by the drone. Then, the three-dimensional spatial position of the blade working plane of the target wind turbine is determined according to the heading of the drone and the image captured by the drone. At least one acquisition point is determined according to the three-dimensional spatial position of the blade working plane, and the drone is controlled to move to each acquisition point to shoot the blade working plane to obtain multiple target images. The acquisition point for shooting the blade image is determined according to the three-dimensional spatial position of the blade working plane, so as to avoid interference between the drone and the blade working plane and improve the safety of the drone. Then, the multiple target images are classified into each blade according to the proportion of blade pixels in the multiple target images to obtain the blade image of the target wind turbine, thereby realizing blade image acquisition in a wind turbine operating state. No manual control is required throughout the process and no power generation loss will occur. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of an embodiment of a method for collecting blade images in a wind turbine operating state provided by the present invention;

[0050] Figure 2 A schematic diagram of identifying key points of a fan in an upper image in an embodiment of a method for acquiring blade images in a fan operating state provided by the present invention;

[0051] Figure 3A schematic diagram of physical parameters of a target fan in an embodiment of a method for acquiring blade images under a fan operating state provided by the present invention;

[0052] Figure 4 A schematic diagram of the orientation of the nacelle of the target wind turbine in the upper image in an embodiment of the blade image acquisition method of the wind turbine in the running state provided by the present invention;

[0053] Figure 5 A schematic diagram of acquisition points in an embodiment of a method for acquiring blade images in a wind turbine operating state provided by the present invention;

[0054] Figure 6 A schematic diagram of a drone taking photos at a collection point in an embodiment of a method for collecting blade images in a wind turbine operating state provided by the present invention;

[0055] Figure 7 A schematic diagram of the periodic change of the pixel ratio of blades in an embodiment of the blade image acquisition method provided by the present invention in the wind turbine operation state;

[0056] Figure 8 A structural principle diagram of an embodiment of a blade image acquisition device provided by the present invention when a fan is in operation;

[0057] Fig. 9 A schematic diagram of the principles of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] The blade image acquisition method under the running state of the wind turbine provided by the present invention can be executed by a terminal, and the terminal is a device with computing capability. The terminal can be a drone for taking images of the target wind turbine or a device that is communicatively connected to a drone for taking images of the target wind turbine.

[0060] Embodiment 1

[0061] like Figure 1 As shown, in one embodiment of the method for collecting blade images in the wind turbine running state, the method comprises the steps of:

[0062] S100, controlling the drone to fly above the target wind turbine and shooting vertically downward to obtain at least one upper image, and adjusting the position of the drone according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image.

[0063] In this embodiment, the image obtained by shooting vertically downward from the top of the target wind turbine by the drone is called an upper image. In the blade image acquisition method provided in this embodiment when the wind turbine is in operation, the drone is first controlled to shoot at least one upper image, and then the position of the drone is adjusted according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image. When the drone shoots the upper image, the flight altitude of the drone should be greater than the sum of the impeller radius and the tower height, and the upper image should include a bird's-eye view of the target wind turbine. The adjustment of the position of the drone according to the at least one upper image specifically includes:

[0064] S110, identifying key points of the wind turbine in at least one continuously shot upper image according to a pre-trained first neural network, wherein the key points of the wind turbine include a center point of a nacelle tail, a center point of a hub, and a blade tip point;

[0065] S120, determining the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine;

[0066] S130, when the center of the tower is not at the center of the upper image, controlling the drone to move toward the center of the tower and acquiring at least one new continuously shot upper image;

[0067] S140, repeatedly executing the step of identifying the key points of the wind turbine in at least one continuously shot upper image until the center of the tower is at the center position of the upper image.

[0068] In this embodiment, the drone continuously takes at least one upper image each time (for example, 3 or 5 images), and identifies key points of the wind turbine in the upper image taken this time, including the center point of the tail of the nacelle, the center point of the hub, and the tip point of the blade (such as Figure 2 As shown), specifically, the fan key points can be identified by a pre-trained first neural network, and the first neural network can be trained by multiple groups of training data, and each group of training data used to train the first neural network includes a sample image and a fan key point mark in the sample image, so that the first neural network after training can realize the function of identifying the fan key points in the input image.

[0069] After identifying the key points of the wind turbine in the at least one continuously shot upper image, determining the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine specifically includes:

[0070] S121, obtaining an average position of the center point of the tail of the nacelle and an average position of the center point of the hub in the at least one continuously shot upper image;

[0071] S122. Determine the position of the tower center in the upper image according to the ratio of the first physical parameter to the second physical parameter of the target wind turbine and the distance between the average position of the center point of the nacelle tail and the average position of the center point of the hub in at least one continuously shot upper image.

[0072] The first physical parameter is the actual horizontal distance between the hub center and the tower center, that is, Figure 3 The second physical parameter is the actual horizontal distance between the center of the tower and the center of the tail of the nacelle, that is, Figure 3 b. The first physical parameter and the second physical parameter of the target wind turbine can be obtained according to the design drawings of the target wind turbine. Figure 3 As shown, the ratio of the distance from the center point of the tail of the nacelle to the center of the tower and the distance from the center point of the hub to the center of the tower in the upper image should be equal to the ratio of the second physical parameter to the first physical parameter, so that the position of the center of the tower in the upper image can be determined. In this embodiment, the position of the center of the tower in any upper image taken by the drone can be determined, such as the last or first image of the at least one continuously taken upper image.

[0073] When the determined position of the tower center in the upper image is not the center position of the upper image, the drone is controlled to move toward the tower center and obtain at least one new continuously shot upper image, that is, the drone moves a certain distance (which may be a preset value) toward the tower center and then continuously shoots at least one upper image, and then re-executes the above steps S110-S130 for the at least one newly shot continuously shot upper image until the tower center is at the center position of the upper image, thus indicating that the drone is directly above the tower center.

[0074] Please refer again Figure 1 The blade image acquisition method provided in this embodiment under the wind turbine running state also includes the steps of:

[0075] S200: When the tower center is at the center position of the upper image taken by the drone, determine the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone.

[0076] Specifically, the method of determining the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone comprises the steps of:

[0077] S210, obtaining a first straight line according to all the leaf tip points in at least one latest continuously shot upper image;

[0078] S220, obtaining a second straight line according to all the center points of the tail of the cabin and the center points of the wheel hub in at least one latest continuously shot upper image;

[0079] S230, determining the orientation of the nacelle of the target wind turbine in the upper image according to the angle between the first straight line and the second straight line;

[0080] S240, determining the orientation of the cabin of the target wind turbine in the geographic coordinate system according to the heading of the drone and the orientation of the cabin of the target wind turbine in the upper image;

[0081] S250 . Determine the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine.

[0082] The first straight line is obtained by fitting straight lines to all blade tip points in at least one continuous upper image taken last in the process of adjusting the position of the drone so that the tower center of the target wind turbine is at the center position of the upper image. Specifically, the first straight line is obtained by fitting straight lines to all blade tip points using the least squares method, and the second straight line is obtained by fitting straight lines to all nacelle tail center points and hub center points in at least one continuous upper image taken last in the process of adjusting the position of the drone so that the tower center of the target wind turbine is at the center position of the upper image. Determining the orientation of the nacelle of the target wind turbine in the upper image based on the angle between the first straight line and the second straight line specifically includes:

[0083] If the angle between the first straight line and the second straight line is within a preset range, determining the direction of the second straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0084] If the angle between the first straight line and the second straight line is not within the preset range, determining the vertical direction of the first straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0085] Among them, the preset range is (90°-X°, 90°+X°), X is a constant, and X>0. That is to say, when the angle between the first straight line and the second straight line is close to 90°, the direction of the second straight line is selected as the direction of the cabin of the target wind turbine in the upper image, otherwise the direction of the first straight line is selected as the direction of the cabin of the target wind turbine in the upper image. This is because the tip point of the wind turbine blade is easier to detect than the center point of the tail of the cabin and the center point of the hub and can provide more discrete position points. Those skilled in the art will understand that the value of X can be set by itself, for example, X can be 3, 2, etc.

[0086] After determining the orientation of the nacelle of the target wind turbine in the upper image, the orientation of the nacelle of the target wind turbine in the geographic coordinate system is determined in combination with the heading of the drone. Specifically, the heading of the drone refers to the angle between the nose of the drone and the direction of the North Pole. Since the drone shoots vertically downward to obtain the upper image, and the center of the tower of the target wind turbine is at the center of the upper image, Figure 4 As shown, the vertical symmetry axis of the upper image is consistent with the heading of the UAV, then the orientation of the cabin of the target wind turbine in the upper image can be determined in the geographic coordinate system (i.e., the angle between the orientation of the cabin of the target wind turbine and the direction of the North Pole).

[0087] The three-dimensional spatial position of the blade working plane includes the geographical location, area and direction of the blade working plane. Determining the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in the geographical coordinate system, the geographical location of the target wind turbine and the physical parameters of the target wind turbine includes:

[0088] Determining the position of the center point of the blade working plane in the geographic coordinate system according to a third physical parameter of the target wind turbine and the geographic location of the target wind turbine, wherein the third physical parameter is the hub center height;

[0089] determining the area of ​​the blade working plane according to a fourth physical parameter of the target wind turbine, wherein the fourth physical parameter is an impeller radius;

[0090] The orientation of the blade working plane in the geographic coordinate system is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system.

[0091] Specifically, the geographical location of the target wind turbine is the longitude and latitude of the tower of the target wind turbine, such as Figure 3 As shown, the position of the hub center in the geographic coordinate system can be determined according to the longitude and latitude of the tower and the hub center height of the target wind turbine, and the area of ​​the blade working plane of the target wind turbine can be determined according to the impeller radius of the target wind turbine. The orientation of the blade working plane in the geographic coordinate system should be perpendicular to the orientation of the cabin of the target wind turbine in the geographic coordinate system.

[0092] Please refer again Figure 1 The blade image acquisition method provided in this embodiment under the wind turbine running state also includes the steps of:

[0093] S300, determining at least one acquisition point according to the three-dimensional spatial position of the blade working plane, controlling the drone to move to each acquisition point to photograph the blade working plane, and obtaining a plurality of target images.

[0094] At least one acquisition point can be determined based on the three-dimensional space of the blade working plane, and the at least one acquisition point forms an inspection route. The drone flies along the inspection route, hovers at each acquisition point to photograph the blade working plane, and obtains multiple target images.

[0095] like Figure 5 As shown, since the blade length of the target wind turbine may be very large, in order to collect a complete blade image, a plurality of collection points are provided, which are distributed on the front and back sides of the target wind turbine to collect the front and back images of the blades of the target wind turbine. Specifically, the height of each of the at least one collection point is consistent with the center height of the blade working plane and the height of the hub center of the target wind turbine. It is worth noting that the consistency mentioned here is not limited to the exact same numerical value, but there may be a certain error range. The collection points should satisfy that the overlap area of ​​the image taken by the drone at each collection point and the image taken at the adjacent collection point is greater than the first preset threshold, and the first preset threshold may be 1 / 2, 1 / 3, etc. At the same time, in order to ensure the safety of the drone, not to interfere with the blade working plane, and not to be affected by the airflow disturbance generated by the blade working plane, the distance from each collection point to the blade working plane is greater than the second preset threshold, and the second preset threshold may be 5-10 meters.

[0096] The blade surface of the target wind turbine may not be a plane. In a possible implementation, controlling the drone to move to each collection point to photograph the blade working plane includes:

[0097] The drone is controlled to photograph the blade working plane at each collection point by respectively adopting a preset angle upward shooting and a preset angle downward shooting method.

[0098] like Figure 6 As shown, the preset angle may be 30°. Of course, those skilled in the art may understand that the preset angle may be adjusted according to the blade shape of the target wind turbine, and the present invention does not make any specific limitation to this.

[0099] In a possible implementation, it can be determined whether the shooting mode of the drone for shooting the target image is video shooting or timed photo shooting according to the linear rotation speed of the blades of the target wind turbine. Specifically, when the drone is in a hovering state for shooting, the rotation of the blades of the target wind turbine will cause the proportion of the blades in the shooting picture to change periodically. The linear rotation speed of the blades of the target wind turbine can be obtained according to the change cycle of the proportion of the blades in the shooting picture and the blade length of the target wind turbine. When the linear rotation speed of the blades of the target wind turbine is greater than a certain value, the target image is shot in a video shooting mode. When the linear rotation speed of the blades of the target wind turbine is less than a certain value, the target image is shot in a timed photo shooting mode. Specifically, the proportion of the blades in the shooting picture can be calculated by counting the pixel proportion of the blade part. The blade part in the shot image can be extracted by a pre-trained second neural network to obtain the pixel proportion of the blade part. The second neural network will be specifically described later.

[0100] Please refer again Figure 1 The blade image acquisition method provided in this embodiment under the operation state of the fan further includes the following steps:

[0101] S400: Classify the multiple target images into leaves according to the percentage of leaf pixels in the multiple target images.

[0102] Specifically, the classifying the multiple target images into respective leaves according to the leaf pixel ratios in the multiple target images includes:

[0103] S410, extracting the leaf portion in each of the target images through the trained second neural network to obtain the leaf pixel ratio in each of the target images;

[0104] S420: Classify the multiple target images into individual blades according to a change cycle of the blade pixel ratio in the multiple target images and the number of blades of the target wind turbine.

[0105] The second neural network is trained by multiple groups of sample data, and each group of sample data used to train the second neural network includes a sample image and a fan blade area label in the sample image, so that the trained second neural network can extract the fan blade area in the image input to the second neural network.

[0106] After extracting the leaf portion in the target image through the second neural network, the percentage of leaf pixels in the target image is counted, such as Figure 7As shown, as described above, the percentage of blade pixels in the multiple target images should change periodically, so the multiple target images can be classified into each blade according to the change period of the percentage of blade pixels in the multiple target images and the number of blades of the target fan. For example, assume that the target fan has three blades, numbered 1, 2, and 3. The percentage of blade pixels of each blade in the continuous target images should change periodically in the order of less-more-less, so the continuous target images with this periodic change can be classified as the blade numbered 1, the next target image with this periodic change can be classified as the blade numbered 2, the next target image with this periodic change can be classified as the blade numbered 3, and then the next target image with this periodic change can be classified as the blade numbered 1, and so on.

[0107] After all target images are classified, images of each blade in the target wind turbine are obtained. Although there is no complete blade image, images of various parts of the blade are collected, which does not affect the use of the image for blade quality detection.

[0108] In summary, the present embodiment provides a method for acquiring blade images under the operating state of a wind turbine. The method first controls the drone to fly above the target wind turbine and shoots an image of the target wind turbine above vertically downward. The position of the drone is adjusted according to the captured image so that the center of the tower of the target wind turbine is at the center of the image above captured by the drone. The three-dimensional spatial position of the blade working plane of the target wind turbine is determined according to the heading of the drone and the image above captured by the drone. At least one acquisition point is determined according to the three-dimensional spatial position of the blade working plane. The drone is controlled to move to each acquisition point to shoot the blade working plane to obtain multiple target images. The acquisition point for shooting the blade image is determined according to the three-dimensional spatial position of the blade working plane, so as to avoid interference between the drone and the blade working plane and improve the safety of the drone. The multiple target images are then classified into each blade according to the proportion of blade pixels in the multiple target images to obtain the blade image of the target wind turbine. This realizes blade image acquisition under the operating state of the wind turbine. No manual control is required throughout the process and no power generation loss will occur.

[0109] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present specification are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0110] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0111] Embodiment 2

[0112] Based on the above embodiments, the present invention also provides a blade image acquisition device under the operation state of the fan. Figure 8 As shown, the blade image acquisition device in the fan running state includes:

[0113] An upper image shooting control module, the upper image shooting control module is used to control the drone to fly above the target wind turbine and shoot vertically downward to obtain at least one upper image, and adjust the position of the drone according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image, as described in the first embodiment;

[0114] A blade working plane determination module, the blade working plane determination module is used to determine the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone when the center of the tower is at the center position of the upper image taken by the drone, as described in the first embodiment;

[0115] A target image shooting control module, the target image shooting control module is used to determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to shoot the blade working plane, and obtain multiple target images, as described in the first embodiment;

[0116] A blade image classification module is used to classify the multiple target images into individual blades according to the proportion of blade pixels in the multiple target images, so as to obtain the blade image of the target wind turbine, as specifically described in the first embodiment.

[0117] Embodiment 3

[0118] Based on the above embodiments, the present invention also provides a terminal, such as Fig. 9 As shown, the terminal includes a processor 10 and a memory 20. Fig. 9 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0119] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a blade image acquisition program 30 in a wind turbine running state, and the blade image acquisition program 30 in a wind turbine running state can be executed by the processor 10, thereby realizing the blade image acquisition method in a wind turbine running state in the present application.

[0120] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other chip, which is used to run the program code or process data stored in the memory 20, such as executing the blade image acquisition method when the wind turbine is in operation.

[0121] In one embodiment, when the processor 10 executes the blade image acquisition program 30 in the memory 20 when the wind turbine is in operation, the following steps are implemented:

[0122] Controlling the drone to fly above the target wind turbine and photographing vertically downward to obtain at least one upper image, and adjusting the position of the drone according to the at least one upper image so that the center of the tower of the target wind turbine is at the center of the upper image;

[0123] When the center of the tower is at the center position of the upper image taken by the drone, the three-dimensional spatial position of the blade working plane of the target wind turbine is determined according to the heading of the drone and the upper image taken by the drone;

[0124] Determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to photograph the blade working plane, and obtain multiple target images;

[0125] The multiple target images are classified into respective blades according to the proportion of blade pixels in the multiple target images to obtain blade images of the target wind turbine.

[0126] Wherein, adjusting the position of the drone according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image includes:

[0127] Identify key points of the wind turbine in at least one continuously shot upper image according to a pre-trained first neural network, wherein the key points of the wind turbine include a center point of a nacelle tail, a center point of a hub, and a blade tip point;

[0128] Determine the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine;

[0129] When the center of the tower is not at the center of the upper image, controlling the drone to move toward the center of the tower and acquiring at least one new continuously shot upper image;

[0130] The step of identifying the key points of the wind turbine in at least one continuously shot upper image is repeatedly performed until the center of the tower is at the center position of the upper image.

[0131] Wherein, determining the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine includes:

[0132] Acquire an average position of the center point of the tail of the nacelle and an average position of the center point of the hub in the at least one continuously shot upper image;

[0133] Determine the position of the tower center in the upper image according to the ratio of the first physical parameter to the second physical parameter of the target wind turbine and the distance between the average position of the center point of the nacelle tail and the average position of the center point of the hub in the at least one continuously shot upper image;

[0134] The first physical parameter is the actual horizontal distance between the center point of the hub and the center of the tower, and the second physical parameter is the actual horizontal distance between the center of the tower and the center of the tail of the nacelle.

[0135] Wherein, determining the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone includes:

[0136] A first straight line is obtained according to all the blade tip points in at least one latest continuously taken upper image;

[0137] A second straight line is obtained according to all the center points of the tail of the nacelle and the center points of the wheel hub in at least one latest continuously shot upper image;

[0138] Determining the orientation of the nacelle of the target wind turbine in the upper image according to the angle between the first straight line and the second straight line;

[0139] Determine the orientation of the cabin of the target wind turbine in the geographic coordinate system according to the heading of the UAV and the orientation of the cabin of the target wind turbine in the upper image;

[0140] The three-dimensional spatial position of the blade working plane is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine.

[0141] Wherein, determining the orientation of the nacelle of the target wind turbine in the image according to the angle between the first straight line and the second straight line includes:

[0142] If the angle between the first straight line and the second straight line is within a preset range, determining the direction of the second straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0143] If the angle between the first straight line and the second straight line is not within the preset range, determining the vertical direction of the first straight line as the direction of the nacelle of the target wind turbine in the upper image;

[0144] The preset range is (90°-X°, 90°+X°), X is a constant, and X>0.

[0145] Wherein, determining the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine includes:

[0146] Determining the position of the center point of the blade working plane in the geographic coordinate system according to a third physical parameter of the target wind turbine and the geographic location of the target wind turbine, wherein the third physical parameter is the hub center height;

[0147] determining the area of ​​the blade working plane according to a fourth physical parameter of the target wind turbine, wherein the fourth physical parameter is an impeller radius;

[0148] The orientation of the blade working plane in the geographic coordinate system is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system.

[0149] Among them, the height of at least one acquisition point is consistent with the center height of the blade working plane; the acquisition points satisfy that the overlapping area of ​​the image taken by the drone at each acquisition point and the image taken at the adjacent acquisition point is greater than a first preset threshold; the distance from each acquisition point to the blade working plane is greater than a second preset threshold.

[0150] Wherein, controlling the drone to move to each of the collection points to photograph the blade working plane includes:

[0151] The drone is controlled to photograph the blade working plane at each collection point by respectively adopting a preset angle upward shooting and a preset angle downward shooting method.

[0152] The step of classifying the plurality of target images into respective leaves according to the leaf pixel ratios in the plurality of target images comprises:

[0153] Extracting the leaf portion in each of the target images through the trained second neural network to obtain the leaf pixel ratio in each of the target images;

[0154] The multiple target images are classified into respective blades according to a change period of the blade pixel ratio in the multiple target images and the number of blades of the target wind turbine.

[0155] Embodiment 4

[0156] The present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the blade image acquisition method when the wind turbine is in operation as described above.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for collecting blade images when a fan is in operation, characterized in that: The method comprises: Controlling the drone to fly above the target wind turbine and photographing vertically downward to obtain at least one upper image, and adjusting the position of the drone according to the at least one upper image so that the center of the tower of the target wind turbine is at the center of the upper image; When the target wind turbine is in operation, after the tower center is at the center of the upper image taken by the drone, the three-dimensional spatial position of the blade working plane of the target wind turbine is determined according to the heading of the drone and the upper image taken by the drone; Determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to photograph the blade working plane, and obtain multiple target images; Classifying the multiple target images into respective blades according to the proportion of blade pixels in the multiple target images to obtain blade images of the target wind turbine; The step of determining the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone comprises: A first straight line is obtained according to all the blade tip points in at least one latest continuously taken upper image; A second straight line is obtained according to all the center points of the tail of the nacelle and the center points of the wheel hub in at least one latest continuously shot upper image; Determining the orientation of the nacelle of the target wind turbine in the upper image according to the angle between the first straight line and the second straight line; Determine the orientation of the cabin of the target wind turbine in the geographic coordinate system according to the heading of the UAV and the orientation of the cabin of the target wind turbine in the upper image; Determine the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine; Fitting straight lines to all blade tip points by the least square method to obtain the first straight line, and fitting straight lines to all nacelle tail center points and hub center points in at least one continuous upper image taken last in the process of adjusting the position of the drone so that the tower center of the target wind turbine is at the center position of the upper image by the least square method to obtain the second straight line; The classifying the plurality of target images into respective leaves according to the leaf pixel ratios in the plurality of target images comprises: Extracting the leaf portion in each of the target images through the trained second neural network to obtain the leaf pixel ratio in each of the target images; The multiple target images are classified into respective blades according to a change period of the blade pixel ratio in the multiple target images and the number of blades of the target wind turbine.

2. The method for collecting blade images in a wind turbine operating state according to claim 1, characterized in that: The step of adjusting the position of the drone according to the at least one upper image so that the center of the tower of the target wind turbine is at the center of the upper image includes: Identify key points of the wind turbine in at least one continuously shot upper image according to a pre-trained first neural network, wherein the key points of the wind turbine include a center point of a nacelle tail, a center point of a hub, and a blade tip point; Determine the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine; When the center of the tower is not at the center of the upper image, controlling the drone to move toward the center of the tower and acquiring at least one new continuously shot upper image; The step of identifying the key points of the wind turbine in at least one continuously shot upper image is repeatedly performed until the center of the tower is at the center position of the upper image.

3. The method for collecting blade images in a wind turbine operating state according to claim 2, characterized in that: Determining the position of the tower center of the target wind turbine in the upper image according to the physical parameters of the target wind turbine includes: Acquire an average position of the center point of the tail of the nacelle and an average position of the center point of the hub in the at least one continuously shot upper image; Determine the position of the tower center in the upper image according to the ratio of the first physical parameter to the second physical parameter of the target wind turbine and the distance between the average position of the center point of the nacelle tail and the average position of the center point of the hub in the at least one continuously shot upper image; The first physical parameter is the actual horizontal distance between the center point of the hub and the center of the tower, and the second physical parameter is the actual horizontal distance between the center of the tower and the center of the tail of the nacelle.

4. The method for collecting blade images in a wind turbine operating state according to claim 1, characterized in that: The determining the orientation of the nacelle of the target wind turbine in the image according to the angle between the first straight line and the second straight line includes: If the angle between the first straight line and the second straight line is within a preset range, determining the direction of the second straight line as the direction of the nacelle of the target wind turbine in the upper image; If the angle between the first straight line and the second straight line is not within the preset range, determining the vertical direction of the first straight line as the direction of the nacelle of the target wind turbine in the upper image; The preset range is (90°-X°, 90°+X°), X is a constant, and X>0.

5. The method for collecting blade images in a wind turbine operating state according to claim 1, characterized in that: Determining the three-dimensional spatial position of the blade working plane according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system, the geographic location of the target wind turbine, and the physical parameters of the target wind turbine includes: Determining the position of the center point of the blade working plane in the geographic coordinate system according to a third physical parameter of the target wind turbine and the geographic location of the target wind turbine, wherein the third physical parameter is the hub center height; determining the area of ​​the blade working plane according to a fourth physical parameter of the target wind turbine, wherein the fourth physical parameter is an impeller radius; The orientation of the blade working plane in the geographic coordinate system is determined according to the orientation of the nacelle of the target wind turbine in the geographic coordinate system.

6. The method for collecting blade images in a wind turbine operating state according to claim 1, characterized in that: The height of at least one acquisition point is consistent with the center height of the blade working plane; the acquisition points satisfy that the overlapping area of ​​the image taken by the drone at each acquisition point and the image taken at an adjacent acquisition point is greater than a first preset threshold; the distance from each acquisition point to the blade working plane is greater than a second preset threshold.

7. The method for collecting blade images in a wind turbine operating state according to claim 1, characterized in that: The controlling the drone to move to each of the collection points to photograph the blade working plane includes: The drone is controlled to photograph the blade working plane at each collection point by respectively adopting a preset angle upward shooting and a preset angle downward shooting method.

8. A blade image acquisition device in a wind turbine operating state, characterized in that: include: An upper image shooting control module, the upper image shooting control module is used to control the UAV to fly above the target wind turbine and shoot vertically downward to obtain at least one upper image, and adjust the position of the UAV according to the at least one upper image so that the tower center of the target wind turbine is at the center position of the upper image; A blade working plane determination module, the blade working plane determination module is used to determine the three-dimensional spatial position of the blade working plane of the target wind turbine according to the heading of the drone and the upper image taken by the drone when the center of the tower is at the center position of the upper image taken by the drone in the operating state of the target wind turbine; The blade working plane determination module is also used to obtain a first straight line according to all blade tip points in at least one latest continuously shot upper image; obtain a second straight line according to all cabin tail center points and hub center points in at least one latest continuously shot upper image; determine the orientation of the cabin of the target wind turbine in the upper image according to the angle between the first straight line and the second straight line; determine the orientation of the cabin of the target wind turbine in the geographic coordinate system according to the heading of the drone and the orientation of the cabin of the target wind turbine in the upper image; determine the three-dimensional spatial position of the blade working plane according to the orientation of the cabin of the target wind turbine in the geographic coordinate system, the geographical location of the target wind turbine and the physical parameters of the target wind turbine; The blade working plane determination module is further used to fit straight lines to all blade tip points by the least square method to obtain the first straight line, and to fit straight lines to all nacelle tail center points and hub center points in at least one continuous upper image taken last in the process of adjusting the position of the drone so that the tower center of the target wind turbine is at the center position of the upper image by the least square method to obtain the second straight line; A target image shooting control module, the target image shooting control module is used to determine at least one acquisition point according to the three-dimensional spatial position of the blade working plane, control the drone to move to each acquisition point to shoot the blade working plane, and obtain multiple target images; A blade image classification module, wherein the blade image classification module is used to classify the multiple target images into individual blades according to the percentage of blade pixels in the multiple target images, so as to obtain a blade image of the target wind turbine; The leaf image classification module is also used to extract the leaf part in each of the target images through the trained second neural network to obtain the leaf pixel ratio in each of the target images; The multiple target images are classified into respective blades according to a change period of the blade pixel ratio in the multiple target images and the number of blades of the target wind turbine.

9. A terminal, characterized in that: The terminal includes: a processor, a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the computer-readable storage medium to execute the steps of the method for acquiring blade images in the operating state of the wind turbine as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the blade image acquisition method in the wind turbine operating state as described in any one of claims 1-7.

Images