Fan automatic inspection method and system based on edge device

Abstract

The invention discloses a fan automatic inspection method based on an edge device, and the method comprises the following steps: S1, obtaining a front image of a fan after any stop position through a camera configured by an unmanned plane, the front image comprising a hub and three blades of the fan; s2, the actual distance between the unmanned aerial vehicle and a fan hub is obtained through laser equipment arranged on the unmanned aerial vehicle, and the actual distance, the unmanned aerial vehicle parameters and the environment parameters are transmitted to an edge calculation device of an unmanned aerial vehicle remote controller; and S3, transmitting the front image to an edge-based computing device, and performing image recognition to obtain an optimal inspection route. Detection is carried out after the fan stops at the unlimited position, the efficiency is higher, and the cost is lower.

Description

Technical Field

[0001] This patent application relates to the field of wind turbine inspection technology, and in particular to an automatic wind turbine inspection method and system based on edge devices. Background Technology

[0002] With the rapid development of renewable energy and the increasing maturity of wind power technology, the scale of wind turbine units is constantly expanding, making the operation and maintenance management of wind farms increasingly important. However, wind turbines inevitably experience various faults during operation, one of which significantly impacts power generation: blade damage. Currently, with the rapid development of drones, using them for wind turbine inspection has become a trend. Because wind turbine blade damage has certain characteristics in infrared imaging, drones can be used for automatic cruise detection, analyzing captured images to quickly and accurately diagnose blade damage. However, current mainstream wind turbine shutdown inspections have strict requirements regarding the stopping position of the blades, such as a Y-shape, making shutdown difficult. Therefore, we propose an automatic wind turbine inspection method and system based on edge devices. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, the first objective of this invention is to propose an automatic wind turbine inspection method based on an edge computing device. This method uses an edge computing device to perform image recognition, based on the two-dimensional vertical plane angle of the wind turbine blades, and arranges the three two-dimensional vertical plane angles in ascending order to determine the shooting order. Based on the point value, height value, and horizontal plane coordinate value, the coordinate values ​​of each shooting point are obtained, and fixed-point inspection is performed to obtain the optimal inspection route.

[0004] The second objective of this invention is to provide an automatic wind turbine inspection system based on edge devices.

[0005] The third objective of this invention is to provide an electronic device.

[0006] The fourth objective of this invention is to provide a computer-readable storage medium.

[0007] To achieve the above objectives, a first aspect of the present invention provides an automatic wind turbine inspection method based on an edge device, comprising the following steps:

[0008] S1. Obtain a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine.

[0009] S2. Obtain the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmit the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller;

[0010] S3. Transmit the frontal image to an edge computing device for image recognition and obtain the optimal inspection route.

[0011] Furthermore, the drone parameters and environmental parameters include wind turbine blade length, inspection speed, and photo interval distance.

[0012] Further, step S3, transmitting the frontal image to an edge computing device for image recognition to obtain the optimal inspection route, includes:

[0013] S31. Obtain the first frontal image taken directly at the hub, disassemble the frontal image, and calculate the two-dimensional vertical plane angle of the three blades relative to the ground horizontal plane, as well as the three-dimensional coordinates of latitude, longitude, and altitude of the UAV and the orientation angle facing the wind turbine.

[0014] S32. Compare the two-dimensional vertical plane angles of the three blades, and arrange the three two-dimensional vertical plane angles in ascending order to determine the shooting order;

[0015] S33. Based on the preset photo interval distance and the length of the wind turbine blades, obtain the position and height values ​​of the blades for the next photo.

[0016] S34. Obtain the horizontal plane coordinates of the blade based on the horizontal projection value and the orientation angle;

[0017] S35. Obtain the actual coordinate position of the blade based on the horizontal plane coordinates of the blade and the actual latitude and longitude of the Earth.

[0018] S36. Repeat steps S33-S35 to obtain the actual coordinate positions of other blades, perform fixed-point inspections, and obtain the optimal inspection route.

[0019] Furthermore, the formula for calculating the position value of a single leaf during photography in step S33 is as follows:

[0020] NA = L / M + 1;

[0021] Where L is the blade length and M is the photo interval distance;

[0022] The formula for calculating the height value of any shooting point on any leaf is:

[0023] zi = (i*M)*sin(α*π / 180);

[0024] i represents the i-th point of a single blade, and α represents the angle between two perpendicular planes.

[0025] Further, obtaining the horizontal plane coordinates of the blade based on the horizontal projection value and the orientation angle includes:

[0026] The horizontal projection value of the i-th point is obtained using the following formula:

[0027] Li = (i*M)*cos(α*π / 180);

[0028] The horizontal coordinates are:

[0029] xi=(i*M)*cos(α*π / 180)*sin(θ);

[0030] yi=(i*M)*cos(α*π / 180)*cos(θ);

[0031] θ is the orientation angle transformed relative to the Earth coordinate system.

[0032] Furthermore, obtaining the actual coordinate position of the blade based on its horizontal plane coordinates and the actual latitude and longitude of the Earth includes:

[0033] The formula for converting the horizontal plane coordinates of the blade to actual Earth latitude and longitude is as follows:

[0034] Xi = xi / EARMH_RADIUS;

[0035] Yi=yi / EARMH_RADIUS*cos(X*π / 180);

[0036] Zi = zi;

[0037] EARMH_RADIUS is the Earth's radius, and X is the X value of the three-dimensional height coordinate;

[0038] The coordinates of the i-th point are (X+Xi, Y+Yi, Z+Zi);

[0039] Where (X, Y, Z) are the three-dimensional height coordinates.

[0040] A second aspect of the present invention provides an automatic wind turbine inspection system based on an edge device, comprising:

[0041] The first acquisition module acquires a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine.

[0042] The second acquisition module acquires the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmits the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller;

[0043] The recognition module is used to transmit the frontal image to an edge computing device for image recognition and to obtain the optimal inspection route.

[0044] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the automatic wind turbine inspection method based on an edge device as described in any of the preceding claims.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automatic wind turbine inspection method based on an edge device as described in any of the preceding claims.

[0046] Compared with existing technologies, the advantages of this invention are as follows: This automatic wind turbine inspection method based on edge devices acquires a frontal image of the wind turbine at any stopping position using a camera mounted on a drone, obtains the actual distance between the drone and the wind turbine hub using a laser device mounted on the drone, and transmits the actual distance, drone parameters, and environmental parameters to the edge computing device of the drone remote controller; the frontal image is then transmitted to the edge computing device for image recognition to obtain the optimal inspection route. Inspection is performed after the wind turbine has stopped at any location without restriction, resulting in higher efficiency and lower cost. Attached Figure Description

[0047] Figure 1 This is a flowchart of the automatic wind turbine inspection method based on edge devices according to the present invention;

[0048] Figure 2 This is a detailed flowchart of step S3 of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention;

[0050] Figure 4 This is a schematic diagram of the automatic inspection path of the present invention. Detailed Implementation

[0051] The following specific examples illustrate the implementation of this patent application. Those skilled in the art can easily understand other advantages and effects of this patent application from the content disclosed in this specification. This patent application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this patent application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0052] The automatic wind turbine inspection method and system based on edge devices provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0053] In this embodiment of the application, an edge-device-based automatic wind turbine inspection system can be used to execute an edge-device-based automatic wind turbine inspection method, thereby achieving automatic inspection of wind turbine blades, obtaining the optimal inspection route, and performing inspections after the wind turbine is stopped at an unrestricted location, resulting in higher efficiency and lower cost.

[0054] The wind turbine automatic inspection system based on edge devices includes a first acquisition module, a second acquisition module, and an identification module.

[0055] The first acquisition module acquires a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine.

[0056] The second acquisition module acquires the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmits the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller;

[0057] The recognition module is used to transmit the frontal image to an edge computing device for image recognition and to obtain the optimal inspection route.

[0058] This system can be applied to a terminal, specifically executed by the hardware or software within the terminal.

[0059] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0060] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0061] The automatic wind turbine inspection method based on edge devices provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the automatic wind turbine inspection method based on edge devices. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The automatic wind turbine inspection method based on edge devices provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0062] Figure 1 This is a flowchart of an embodiment of the automatic wind turbine inspection method based on edge devices according to the present invention, as follows: Figure 1As shown, the automatic wind turbine inspection method based on edge devices includes the following steps:

[0063] S1. Obtain a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine.

[0064] S2. Obtain the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmit the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller;

[0065] In some instances, drone parameters and environmental parameters include wind turbine blade length, inspection speed, and photo interval distance.

[0066] S3. Transmit the frontal image to an edge computing device for image recognition and obtain the optimal inspection route.

[0067] In this embodiment, for step S3, the frontal image is transmitted to an edge computing device for image recognition to obtain the optimal inspection route, such as... Figure 2 As shown, it specifically includes:

[0068] S31. Obtain the first frontal image taken directly at the hub, disassemble the frontal image, and calculate the two-dimensional vertical plane angle of the three blades relative to the ground horizontal plane, as well as the three-dimensional coordinates of latitude, longitude, and altitude of the UAV and the orientation angle facing the wind turbine.

[0069] S32. Compare the two-dimensional vertical plane angles of the three blades, and arrange the three two-dimensional vertical plane angles in ascending order to determine the shooting order;

[0070] S33. Based on the preset photo interval distance and the length of the wind turbine blades, obtain the position and height values ​​of the blades for the next photo.

[0071] The formula for calculating the position value when photographing a single leaf is:

[0072] NA = L / M + 1;

[0073] Where L is the blade length and M is the photo interval distance;

[0074] The formula for calculating the height value of any shooting point on any leaf is:

[0075] zi = (i*M)*sin(α*π / 180);

[0076] i represents the i-th point of a single blade, and α represents the angle between two perpendicular planes.

[0077] S34. Obtain the horizontal plane coordinates of the blade based on the horizontal projection value and the orientation angle;

[0078] In this step, firstly, the horizontal projection value of the i-th point is obtained, and its calculation formula is as follows:

[0079] Li = (i*M)*cos(α*π / 180);

[0080] Based on the horizontal projection values, the horizontal plane coordinates are obtained as follows:

[0081] xi=(i*M)*cos(α*π / 180)*sin(θ);

[0082] yi=(i*M)*cos(α*π / 180)*cos(θ);

[0083] θ is the orientation angle transformed relative to the Earth coordinate system.

[0084] S35. Obtain the actual coordinate position of the blade based on the horizontal plane coordinates of the blade and the actual latitude and longitude of the Earth.

[0085] In this step, the horizontal coordinates of the blade are switched to actual Earth latitude and longitude, and the calculation formula is as follows:

[0086] Xi = xi / EARMH_RADIUS;

[0087] Yi=yi / EARMH_RADIUS*cos(X*π / 180);

[0088] Zi = zi;

[0089] EARMH_RADIUS is the Earth's radius, and X is the X value of the three-dimensional height coordinate;

[0090] The coordinates of the i-th point are (X+Xi, Y+Yi, Z+Zi);

[0091] Where (X, Y, Z) are the three-dimensional height coordinates.

[0092] S36. Repeat steps S33-S35 to obtain the actual coordinate positions of other blades, perform fixed-point inspections, and obtain the optimal inspection route, such as... Figure 4 As shown.

[0093] The automatic wind turbine inspection system based on edge devices in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the system.

[0094] The automatic wind turbine inspection system based on edge devices in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0095] The automatic wind turbine inspection system based on edge devices provided in this application can achieve… Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0096] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements the various processes of the above-described embodiment of the automatic wind turbine inspection method based on edge devices and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0097] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0098] The present invention also proposes a computer-readable storage medium storing a computer program, which is configured to run and implement the automatic wind turbine inspection method based on edge devices as proposed in the embodiments of the present invention.

[0099] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0100] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0102] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0105] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0106] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for automatic inspection of wind turbines based on edge devices, characterized in that, Includes the following steps: S1. Obtain a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine. S2. Obtain the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmit the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller; S3. Transmit the frontal image to an edge computing device for image recognition and obtain the optimal inspection route.

2. The automatic wind turbine inspection method based on edge devices according to claim 1, characterized in that, The drone parameters and environmental parameters include wind turbine blade length, inspection speed, and photo interval distance.

3. The automatic wind turbine inspection method based on edge devices according to claim 1, characterized in that, Step S3 involves transmitting the frontal image to an edge computing device for image recognition to obtain the optimal inspection route, including: S31. Obtain the first frontal image taken directly at the hub, disassemble the frontal image, and calculate the two-dimensional vertical plane angle of the three blades relative to the ground horizontal plane, as well as the three-dimensional coordinates of latitude, longitude, and altitude of the UAV and the orientation angle facing the wind turbine. S32. Compare the two-dimensional vertical plane angles of the three blades, and arrange the three two-dimensional vertical plane angles in ascending order to determine the shooting order; S33. Based on the preset photo interval distance and the length of the wind turbine blades, obtain the position and height values ​​of the blades for the next photo. S34. Obtain the horizontal plane coordinates of the blade based on the horizontal projection value and the orientation angle; S35. Obtain the actual coordinate position of the blade based on the horizontal plane coordinates of the blade and the actual latitude and longitude of the Earth. S36. Repeat steps S33-S35 to obtain the actual coordinate positions of other blades, perform fixed-point inspections, and obtain the optimal inspection route.

4. The automatic wind turbine inspection method based on edge devices according to claim 3, characterized in that, The formula for calculating the position value of a single leaf during photography in step S33 is as follows: NA = L / M + 1; Where L is the blade length and M is the photo interval distance; The formula for calculating the height value of any shooting point on any leaf is: zi = (i*M)*sin(α*π / 180); i represents the i-th point of a single blade, and α represents the angle between two perpendicular planes.

5. The automatic wind turbine inspection method based on edge devices according to claim 4, characterized in that, The process of obtaining the horizontal plane coordinates of the blade based on the horizontal projection value and the orientation angle includes: The horizontal projection value of the i-th point is obtained using the following formula: Li = (i*M)*cos(α*π / 180); The horizontal coordinates are: xi=(i*M)*cos(α*π / 180)*sin(θ); yi=(i*M)*cos(α*π / 180)*cos(θ); θ is the orientation angle transformed relative to the Earth coordinate system.

6. The automatic wind turbine inspection method based on edge devices according to claim 5, characterized in that, The method of obtaining the actual coordinate position of the blade based on the horizontal plane coordinates of the blade and the actual latitude and longitude of the Earth includes: The formula for converting the horizontal plane coordinates of the blade to actual Earth latitude and longitude is as follows: Xi = xi / EARMH_RADIUS; Yi=yi / EARMH_RADIUS*cos(X*π / 180); Zi = zi; EARMH_RADIUS is the Earth's radius, and X is the X value of the three-dimensional height coordinate; The coordinates of the i-th point are (X+Xi, Y+Yi, Z+Zi); Where (X, Y, Z) are the three-dimensional height coordinates.

7. An automatic wind turbine inspection system based on edge devices, characterized in that, include: The first acquisition module acquires a frontal image of the wind turbine at any stopping position using a camera configured on the drone. The frontal image includes the hub and three blades of the wind turbine. The second acquisition module acquires the actual distance between the drone and the wind turbine hub through the laser device configured on the drone, and transmits the actual distance, drone parameters and environmental parameters to the edge computing device of the drone remote controller; The recognition module is used to transmit the frontal image to an edge computing device for image recognition and to obtain the optimal inspection route.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the automatic wind turbine inspection method based on an edge device as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the automatic wind turbine inspection method based on edge devices as described in any one of claims 1-6.

Images