Air route planning method and device for non-stop routing inspection of fan

Abstract

The invention discloses a route planning method and device for non-stop inspection of a fan, relates to the technical field of fan inspection, and mainly aims to perform accurate route planning on non-stop inspection of the fan and improve the efficiency and accuracy of non-stop inspection of fan blades. According to the main technical scheme, the method comprises the following steps: determining a shooting course corresponding to a mobile shooting terminal according to a windward angle of a target fan, wherein the windward angle is determined by performing key area identification and line connection on radar point cloud data of the target fan; determining waypoint dwell time according to the rotating speed of the target fan, wherein the rotating speed is determined based on position deviation calculation of adjacent images in the blade image data of the target fan; determining the number of waypoints and the distance between the waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera; and deploying shooting waypoints in the shooting course according to the waypoint number, the waypoint spacing and the waypoint staying time to obtain a shooting route of the mobile shooting terminal. The method is used for non-stop routing inspection of the fan.

Description

Technical Field

[0001] The present application relates to the technical field of wind turbine inspection, and in particular to a route planning method and device for non-stop inspection of wind turbines. Background Art

[0002] With the development of renewable energy, wind power generation has been widely used as a clean and sustainable form of energy. The maintenance and inspection of wind turbines are key links to ensure the normal operation of wind power generation systems. Traditional wind turbine inspections usually require shutdown, that is, the wind turbine blades are inspected in a locked state, which not only affects the power generation efficiency, but also increases maintenance costs. Based on this, the use of mobile shooting terminals such as drones to conduct non-stop inspections of wind turbine blades has become the mainstream inspection method.

[0003] At present, most of the existing non-stop inspections of wind turbine blades remain in the manual control stage, that is, the wind turbine blades are photographed by manually controlling mobile shooting terminals such as drones. Although this method can capture images of wind turbine blades, it lacks precise route planning, resulting in image quality being easily affected by shooting angles, positions, timing, etc., thereby reducing the efficiency and accuracy of non-stop inspections of wind turbine blades. Summary of the invention

[0004] In view of the above problems, the present application provides a route planning method and device for non-stop inspection of wind turbines, the main purpose of which is to accurately plan the route for non-stop inspection of wind turbines and improve the efficiency and accuracy of non-stop inspection of wind turbine blades.

[0005] In order to solve the above technical problems, this application proposes the following solutions:

[0006] In a first aspect, the present application provides a route planning method for non-stop inspection of a wind turbine, the method comprising:

[0007] Determine the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine, wherein the windward angle is determined based on identifying and connecting key areas of the radar point cloud data of the target wind turbine;

[0008] Determining a waypoint dwell time according to a rotation speed of the target wind turbine, wherein the rotation speed is determined based on position deviation calculation of adjacent images in blade image data of the target wind turbine;

[0009] Determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera;

[0010] The shooting waypoints are deployed on the shooting direction according to the number of waypoints, the distance between waypoints and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal.

[0011] In a second aspect, the present application provides a route planning device for non-stop inspection of a wind turbine, the device comprising:

[0012] A first determination unit is used to determine the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine, wherein the windward angle is determined based on key area identification and connection of radar point cloud data of the target wind turbine;

[0013] A second determination unit, configured to determine a waypoint stay time according to a rotation speed of the target wind turbine, wherein the rotation speed is determined based on position deviation calculation of adjacent images in blade image data of the target wind turbine;

[0014] A third determining unit, configured to determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera;

[0015] A planning unit is used to deploy shooting waypoints on the shooting direction obtained by the first determining unit according to the number of waypoints obtained by the third determining unit, the waypoint spacing and the waypoint stay time obtained by the second determining unit to obtain the shooting route of the mobile shooting terminal.

[0016] In order to achieve the above-mentioned purpose, according to the third aspect of the present application, a storage medium is provided, the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the route planning method for non-stop inspection of wind turbines of the first aspect. In order to achieve the above-mentioned purpose, according to the fourth aspect of the present application, a processor is provided, the processor is used to run the program, wherein when the program is running, the route planning method for non-stop inspection of wind turbines of the first aspect is executed.

[0017] By means of the above-mentioned technical scheme, the present application provides a route planning method and device for non-stop inspection of wind turbines. When route planning is required for non-stop inspection of wind turbines, the shooting direction corresponding to the mobile shooting terminal is first determined according to the windward angle of the target wind turbine, and the windward angle is determined based on key area identification and connection of radar point cloud data of the target wind turbine. Then, the waypoint dwell time is determined according to the rotation speed of the target wind turbine, and the rotation speed is determined based on position deviation calculation of adjacent images in the blade image data of the target wind turbine. Then, the number of waypoints and the distance between waypoints are determined according to the shooting range of the mobile shooting terminal and the blade length of the target camera. Finally, the shooting waypoints are deployed on the shooting direction according to the number of waypoints, the distance between waypoints and the dwell time of waypoints to obtain the shooting route of the mobile shooting terminal. The technical solution provided by the present application determines the shooting direction according to the windward angle of the wind turbine, so that the shooting direction is always located in front of or behind the wind turbine, so that the shooting angle on the shooting direction is always optimal, ensuring that the mobile shooting terminal can comprehensively and clearly capture various areas of the wind turbine blades, and determines the waypoint dwell time according to the rotation speed, so that the best time can be selected for shooting during the rotation of the blades to ensure the clarity and integrity of the image, and determines the number of waypoints and the distance between waypoints according to the shooting range and the length of the blades, which can improve the rationality of the waypoint deployment, avoid repeated shooting, and improve the shooting efficiency, and by combining the waypoint dwell time, the number of waypoints and the distance between waypoints to obtain the shooting route on the shooting direction, it can effectively avoid the image quality being affected by the shooting angle, position, timing, etc., thereby ensuring the image quality of the wind turbine blades, and thus improving the efficiency and accuracy of non-stop inspections of the wind turbine blades.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1 A flow chart of a route planning method for non-stop inspection of a wind turbine provided in an embodiment of the present application is shown;

[0021] Figure 2 A flow chart of another route planning method for non-stop inspection of wind turbines provided in an embodiment of the present application is shown;

[0022] Figure 3A block diagram showing a route planning device for non-stop inspection of wind turbines provided in an embodiment of the present application is shown;

[0023] Figure 4 A block diagram showing the composition of another route planning device for non-stop wind turbine inspection provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0025] At present, most of the existing non-stop inspections of wind turbine blades remain in the manual control stage, that is, the wind turbine blades are photographed by manually controlling mobile shooting terminals such as drones. Although this method can capture images of wind turbine blades, it lacks precise route planning, resulting in image quality being easily affected by shooting angles, positions, timing, etc., thereby reducing the efficiency and accuracy of non-stop inspections of wind turbine blades.

[0026] After research, the inventor found that the shooting route can be planned by combining the windward angle, rotation speed, and shooting range of the mobile shooting terminal of the wind turbine, that is, the heading and shooting direction are determined by the windward angle, the dwell time of the waypoint is determined by the rotation speed, and the spacing and number of waypoints are deployed by the shooting range of the mobile shooting terminal and the actual length of the blade, and the shooting route of the wind turbine blades is accurately planned by combining the heading, spacing, number, and dwell time. In this way, by combining the waypoint dwell time, the number of waypoints, and the distance between waypoints to obtain the shooting route on the shooting heading, the image quality can be effectively prevented from being affected by the shooting angle, position, timing, etc., and the image quality of the wind turbine blades can be guaranteed, thereby improving the efficiency and accuracy of non-stop inspection of the wind turbine blades.

[0027] Based on the above considerations, the embodiment of the present application provides a route planning method for non-stop inspection of wind turbines, which is applied to a mobile shooting terminal, which is specifically a drone with shooting function. The mobile shooting terminal includes a laser radar and a gimbal camera. The laser radar is used to collect radar point cloud data, and the gimbal camera is used to capture blade image data. Through this method, accurate route planning can be performed for non-stop inspection of wind turbines, thereby improving the efficiency and accuracy of non-stop inspection of wind turbine blades. The specific execution steps are as follows: Figure 1 As shown, including:

[0028] 101. Determine the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine.

[0029] Among them, the windward angle is determined based on the key area identification and connection of the radar point cloud data of the target wind turbine.

[0030] In this step, the mobile shooting terminal can collect radar point cloud data from a bird's-eye view directly above the target wind turbine. Specifically, the laser radar on the mobile shooting terminal can be used to collect point cloud data of the target wind turbine, and the point cloud data includes the three-dimensional coordinate information of the cabin and the blades. The key areas are identified through clustering algorithms, DBSCAN and other point cloud data processing technologies. The key areas include the cabin area and the blade area, which can be specifically identified through feature extraction. The automatic line finding algorithm is used to connect the key points in the cabin area and the blade area. The key points include the center point, endpoints, etc. The cabin orientation line and the blade orientation line are obtained by connecting the center point and the endpoint. By calculating the angle of the line relative to the true north direction (geographic north direction), the windward angle of the wind turbine is obtained.

[0031] After obtaining the windward angle, since the cabin direction and the blade direction are both known, the cabin direction can be used as the y-axis, the blade direction as the x-axis, and the z-axis perpendicular to the x-axis-y-axis plane to establish a spatial coordinate system. The x-axis is used as the moving heading, and the shooting direction of the mobile shooting terminal is rotated and adjusted according to the windward angle to determine the shooting heading.

[0032] 102. Determine the waypoint stay time based on the speed of the target wind turbine.

[0033] The rotation speed is determined based on position deviation calculation of adjacent images in the blade image data of the target wind turbine.

[0034] In this step, the mobile shooting terminal can capture continuous images of the wind turbine blades in front of or behind the target wind turbine. Specifically, a pan-tilt camera can be used to shoot the movement of the wind turbine blades at a higher frequency (eg, 30 frames per second).

[0035] Select an easily identifiable feature point on the fan blade as the target position, such as the tip of the blade or a distinct mark. Use feature point tracking algorithms such as the Lucas-Kanade optical flow method and the KLT tracking algorithm to track the selected feature point in continuous adjacent images. Since the fan does not stop, that is, the blades are rotating, there is a position deviation of the target position in the adjacent images. Therefore, the position deviation of the target position in the adjacent images can be calculated, such as calculating the rotation arc length. The rotation speed of the fan blade is calculated by the rotation arc length and the shooting time interval of adjacent images.

[0036] Since one or more groups of images of the same area on different blades need to be taken at the same waypoint, the shooting requirements can be set in advance, such as the number of groups of blades to be photographed at each waypoint. The waypoint stay time is finally determined based on the shooting requirements and the rotation speed.

[0037] For example, assuming that the rotation period of the blade is determined to be 6 seconds by the rotation speed, if one set of blade images needs to be taken at each waypoint, the waypoint stay time is 6 seconds, and if three sets of blade images need to be taken at each waypoint, the waypoint stay time is 18 seconds.

[0038] 103. Determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera.

[0039] In this step, the shooting range is used to characterize the effective distance covered by the mobile shooting terminal, that is, the effective field of view, which can be calculated by the focal length and sensor size of the mobile shooting terminal. It should be noted that when shooting a large area, the coverage of a single image is limited, and multiple images need to be stitched into a panoramic view. The blade length is usually long, and multiple waypoints may need to be set. Each waypoint shoots a part of the blade. Finally, the partial areas corresponding to all waypoints are stitched into a complete wind turbine blade. When stitching, it is necessary to set the overlap rate to ensure that there is enough overlap between adjacent images to facilitate image alignment and fusion. Therefore, the image overlap rate can be set in advance, usually about 20%, and the specific value can be customized by the user according to the accuracy requirements of the image. When the blade length is known, and the image overlap rate and the effective field of view are determined, the number of waypoints and the distance between waypoints can be calculated. Specifically:

[0040] Number of waypoints = blade length / effective field of view × (1-overlap rate);

[0041] Waypoint spacing = blade length × (1-overlap rate) / number of waypoints.

[0042] 104. Deploy shooting waypoints on the shooting direction according to the number of waypoints, the distance between waypoints, and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal.

[0043] In this step, the waypoint position can be determined on the shooting direction according to the calculated number of waypoints and the distance between waypoints. Specifically, each waypoint position can be numbered to ensure that the mobile shooting terminal performs the shooting task in sequence. At the same time, the waypoint stay time is added to the waypoint position as a timing tag to ensure that the mobile shooting terminal accurately controls the stay time and shooting timing of each waypoint. Combined with the flight speed and waypoint distance of the mobile shooting terminal, the movement time between adjacent waypoints is calculated, and all waypoints and their timing tags are combined into a waypoint sequence. The shooting route of the drone is generated based on the waypoint sequence to ensure that the mobile shooting terminal shoots according to the predetermined path and time.

[0044] Based on the above Figure 1It can be seen from the implementation method that the present application provides a route planning method for non-stop inspection of wind turbines. When route planning is required for non-stop inspection of wind turbines, the shooting direction corresponding to the mobile shooting terminal is first determined according to the windward angle of the target wind turbine, and the windward angle is determined based on the key area identification and connection of the radar point cloud data of the target wind turbine. Then, the waypoint dwell time is determined according to the rotation speed of the target wind turbine, and the rotation speed is determined based on the position deviation calculation of adjacent images in the blade image data of the target wind turbine. Then, the number of waypoints and the distance between waypoints are determined according to the shooting range of the mobile shooting terminal and the blade length of the target camera. Finally, the shooting waypoints are deployed on the shooting direction according to the number of waypoints, the distance between waypoints and the dwell time of waypoints to obtain the shooting route of the mobile shooting terminal. The technical solution provided by the present application determines the shooting direction according to the windward angle of the wind turbine, so that the shooting direction is always located in front of or behind the wind turbine, thereby making the shooting angle on the shooting direction always optimal, ensuring that the mobile shooting terminal can comprehensively and clearly capture various areas of the wind turbine blades, and determines the waypoint dwell time according to the rotation speed, so that the best time can be selected for shooting during the rotation of the blades to ensure the clarity and integrity of the image, and determines the number of waypoints and the distance between waypoints according to the shooting range and blade length, which can improve the rationality of waypoint deployment, avoid repeated shooting, and improve shooting efficiency, and by combining the waypoint dwell time, the number of waypoints and the distance between waypoints to obtain the shooting route on the shooting direction, it can effectively avoid the image quality being affected by the shooting angle, position, timing, etc., thereby ensuring the image quality of the wind turbine blades, and thereby improving the efficiency and accuracy of non-stop inspections of the wind turbine blades.

[0045] Further, the preferred embodiment of the present application is in the above Figure 1 Based on this, a detailed description of the route planning process for non-stop inspection of wind turbines is given. The specific steps are as follows: Figure 2 As shown, including:

[0046] 201. Collect radar point cloud data of a target wind turbine at a first location.

[0047] The first position is located directly above the target wind turbine.

[0048] In this step, more complete and clearly distributed radar point cloud data can be collected from directly above the wind turbine, which means that all parts of the wind turbine can be fully covered. Therefore, based on the height of the wind turbine and the flight capability of the mobile shooting terminal, a suitable flight altitude is selected, usually 10-50 meters above the top of the wind turbine, as the first position. When the mobile shooting terminal flies and hovers at this position, the laser radar is used to collect the point cloud data of the wind turbine, including the three-dimensional coordinate information of the cabin and blades, and the collected point cloud data is stored in the storage device of the mobile shooting terminal, or transmitted to the ground station by wireless.

[0049] It should be noted that when using a laser radar to collect point cloud data of a wind turbine, the yaw angle of the mobile shooting terminal can be controlled to be 0, so that the windward angle can be directly calculated in step 202.

[0050] 202. Identify and segment the key areas in the radar point cloud data, and calculate the windward angle of the target wind turbine according to the connecting lines of the key areas.

[0051] Among them, the windward angle is the angle between the cabin direction and the north direction, and the key areas include the cabin area and the blade area.

[0052] It should be noted that the specific execution process of identifying and segmenting the key areas in the radar point cloud data and calculating the windward angle of the target wind turbine according to the connecting lines of the key areas is as follows: identifying and segmenting the cabin area and the blade area according to the point cloud distribution corresponding to the radar point cloud data; connecting key points between the cabin area and the blade area using an automatic line-finding algorithm, the key point connecting lines include the connecting lines representing the cabin orientation and the connecting lines representing the blade orientation; determining the cabin direction of the target wind turbine based on the key point connecting lines, and determining the windward angle based on the cabin direction and the north direction.

[0053] In this step, the collected point cloud data is imported into the point cloud processing software built into the mobile shooting terminal or installed in the ground station, such as CloudCompare, PCL (Point Cloud Library), etc. It is denoised and segmented by the point cloud processing software. The radar point cloud data is divided into multiple clusters by using clustering algorithms such as DBSCAN or K-means. Since the cabin and the blade have different characteristics, for example, the cabin is usually a larger and more regular area, and the cabin is usually rectangular or elliptical, while the blade is usually longer and thinner, and the width is relatively small. Therefore, by extracting key features such as size features, shape features, length features, width features, etc., the cabin area can be identified according to the size features and shape features, and the blade area can be identified according to the length features and width features. After determining the cabin area and the blade area, the corresponding key points can be extracted. For example, the cabin area can extract the center point (center of mass of the cabin area) and the front end point (the point farthest from the center of mass in the cabin area), and the blade area can extract the center point (calculate the center of mass of the blade area) and the end point (the point farthest from the center of mass in the blade area). The key points are connected by automatic line finding algorithms such as Canny edge detection and Hough transform to obtain the cabin orientation line and the blade orientation line, that is, the cabin direction vector and the blade direction vector. Since the due north direction vector corresponding to the due north direction is determined, the windward angle can be calculated by the dot product formula.

[0054] 203. Construct a spatial coordinate system based on the cabin direction.

[0055] The y-axis of the spatial coordinate system is the direction of the cabin, the x-axis is the direction of the blade, and the z-axis is perpendicular to the direction of the x-axis-y-axis plane. In this step, since the cabin direction vector of the y-axis and the blade direction vector of the x-axis are determined, the direction vector of the z-axis can be obtained by cross product calculation. The y-axis, x-axis, and z-axis are standardized to obtain a spatial coordinate system, so as to execute step 204.

[0056] 204. Taking the direction of the x-axis as the moving heading of the mobile shooting terminal, adjusting the shooting direction of the mobile shooting terminal to be opposite to the target wind turbine by rotating the mobile shooting terminal at the windward angle, and obtaining the shooting heading.

[0057] In this step, since the direction of the x-axis is parallel to the direction of the blades, the direction of the x-axis is used as the moving heading of the mobile shooting terminal, so that the mobile shooting terminal can capture the complete wind turbine blades as it moves. In addition, the windward angle refers to the angle between the cabin direction and the north direction. Based on this angle, the yaw angle of the mobile shooting terminal is used, that is, the shooting direction is rotated according to this angle, so that the shooting direction is always facing the wind turbine blades. Specifically, the shooting direction of the mobile shooting terminal is adjusted according to the windward angle θ and the rotation angle of the north direction, and the shooting direction of the mobile shooting terminal is rotated from the north direction by an angle of θ using a rotation matrix. The calculated moving heading and the rotated shooting direction are used as the shooting heading to ensure that the mobile shooting terminal can fully and clearly capture all parts of the wind turbine blades on the shooting heading, thereby ensuring the image quality during subsequent shooting.

[0058] It should be noted that, in this step, before the shooting direction of the mobile shooting terminal is adjusted to be relative to the target wind turbine by rotating the windward angle, since the windward angle changes dynamically with the wind direction, that is, it will be affected by the wind direction, in order to ensure that the shooting direction is facing the blades of the target wind turbine. The wind direction can also be obtained in real time, and the influence of the wind direction on the windward angle can be quantified. The shooting direction of the mobile shooting terminal can be rotated and adjusted together with the windward angle according to the influence degree to ensure the accuracy of the shooting direction. Specifically, the real-time wind direction is obtained from the meteorological station or the wind speed and wind direction sensors on the mobile shooting terminal measure the wind direction in real time, and the angle of the wind direction is calculated, that is, the angle between the wind direction and the north direction. A wind direction influence factor is predefined to indicate the influence of the wind direction on the windward angle, and the wind direction influence angle is added to the original windward angle to obtain the adjusted windward angle, and the shooting direction of the mobile shooting terminal is adjusted to be relative to the target wind turbine by rotating the adjusted windward angle.

[0059] 205. Capture blade image data of the target wind turbine at a second position.

[0060] The second position is located directly in front of or directly behind the target wind turbine.

[0061] In this step, since the blades are usually perpendicular to the ground and when they rotate without stopping, the movement of the blades can only be clearly captured in front of or behind them. Therefore, according to the height of the wind turbine and the flight capability of the mobile shooting terminal, a suitable flight altitude can be selected, usually the middle or slightly higher position of the wind turbine blades, as the second position. It should be noted that the second position is deployed after the windward angle is determined. When the mobile shooting terminal flies and hovers at this position, a gimbal camera is used to capture continuous images of the blades at a higher frequency (for example, 30 frames per second), and the collected image data is stored in the storage device of the mobile shooting terminal, or transmitted to a ground station via wireless.

[0062] 206. Calculate the rotation speed based on the position deviation and time interval corresponding to adjacent images in the blade image data.

[0063] The position deviation is the position deviation of the target position corresponding to the same feature point in adjacent images, which can be specifically represented by the rotation arc length. The specific execution process of determining the rotation speed based on the position deviation and time interval corresponding to adjacent images in the blade image data is as follows: selecting the same target position in adjacent images; calculating the rotation arc length corresponding to the same target position in adjacent images; and determining the rotation speed according to the rotation arc length and time interval.

[0064] In this step, the collected blade image data is imported into the image processing software built into the mobile shooting terminal or installed in the ground station, such as OpenCV, MATLAB, etc. The image processing software is used to perform denoising and feature point extraction. Among them, denoising can use a filtering algorithm to remove noise in the image to improve the image quality, and feature point extraction can select an easily identifiable feature point on the fan blade as the target position, such as the tip of the blade or a clear mark. Use feature point tracking algorithms such as the Lucas-Kanade optical flow method and the KLT tracking algorithm to track the selected feature points in continuous adjacent images, and the rotation arc length can be obtained by feature point tracking. Since the shooting time interval of adjacent images is fixed, the ratio of the rotation arc length to the time interval is the speed of the fan. The specific formula is: R = L / Δt. After obtaining the speed, it can be converted into the required revolutions per minute (RPM) or revolutions per second (RPS).

[0065] 207. Obtain preset shooting requirements.

[0066] The shooting requirement is used to represent the number of groups of blades to be photographed at the same waypoint.

[0067] In this step, the shooting requirements can be pre-stored in the storage device of the mobile shooting terminal or the ground station, etc. The user can define the number of shooting groups by setting the configuration page or interface, that is, the specific number of groups to be shot for the same area of ​​different leaves. The number of groups can be determined by the number of images that need to be shot for the same area of ​​different leaves at the same waypoint.

[0068] 208. Determine the waypoint stay time based on shooting requirements and rotation speed.

[0069] In this step, the rotation cycle of the fan blades can be determined by the rotation speed, and both single-group shooting and full-group shooting can capture all of them within one rotation cycle. The difference is that full-group shooting needs to stay for one rotation cycle, while single-group shooting does not. Therefore, the total shooting time of each waypoint can be calculated based on the number of shooting groups and the rotation cycle, that is, the waypoint stay time, which is specifically: stay time = number of groups × rotation cycle.

[0070] In addition, in combination with step 207, the shooting time and shooting interval of a single image may be added to the shooting requirements. Thus, the waypoint stay time calculation in this step can be combined with the shooting time and shooting interval of a single image, thereby improving the accuracy of shooting.

[0071] 209. Determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera.

[0072] This step is combined with the description of step 103 in the above method, and the same content is not repeated here. It should be noted that the specific execution process of determining the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera is: determining the effective field of view angle according to the shooting range, and the effective field of view angle is used to characterize the effective distance covered by the mobile shooting terminal when shooting; obtaining the blade length and the preset image overlap rate; calculating the number of waypoints and the distance between waypoints based on the blade length, the effective field of view angle and the image overlap rate.

[0073] In this step, the shooting range refers to the maximum distance that the gimbal camera in the mobile shooting terminal can cover, and the effective field of view is used to characterize the effective distance covered by the gimbal camera in the mobile shooting terminal when shooting. The blade length is known, and the image overlap rate is also pre-set. Therefore, after obtaining the blade length and the image overlap rate, the blade length is used as the total distance, and the image overlap rate is introduced into the effective field of view. The number of waypoints required can be calculated, and after obtaining the number of waypoints, the required waypoint spacing can be calculated.

[0074] For example, assuming that the blade length is 50 meters, the effective viewing angle is 10 meters, and the image overlap rate is 20%, then:

[0075] Calculate the number of waypoints: N = 50 / 10×(1-0.2) = 50 / 10×0.8 = 508≈6.25. Since the number of waypoints must be an integer, N = 6.

[0076] Calculate the waypoint spacing:

[0077] S = 50 × (1-0.2) / 6 = 50 × 0.86 = 40 / 6 ≈ 6.67 meters.

[0078] 210. Deploy shooting waypoints on the shooting direction according to the number of waypoints, the distance between waypoints, and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal.

[0079] This step is combined with the description of step 104 in the above method, and the same content is not repeated here. It should be noted that the specific execution process of deploying shooting waypoints on the shooting direction according to the number of waypoints, the distance between waypoints and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal is: determining the waypoint position on the shooting direction according to the number of waypoints and the distance between waypoints; adding the waypoint stay time as a time sequence tag to the waypoint position to obtain the shooting waypoint; generating the shooting route of the mobile shooting terminal based on the shooting waypoints.

[0080] In this step, since the shooting heading is determined, the waypoint position of the shooting heading can be determined based on the number of waypoints and the distance between waypoints. Specifically, since the y-axis in the spatial coordinate system is the direction of the cabin, the x-axis is the direction of the blade, and the z-axis is the direction perpendicular to the x-axis-y-axis plane and pointing to the sky, a starting position can be determined in the spatial coordinate system, usually the starting point or end point of the wind turbine blade. Assuming the starting position is P0 (0,0,0), the position Pi of the i-th waypoint is: Pi = P0 + i × S, where i is from 0 to N-1. If the number of waypoints N is 6 and the waypoint spacing S is 6.67 meters, the waypoint positions are: P0 = (0,6.67,0), P1 = (0,6.67,0), P2 = (0,13.34,0), P3 = (0,20.01,0), P4 = (0,26.68,0), P5 = (0,33.35,0). All waypoints and their time tags are combined into a waypoint sequence, and the shooting route of the drone is generated based on the waypoint sequence to ensure that the mobile shooting terminal shoots according to the predetermined path and time.

[0081] Furthermore, as a response to the above Figure 1-2The implementation of the method embodiment shown, the embodiment of the present application provides a route planning device for non-stop inspection of wind turbines, which is applied to a mobile shooting terminal, and the mobile shooting terminal is specifically a drone with a shooting function. The mobile shooting terminal includes a laser radar and a gimbal camera, the laser radar is used to collect radar point cloud data, and the gimbal camera is used to capture blade image data. The device is used to accurately plan routes for non-stop inspections of wind turbines, thereby improving the efficiency and accuracy of non-stop inspections of wind turbine blades. The embodiment of the device corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will no longer repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to the implementation of all the contents of the aforementioned method embodiment. Specifically, Figure 3 As shown, the device comprises:

[0082] A first determining unit 31 is used to determine a shooting direction corresponding to the mobile shooting terminal according to a windward angle of a target wind turbine, wherein the windward angle is determined based on key area identification and connection of radar point cloud data of the target wind turbine;

[0083] A second determination unit 32, configured to determine a waypoint stay time according to a rotation speed of the target wind turbine, wherein the rotation speed is determined based on position deviation calculation of adjacent images in the blade image data of the target wind turbine;

[0084] A third determining unit 33 is used to determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera;

[0085] The planning unit 34 is used to deploy shooting waypoints on the shooting direction obtained by the first determining unit 31 according to the number of waypoints obtained by the third determining unit 33, the waypoint spacing and the waypoint stay time obtained by the second determining unit 32, so as to obtain the shooting route of the mobile shooting terminal.

[0086] Further, such as Figure 4 As shown, the first determining unit 31 includes:

[0087] A collection module 311 is used to collect radar point cloud data of the target wind turbine at a first position, where the first position is located directly above the target wind turbine;

[0088] The processing module 312 is used to identify the key areas in the radar point cloud data obtained by the acquisition module 311, and calculate the windward angle of the target wind turbine according to the connection lines between the key areas, where the windward angle is the angle between the cabin direction and the due north direction;

[0089] A construction module 313 is used to construct a spatial coordinate system based on the nacelle direction, wherein the y-axis of the spatial coordinate system is the nacelle direction, the x-axis is the blade direction, and the z-axis is perpendicular to the direction of the x-axis-y-axis plane;

[0090] The first determination module 314 is used to use the direction of the x-axis as the moving heading of the mobile shooting terminal, and to adjust the shooting direction of the mobile shooting terminal to be opposite to the target wind turbine by the windward angle to obtain the shooting heading.

[0091] Further, such as Figure 4 As shown, the processing module 312 is specifically used for:

[0092] Identify and segment the cabin area and the blade area according to the point cloud distribution corresponding to the radar point cloud data;

[0093] Using an automatic line-finding algorithm to connect key points between the nacelle area and the blade area, the key point connection lines include a line representing the nacelle orientation and a line representing the blade orientation;

[0094] The nacelle direction of the target wind turbine is determined based on the key point connection line, and the windward angle is determined according to the nacelle direction and the due north direction.

[0095] Further, such as Figure 4 As shown, the second determining unit 32 includes:

[0096] A capture module 321 is used to capture blade image data of the target wind turbine at a second position, where the second position is located directly in front of or directly behind the target wind turbine;

[0097] A first calculation module 322, configured to calculate the rotation speed based on the position deviation and time interval corresponding to adjacent images in the blade image data obtained by the capture module 321;

[0098] A first acquisition module 323 is used to acquire a preset shooting requirement, where the shooting requirement is used to represent the number of groups of blades to be photographed at the same waypoint;

[0099] The second determination module 324 is used to determine the waypoint stay time according to the shooting requirement obtained by the first acquisition module 313 and the rotation speed obtained by the first calculation module 322 .

[0100] Further, such as Figure 4 As shown, the first calculation module 322 is specifically used for:

[0101] Selecting the same target position in the adjacent images;

[0102] Calculate the rotation arc length corresponding to the same target position in the adjacent images;

[0103] The rotation speed is determined based on the rotation arc length and the time interval.

[0104] Further, such as Figure 4 As shown, the third determining unit 33 includes:

[0105] A third determining module 331 is used to determine an effective viewing angle according to the shooting range, where the effective viewing angle is used to represent an effective distance covered by the mobile shooting terminal during shooting;

[0106] A second acquisition module 332 is used to acquire the blade length and a preset image overlap rate;

[0107] The second calculation module 333 is used to calculate the number of waypoints and the distance between waypoints based on the blade length obtained by the second acquisition module 332 , the effective viewing angle obtained by the third determination module 331 , and the image overlap rate obtained by the second acquisition module 332 .

[0108] Further, such as Figure 4 As shown, the planning unit 34 includes:

[0109] A fourth determining module 341 is used to determine the waypoint positions on the shooting direction according to the number of waypoints and the distance between waypoints;

[0110] An adding module 342, configured to add the waypoint stay time as a time sequence tag to the waypoint position obtained by the fourth determining module 341, to obtain the shooting waypoint;

[0111] The generating module 343 is used to generate the shooting route of the mobile shooting terminal based on the shooting waypoints obtained by the adding module 342.

[0112] Furthermore, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the above Figure 1-2 The route planning method for non-stop inspection of wind turbines described in.

[0113] Furthermore, the embodiment of the present application also provides a processor, the processor is used to run a program, wherein the program executes the above Figure 1-2 The route planning method for non-stop inspection of wind turbines described in.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] It is understandable that the related features in the above methods and devices can be referenced to each other. In addition, the "first", "second" and the like in the above embodiments are used to distinguish the embodiments, and do not represent the advantages and disadvantages of the embodiments.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the application is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the application described herein, and the description of the specific language above is for the purpose of disclosing the best mode of implementation of the application.

[0118] In addition, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0124] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0125] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0126] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0128] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A route planning method for non-stop inspection of wind turbines, characterized in that: Applied to a mobile shooting terminal, the method includes: Determine the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine, wherein the windward angle is determined based on identifying and connecting key areas of the radar point cloud data of the target wind turbine; Determining a waypoint dwell time according to a rotation speed of the target wind turbine, wherein the rotation speed is determined based on position deviation calculation of adjacent images in blade image data of the target wind turbine; Determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera; The shooting waypoints are deployed on the shooting direction according to the number of waypoints, the distance between waypoints and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal.

2. The method according to claim 1, characterized in that Determining the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine includes: Collecting radar point cloud data of the target wind turbine at a first position, where the first position is located directly above the target wind turbine; Identify and segment the key areas in the radar point cloud data, and calculate the windward angle of the target wind turbine according to the connecting lines between the key areas, where the windward angle is the angle between the cabin direction and the due north direction; A spatial coordinate system is constructed based on the nacelle direction, wherein the y-axis of the spatial coordinate system is the nacelle direction, the x-axis is the blade direction, and the z-axis is perpendicular to the direction of the x-axis-y-axis plane; The direction of the x-axis is used as the moving heading of the mobile shooting terminal, and the shooting direction of the mobile shooting terminal is adjusted by the windward angle to obtain the shooting heading.

3. The method according to claim 2, characterized in that Identifying key areas in the radar point cloud data, and calculating the windward angle of the target wind turbine according to the connecting lines between the key areas, including: Identify and segment the cabin area and the blade area according to the point cloud distribution corresponding to the radar point cloud data; Using an automatic line-finding algorithm to connect key points between the nacelle area and the blade area, the key point connection lines include a line representing the nacelle orientation and a line representing the blade orientation; The nacelle direction of the target wind turbine is determined based on the key point connection line, and the windward angle is determined according to the nacelle direction and the due north direction.

4. The method according to claim 1, characterized in that: Determining the waypoint stay time according to the rotation speed of the target wind turbine includes: Capturing blade image data of the target wind turbine at a second position, where the second position is located directly in front of or directly behind the target wind turbine; Calculating the rotation speed based on position deviations and time intervals corresponding to adjacent images in the blade image data; Acquire a preset shooting requirement, where the shooting requirement is used to represent the number of groups of blades to be photographed at the same waypoint; The waypoint stay time is determined according to the shooting requirement and the rotation speed.

5. The method according to claim 4, characterized in that Determining the rotation speed based on position deviations and time intervals corresponding to adjacent images in the blade image data includes: Selecting the same target position in the adjacent images; Calculate the rotation arc length corresponding to the same target position in the adjacent images; The rotation speed is determined based on the rotation arc length and the time interval.

6. The method according to claim 1, characterized in that Determining the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera includes: Determine an effective viewing angle according to the shooting range, where the effective viewing angle is used to characterize the effective distance covered by the mobile shooting terminal during shooting; Obtaining the blade length and a preset image overlap rate; The number of waypoints and the distance between waypoints are calculated based on the blade length, the effective viewing angle, and the image overlap rate.

7. The method according to any one of claims 1 to 6, characterized in that Deploying shooting waypoints on the shooting direction according to the number of waypoints, the distance between waypoints, and the stay time of waypoints to obtain the shooting route of the mobile shooting terminal includes: Determine the waypoint positions on the shooting direction according to the number of waypoints and the distance between waypoints; Adding the waypoint stay time as a time sequence tag to the waypoint position to obtain the shooting waypoint; The shooting route of the mobile shooting terminal is generated based on the shooting waypoints.

8. A route planning device for non-stop inspection of wind turbines, characterized in that: Applied to a mobile shooting terminal, the device comprises: A first determination unit is used to determine the shooting direction corresponding to the mobile shooting terminal according to the windward angle of the target wind turbine, wherein the windward angle is determined based on key area identification and connection of radar point cloud data of the target wind turbine; A second determination unit, configured to determine a waypoint stay time according to a rotation speed of the target wind turbine, wherein the rotation speed is determined based on position deviation calculation of adjacent images in blade image data of the target wind turbine; A third determining unit, configured to determine the number of waypoints and the distance between waypoints according to the shooting range of the mobile shooting terminal and the blade length of the target camera; A planning unit is used to deploy shooting waypoints on the shooting direction obtained by the first determining unit according to the number of waypoints obtained by the third determining unit, the waypoint spacing and the waypoint stay time obtained by the second determining unit to obtain the shooting route of the mobile shooting terminal.

9. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the route planning method for non-stop inspection of a wind turbine as described in any one of claims 1 to 7.

10. A processor, characterized in that: The processor is used to run a program, wherein when the program is run, the route planning method for non-stop inspection of a wind turbine as described in any one of claims 1 to 7 is executed.

Images