Component Recognition Method, Device, Equipment and Medium Based on Wind Turbine Point Cloud Data

Abstract

The present invention discloses a component recognition method, device, equipment and medium based on fan point cloud data. The method includes: obtaining fan point cloud data, projecting the fan point cloud data onto a blade plane and performing skeleton extraction to obtain a fan skeleton image, where the fan skeleton image is a binary image; performing skeleton segmentation on the fan skeleton image by using a preset path finding method, and combining the characteristics of fan components to obtain a component fitting function for each fan component; dividing the fan point cloud data according to each component fitting function to obtain component point cloud data for each fan component. The present invention solves the problem that current computers cannot accurately identify fan components, resulting in a large amount of manpower required for fan inspection work, and realizes rapid and accurate division of fan point cloud data to obtain component point cloud data for each fan component, so as to facilitate precise inspection of each component, quickly complete the inspection task, save time, effort and money, and ensure the safe operation of wind turbine generators.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of image processing, and in particular, to a method, device, equipment and medium for component recognition based on wind turbine point cloud data. Background Art

[0002] With the development of technology, the operation and maintenance methods of new energy power generation enterprises such as wind power and photovoltaic are gradually transitioning from traditional manual inspections to automated and intelligent inspections. In this process, the increasingly mature unmanned aerial vehicle (UAV) technology has gradually become the main reliance of power production enterprises.

[0003] In recent years, domestic power inspection work mainly involves inspection personnel using a UAV flight platform as a carrier, equipped with a high-resolution visible light camera to observe and record the surface state of the blades, and at the same time transmit the images back to the operation and maintenance personnel. When a suspicious point is found, the inspection equipment is operated to further check in detail and collect high-definition images as records. UAV inspections can cover every detail of the wind turbine without omission, providing a full-range and multi-angle "physical examination" for the blades. Applying UAVs to the inspection of wind turbine blades can greatly improve the efficiency of visual inspection, reduce downtime losses, and prevent the expansion of blade failures.

[0004] Since the wind turbine images are collected by the UAV from various angles of the wind turbine, it is difficult for a computer to distinguish the wind turbine components in the images during problem troubleshooting. Therefore, at present, it mainly relies on manual inspection of the transmitted images to find possible faults of the wind turbine, and a large amount of manpower is still required for the inspection of the wind turbine. Summary of the Invention

[0005] The present invention provides a method, device, equipment and medium for component recognition based on wind turbine point cloud data to accurately identify the wind turbine component data in the point cloud data of the power generation wind turbine.

[0006] In a first aspect, an embodiment of the present invention provides a method for component recognition based on wind turbine point cloud data, including:

[0007] Obtain wind turbine point cloud data, project the wind turbine point cloud data onto the blade plane and perform skeleton extraction to obtain a wind turbine skeleton image, where the wind turbine skeleton image is a binary image;

[0008] Perform skeleton segmentation on the wind turbine skeleton image using a preset path finding method, and combine the characteristics of the wind turbine components to obtain the component fitting functions of each wind turbine component;

[0009] Divide the wind turbine point cloud data according to each component fitting function to obtain the component point cloud data of each wind turbine component.

[0010] Optionally, the step of projecting the wind turbine point cloud data onto the blade plane and performing skeleton extraction to obtain a wind turbine skeleton image includes:

[0011] Perform coordinate transformation on the point cloud data of the fan to obtain the point cloud data in the geodetic coordinate system;

[0012] Determine the blade plane according to the point cloud data in the geodetic coordinate system, and project the point cloud data in the geodetic coordinate system onto the blade plane to obtain the initial image of the fan;

[0013] Perform dilation operation and erosion operation on the initial image of the fan to obtain the skeleton image of the fan.

[0014] Optionally, the method of performing skeleton segmentation on the skeleton image of the fan by using a preset path finding method and combining the characteristics of fan components to obtain the component fitting functions of each fan component includes:

[0015] Perform skeleton division on the initial skeleton in the skeleton image of the fan by using a preset path finding method to obtain at least one set of initial path points, and perform burr filtering on the skeleton image of the fan according to each set of initial path points to obtain the target skeleton image;

[0016] Perform skeleton division on the target skeleton in the target skeleton image by using a preset path finding method to obtain at least one set of component path points;

[0017] Perform linear fitting on each set of component path points respectively to obtain the corresponding component fitting functions, and perform component name labeling on each component fitting function according to the characteristics of fan components.

[0018] Optionally, the steps of the preset path finding method include:

[0019] Determine all the end points of the skeleton in the image, where the image is the skeleton image of the fan or the target skeleton image, and the skeleton is the initial skeleton or the target skeleton;

[0020] For each end point, establish a set of path points, add the end point to the set of path points, use the end point as the path finding starting point, determine the number of connected points of the path point connected to the path finding starting point, if the number of connected points is equal to 1, add the path point to the set of path points, and re-determine the path point as the new path finding starting point to search for path points, where the set of path points is the set of initial path points or the set of component path points.

[0021] Optionally, the method for determining the end point includes:

[0022] For each non-edge pixel point in the image, divide a nine-pixel point set with the non-edge pixel point as the center, determine the total pixel value of the nine-pixel point set, and determine the non-edge pixel point corresponding to the nine-pixel point set with the total pixel value of 2 as the end point;

[0023] For each edge pixel point in the image, a six-pixel point set is divided with the edge pixel point as the center, the total pixel value of the six-pixel point set is determined, and the edge pixel point corresponding to the six-pixel point set with a total pixel value of 2 is determined as an end point;

[0024] For each corner pixel point in the image, a four-pixel point set is divided with the corner pixel point as the center, the total pixel value of the four-pixel point set is determined, and the corner pixel point corresponding to the four-pixel point set with a total pixel value of 2 is determined as an end point.

[0025] Optionally, the burr filtering of the fan skeleton image according to each of the initial path point sets to obtain a target skeleton image includes:

[0026] Determine the initial path point numbers of the initial path points included in each of the initial path point sets;

[0027] Filter out the initial path point sets with the initial path point numbers less than the burr point number threshold to obtain a target skeleton image.

[0028] Optionally, after obtaining the component point cloud data of each of the fan components, it further includes:

[0029] Optimize each of the component point cloud data to determine the component attitude information of each of the fan components.

[0030] In a second aspect, an embodiment of the present invention further provides a component recognition device based on fan point cloud data, and the device includes:

[0031] A skeleton image determination module, configured to obtain fan point cloud data, project the fan point cloud data onto a blade plane and perform skeleton extraction to obtain a fan skeleton image;

[0032] A component skeleton segmentation module, configured to perform skeleton segmentation on the fan skeleton image by using a preset path finding method, and combine the characteristics of the fan components to obtain the component fitting functions of each fan component;

[0033] A point cloud data recognition module, configured to divide the fan point cloud data according to each of the component fitting functions to obtain the component point cloud data of each of the fan components.

[0034] In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the component recognition method based on fan point cloud data as described in any embodiment of the present invention.

[0035] Fourthly, an embodiment of the present invention further provides a storage medium containing computer-executable instructions, which are used to execute the component recognition method based on wind turbine point cloud data as described in any embodiment of the present invention when executed by a computer processor.

[0036] By acquiring the wind turbine point cloud data, projecting the wind turbine point cloud data onto the blade plane and performing skeleton extraction to obtain a wind turbine skeleton image, using a preset path-finding method to segment the wind turbine skeleton image, combining the characteristics of wind turbine components to obtain the component fitting functions of each wind turbine component, and dividing the wind turbine point cloud data according to each component fitting function to obtain the component point cloud data of each wind turbine component, the present invention solves the problem that the current computer cannot accurately identify wind turbine components, resulting in a large amount of manpower required for wind turbine inspection work. It realizes the rapid and accurate division of wind turbine point cloud data to obtain the component point cloud data of each wind turbine component, so as to facilitate precise inspection of each wind turbine component, quickly complete the inspection task, save time, effort and money, and ensure the safe operation of the wind turbine unit at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flowchart of a component recognition method based on wind turbine point cloud data provided in Embodiment 1 of the present invention;

[0038] Figure 2a is a flowchart of a component recognition method based on wind turbine point cloud data provided in Embodiment 2 of the present invention;

[0039] Figure 2b is a first principle schematic diagram of a component recognition method based on wind turbine point cloud data provided in Embodiment 2 of the present invention;

[0040] Figure 2c is a second principle schematic diagram of a component recognition method based on wind turbine point cloud data provided in Embodiment 2 of the present invention;

[0041] Figure 2d is a third principle schematic diagram of a component recognition method based on wind turbine point cloud data provided in Embodiment 2 of the present invention;

[0042] Figure 2e is a fourth principle schematic diagram of a component recognition method based on wind turbine point cloud data provided in Embodiment 2 of the present invention;

[0043] Figure 3 is a structural block diagram of a component recognition device based on wind turbine point cloud data provided in Embodiment 3 of the present invention;

[0044] Figure 4 is a structural block diagram of a computer device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. In addition, it should be noted that for the convenience of description, only the parts related to the present invention rather than all the structures are shown in the drawings. In addition, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0046] Embodiment 1

[0047] Figure 1 As shown in the flowchart of a component recognition method based on fan point cloud data provided in Embodiment 1 of the present invention, this embodiment is applicable to the situation of component recognition of the fan point cloud data of a power generation fan. This method can be executed by a component recognition device based on fan point cloud data, and this device can be implemented by software and / or hardware.

[0048] As Figure 1 shown, the method specifically includes the following steps:

[0049] Step 110: Obtain the fan point cloud data, project the fan point cloud data onto the blade plane and perform skeleton extraction to obtain the fan skeleton image.

[0050] Among them, the fan point cloud data can be three-dimensional data collected by lidar. The fan skeleton image can be a binary image.

[0051] Specifically, during the inspection of the power generation fan, the drone lidar collects data, and after the data is uploaded to the on-board computer, data analysis is performed. The coordinate system change can be performed after obtaining the fan point cloud data, changing the point cloud data in the drone radar coordinate system to the geodetic coordinate system, and analyzing the fan point cloud data to determine the blade plane of the fan, projecting the three-dimensional point cloud data onto the blade plane to obtain a two-dimensional fan image. Continue to perform skeleton extraction operations such as dilation and erosion on the fan image to obtain the fan skeleton image.

[0052] Step 120: Perform skeleton segmentation on the fan skeleton image using a preset pathfinding method, and combine the characteristics of the fan components to obtain the component fitting functions of each fan component.

[0053] In this embodiment, for the fan skeleton graph in the fan skeleton image, a preset pathfinding method can be used to traverse the skeleton path from the skeleton endpoint, perform segmentation when there is a bifurcation in the skeleton path, and perform linear fitting on the segmented skeleton to obtain the component fitting functions of each fan component. In practical applications, a power generation fan generally has three blades, and the angle between each blade is 120°, and the fan tower is generally perpendicular to the ground. According to the above characteristics of the fan components, the fitting functions of the segmented component skeletons can be identified and labeled.

[0054] Optionally, the implementation steps of the preset pathfinding method may include:

[0055] a. Determine all endpoints of the skeleton in the image.

[0056] b. For each endpoint, establish a set of path points, add the endpoint to the set of path points, use the endpoint as the starting point for pathfinding, determine the number of connected points of the path point connected to the starting point for pathfinding. If the number of connected points is equal to 1, add the path point to the set of path points, and re-determine the path point as the new starting point for pathfinding to search for path points.

[0057] Step 130: Divide the wind turbine point cloud data according to the component fitting functions of each component to obtain the component point cloud data of each wind turbine component.

[0058] Specifically, the original wind turbine point cloud data can be classified correspondingly according to the component fitting functions of each wind turbine component to obtain the segmented and classified component point cloud data.

[0059] The technical solution of this embodiment obtains the wind turbine point cloud data, projects the wind turbine point cloud data onto the blade plane and performs skeleton extraction to obtain the wind turbine skeleton image, uses the preset pathfinding method to perform skeleton segmentation on the wind turbine skeleton image, combines the characteristics of the wind turbine components to obtain the component fitting functions of each wind turbine component, and divides the wind turbine point cloud data according to the component fitting functions of each component to obtain the component point cloud data of each wind turbine component, solving the problem that the current computer cannot accurately identify the wind turbine components, resulting in a large amount of manpower required for wind turbine inspection work, realizing the rapid and accurate division of the wind turbine point cloud data to obtain the component point cloud data of each wind turbine component, so as to facilitate accurate inspection of each wind turbine component, quickly complete the inspection task, save time, effort and money, and ensure the safe operation of the wind turbine unit at the same time.

[0060] Embodiment 2

[0061] Figure 2a It is a flowchart of a component recognition method based on wind turbine point cloud data provided by Embodiment 2 of the present invention. On the basis of the above embodiment, this embodiment further optimizes the above-mentioned component recognition method based on wind turbine point cloud data.

[0062] As Figure 2a shown, the method specifically includes:

[0063] Step 210: Obtain the wind turbine point cloud data, perform coordinate transformation on the wind turbine point cloud data to obtain the large-scale point cloud data in the geodetic coordinate system.

[0064] Specifically, during the inspection process, the lidar collects data, and the component recognition device can perform an initial pose transformation on the lidar-acquired fan point cloud data. By adding the rotation transformation and displacement between the lidar and the drone and the RT matrix obtained from the drone attitude transformation, the coordinate system of the fan point cloud data is converted to obtain the point cloud data in the earth coordinate system.

[0065] Step 220: Determine the blade plane based on the point cloud data in the earth coordinate system, and project the point cloud data in the earth coordinate system onto the blade plane to obtain the initial fan image.

[0066] Specifically, since the point cloud data is three-dimensional data, the blade planes where the three blades of the fan are located can be determined based on the point cloud data in the earth coordinate system, and the three-dimensional point cloud data is projected onto the blade planes to obtain a two-dimensional initial fan image. Due to the angle between the laser beams in the multi-line lidar data, the problem of uneven distribution of the point cloud data may occur. To ensure uniform data, downsampling can also be performed on the projected data, and the downsampled two-dimensional data forms the initial fan image.

[0067] Figure 2b It is the first schematic diagram of the principle of a component recognition method based on fan point cloud data provided by the second embodiment of the present invention. Figure 2b It shows the initial fan image obtained by projecting the point cloud data in the earth coordinate system of a certain power generation fan onto the blade plane.

[0068] Step 230: Perform dilation operation and erosion operation on the initial fan image to obtain the fan skeleton image.

[0069] In this embodiment, the dilation operation can be performed on the initial fan image first, and then the erosion operation can be performed to obtain the fan skeleton image.

[0070] Figure 2c It is the second schematic diagram of the principle of a component recognition method based on fan point cloud data provided by the second embodiment of the present invention. Figure 2c It is the image obtained after performing dilation operation on Figure 2b

[0071] Figure 2d It is the third schematic diagram of the principle of a component recognition method based on fan point cloud data provided by the second embodiment of the present invention. Figure 2d It is the image obtained after performing erosion operation on Figure 2c

[0072] Step 240: Use a preset path finding method to divide the initial skeleton in the fan skeleton image to obtain at least one initial path point set, and filter the burrs of the fan skeleton image according to each initial path point set to obtain the target skeleton image.

[0073] Optionally, step 240 can be specifically implemented through the following steps:

[0074] S2401. Determine all endpoints of the initial skeleton in the fan skeleton image.

[0075] Furthermore, the method for determining endpoints may include:

[0076] For each non-edge pixel point in the fan skeleton image, divide a nine-pixel point set centered on the non-edge pixel point, determine the total pixel value of the nine-pixel point set, and determine the non-edge pixel point corresponding to the nine-pixel point set with a total pixel value of 2 as an endpoint;

[0077] For each edge pixel point in the fan skeleton image, divide a six-pixel point set centered on the edge pixel point, determine the total pixel value of the six-pixel point set, and determine the edge pixel point corresponding to the six-pixel point set with a total pixel value of 2 as an endpoint;

[0078] For each corner pixel point in the fan skeleton image, divide a four-pixel point set centered on the corner pixel point, determine the total pixel value of the four-pixel point set, and determine the corner pixel point corresponding to the four-pixel point set with a total pixel value of 2 as an endpoint.

[0079] In this embodiment, the fan skeleton image is a binary image. As Figure 2d shown, the pixel value of the pixel point where the initial skeleton is located can be represented by 1, and the pixel value of the black background can be represented by 0. Pixel points can be divided into non-edge pixel points, edge pixel points, and corner pixel points according to their positions in the image. Corner pixel points can be understood as the four pixel points at the four corners of the image, edge pixel points can be understood as pixel points on the image edge and not corner pixel points, and other pixel points in the image except corner pixel points and edge pixel points can be called non-edge pixel points.

[0080] When looking for the endpoints of the initial skeleton, each pixel and its surrounding pixels can be analyzed to determine whether the pixel is an endpoint of the initial skeleton. For each non-edge pixel, a 3*3 pixel grid can be divided with this non-edge pixel as the center. The set composed of the nine pixels in this grid is called the nine-pixel set of this non-edge pixel. The values of each pixel in the nine-pixel set are added together to obtain the total pixel value. When the total pixel value is 2, it can be considered that this non-edge pixel is an endpoint. For each edge pixel, a set composed of this edge pixel and its surrounding five pixels can be formed with this edge pixel as the center, which is called the six-pixel set of this edge pixel. The values of each pixel in the six-pixel set are added together to obtain the total pixel value. When the total pixel value is 2, it can be considered that this edge pixel is an endpoint. For each corner pixel, a set composed of this corner pixel and its surrounding three pixels can be formed with this corner pixel as the center, which is called the four-pixel set of this corner pixel. The values of each pixel in the four-pixel set are added together to obtain the total pixel value. When the total pixel value is 2, it can be considered that this corner pixel is an endpoint.

[0081] S2402. For each endpoint, establish an initial path point set, add the endpoint to the initial path point set, use the endpoint as the path finding starting point, determine the number of connected points of the path point connected to the path finding starting point. If the number of connected points is equal to 1, add the path point to the initial path point set, and re-determine the path point as the new path finding starting point to find path points.

[0082] In this embodiment, path finding can be performed starting from the endpoint, and the skeleton path points connected to the endpoint are added to the initial path point set corresponding to the endpoint. Stop path finding when reaching a fork point. The basis for determining whether a fork point is reached can be to judge whether the number of connected points of the path point connected to the path finding starting point is equal to 1. If it is equal to 1, it is not a fork point. If it is greater than 1, it can be considered a fork point. Of course, the path points connected to the path finding starting point here do not include the path points that have been added to the initial path point set. In practical applications, a 3*3 pixel grid can be divided with the path finding starting point as the center, and the pixel values of all pixels in the grid are summed. If the sum of the pixel values is greater than 3, it can be considered a fork point. During the path finding process, the pixel values of the path points added to the initial path point set can also be temporarily set to 0. When determining the number of connected points of the path point connected to the path finding starting point, a 3*3 pixel grid can be divided with the path finding starting point as the center, and the pixel values of all pixels in the grid are summed. If the sum of the pixel values is greater than 2, it can be considered a fork point. After path finding for each endpoint, the initial path point sets corresponding to each endpoint can be obtained.

[0083] S2403. Determine the initial path point numbers of the initial path points included in each initial path point set.

[0084] Specifically, the number of initial path points included in each initial path point set may be different. Count the number of initial path points included in each initial path point set to obtain the number of initial path points corresponding to each initial path point set.

[0085] S2404. Filter out the initial path point sets with the number of initial path points less than the burr point threshold to obtain the target skeleton image.

[0086] In this embodiment, the burr point threshold can be preset in advance. When the number of initial path points in a certain initial path point set is less than the burr point threshold, it can be considered that the skeleton path corresponding to this initial path point set is a burr and needs to be filtered out. The remaining initial path point sets after removing the burrs can form the target skeleton image.

[0087] Figure 2e It is the fourth schematic diagram of the principle of a component recognition method based on fan point cloud data provided by the second embodiment of the present invention. Figure 2e The shown skeleton image is the Figure 2d target skeleton image formed by the remaining initial path point sets after burr filtering is performed on

[0088] Step 250. Use a preset path finding method to divide the target skeleton in the target skeleton image to obtain at least one component path point set.

[0089] Optionally, step 250 can be specifically implemented through the following steps:

[0090] S2501. Determine all the endpoints of the target skeleton in the target skeleton image.

[0091] The method for determining the endpoints of the target skeleton can be similar to the method for determining the endpoints of the initial skeleton:

[0092] For each non-edge pixel point in the target skeleton image, divide a nine-pixel point set with the non-edge pixel point as the center, determine the total pixel value of the nine-pixel point set, and determine the non-edge pixel point corresponding to the nine-pixel point set with a total pixel value of 2 as the endpoint;

[0093] For each edge pixel point in the target skeleton image, divide a six-pixel point set with the edge pixel point as the center, determine the total pixel value of the six-pixel point set, and determine the edge pixel point corresponding to the six-pixel point set with a total pixel value of 2 as the endpoint;

[0094] For each corner pixel point in the target skeleton image, divide a four-pixel point set with the corner pixel point as the center, determine the total pixel value of the four-pixel point set, and determine the corner pixel point corresponding to the four-pixel point set with a total pixel value of 2 as the endpoint.

[0095] S2502. For each endpoint, establish a set of component path points, add the endpoint to the set of component path points, use the endpoint as the pathfinding starting point, determine the number of connected points of the path point connected to the pathfinding starting point. If the number of connected points is equal to 1, add the path point to the set of component path points, and re-determine the path point as the new pathfinding starting point to search for path points.

[0096] In this embodiment, pathfinding can be performed starting from the endpoint, and the skeleton path points connected to the endpoint are added to the set of component path points corresponding to the endpoint, and pathfinding stops when reaching a fork point. The basis for determining whether a fork point is reached can be to judge whether the number of connected points of the path point connected to the pathfinding starting point is equal to 1. If it is equal to 1, it is not a fork point, and if it is greater than 1, it can be considered a fork point. Of course, the path points connected to the pathfinding starting point here do not include the path points that have been added to the set of component path points. In practical applications, a 3*3 pixel point grid can be divided with the pathfinding starting point as the center, and the pixel values of the pixel points in the grid are summed. If the sum of the pixel values is greater than 3, it can be considered a fork point. After pathfinding for each endpoint, the set of component path points corresponding to each endpoint can be obtained.

[0097] Step 260. Perform linear fitting on each set of component path points respectively to obtain the corresponding component fitting function, and label the component names for each component fitting function according to the characteristics of the fan components.

[0098] Specifically, linear fitting can be performed on each set of component path points to obtain the feature vector, that is, the position information of different fan components in the geodetic coordinate system. In addition, in practical applications, a wind turbine generally has three blades, and the angles between the blades are 120° each, and the wind turbine tower is generally perpendicular to the ground. According to the above characteristics of the fan components, the fitted component fitting function can be identified and labeled.

[0099] Step 270. Divide the wind turbine point cloud data according to each component fitting function to obtain the component point cloud data of each wind turbine component.

[0100] Specifically, since different component fitting functions can represent the position information of the corresponding fan components, the wind turbine point cloud data can be classified correspondingly based on each component fitting function, and after segmentation and classification, the component point cloud data of each wind turbine component is obtained.

[0101] Step 280. Optimize each component point cloud data to determine the component attitude information of each wind turbine component.

[0102] Specifically, optimization algorithms such as the least squares method can be used to analyze and calculate the component point cloud data to obtain relatively accurate wind turbine component attitude information.

[0103] The technical solution of this embodiment is as follows: by obtaining the wind turbine point cloud data, performing coordinate transformation on the wind turbine point cloud data to obtain the large-scale point cloud data in the geodetic coordinate system, determining the blade plane according to the large-scale point cloud data, projecting the large-scale point cloud data onto the blade plane to obtain the initial wind turbine image, performing dilation operation and erosion operation on the initial wind turbine image to obtain the wind turbine skeleton image, using a preset path finding method to perform skeleton division on the initial skeleton in the wind turbine skeleton image to obtain at least one initial path point set, filtering the burrs of the wind turbine skeleton image according to each initial path point set to obtain the target skeleton image, then using the preset path finding method to perform skeleton division on the target skeleton in the target skeleton image to obtain at least one component path point set, respectively performing linear fitting on each component path point set to obtain the corresponding component fitting function, and according to the characteristics of the wind turbine components, performing component name annotation on each component fitting function, dividing the wind turbine point cloud data according to each component fitting function to obtain the component point cloud data of each wind turbine component, and optimizing each component point cloud data to determine the component attitude information of each wind turbine component. This embodiment solves the problem that the current computer cannot accurately identify wind turbine components, resulting in a large amount of manpower required for wind turbine inspection work, realizes rapid and accurate division of wind turbine point cloud data to obtain the component point cloud data of each wind turbine component, so as to facilitate precise inspection of each wind turbine component, quickly complete the inspection task, save time, effort and money, and ensure the safe operation of the wind turbine unit at the same time.

[0104] Embodiment III

[0105] The component recognition device based on wind turbine point cloud data provided by the embodiments of the present invention can execute the component recognition method based on wind turbine point cloud data provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Figure 3 is the structural block diagram of a component recognition device based on wind turbine point cloud data provided by Embodiment III of the present invention, as Figure 3 shown. The device includes: a skeleton image determination module 310, a component skeleton segmentation module 320, and a point cloud data recognition module 330.

[0106] The skeleton image determination module 310 is used to obtain the wind turbine point cloud data, project the wind turbine point cloud data onto the blade plane and perform skeleton extraction to obtain the wind turbine skeleton image.

[0107] The component skeleton segmentation module 320 is used to perform skeleton segmentation on the wind turbine skeleton image by using a preset path finding method, and combine the characteristics of the wind turbine components to obtain the component fitting functions of each wind turbine component.

[0108] The point cloud data recognition module 330 is used to divide the wind turbine point cloud data according to each component fitting function to obtain the component point cloud data of each wind turbine component.

[0109] The technical solution of this embodiment solves the problem that a large amount of manpower is required for the fan inspection work because the computer cannot accurately identify the fan components at present. By obtaining the fan point cloud data, projecting the fan point cloud data onto the blade plane and performing skeleton extraction, a fan skeleton image is obtained. The fan skeleton image is segmented by using a preset path-finding method, and combined with the characteristics of the fan components, the component fitting functions of each fan component are obtained. According to each component fitting function, the fan point cloud data is divided to obtain the component point cloud data of each fan component, realizing the rapid and accurate division of the fan point cloud data to obtain the component point cloud data of each fan component, so as to facilitate the precise inspection of each fan component, quickly complete the inspection task, save time, labor and money, and ensure the safe operation of the wind turbine at the same time.

[0110] Optionally, the skeleton image determination module 310 includes:

[0111] The point cloud data acquisition unit is used to acquire the fan point cloud data;

[0112] The data coordinate conversion unit is used to perform coordinate transformation on the fan point cloud data to obtain the large-scale point cloud data in the geodetic coordinate system;

[0113] The initial image determination unit is used to determine the blade plane according to the large-scale point cloud data, and project the large-scale point cloud data onto the blade plane to obtain the initial fan image;

[0114] The skeleton image determination unit is used to perform dilation operation and erosion operation on the initial fan image to obtain the fan skeleton image.

[0115] Optionally, the component skeleton segmentation module 320 includes:

[0116] The skeleton burr filtering unit is used to perform skeleton segmentation on the initial skeleton in the fan skeleton image by using a preset path-finding method to obtain at least one initial path point set, and filter the burrs of the fan skeleton image according to each initial path point set to obtain the target skeleton image;

[0117] The skeleton path division unit is used to perform skeleton division on the target skeleton in the target skeleton image by using a preset path-finding method to obtain at least one component path point set;

[0118] The component characteristic fitting unit is used to perform linear fitting on each component path point set respectively to obtain the corresponding component fitting function, and label the component name according to the characteristics of the fan component for each component fitting function.

[0119] Optionally, the steps of the preset path-finding method include:

[0120] Determine all the end points of the skeleton in the image, where the image is the fan skeleton image or the target skeleton image, and the skeleton is the initial skeleton or the target skeleton;

[0121] For each end point, establish a set of path points, add the end point to the set of path points, use the end point as the starting point for path finding, determine the number of connected points of the path points connected to the starting point for path finding. If the number of connected points is equal to 1, add the path point to the set of path points, and re-determine the path point as the new starting point for path finding to search for path points. The set of path points is the initial set of path points or the component set of path points.

[0122] Optionally, the method for determining the end points includes:

[0123] For each non-edge pixel point in the image, divide a nine-pixel point set with the non-edge pixel point as the center, determine the total pixel value of the nine-pixel point set, and determine the non-edge pixel point corresponding to the nine-pixel point set with the total pixel value of 2 as the end point;

[0124] For each edge pixel point in the image, divide a six-pixel point set with the edge pixel point as the center, determine the total pixel value of the six-pixel point set, and determine the edge pixel point corresponding to the six-pixel point set with the total pixel value of 2 as the end point;

[0125] For each corner pixel point in the image, divide a four-pixel point set with the corner pixel point as the center, determine the total pixel value of the four-pixel point set, and determine the corner pixel point corresponding to the four-pixel point set with the total pixel value of 2 as the end point.

[0126] Optionally, the filtering the burrs from the fan skeleton image according to each of the initial sets of path points to obtain a target skeleton image includes:

[0127] Determine the initial number of path points of the initial path points included in each of the initial sets of path points;

[0128] Filter out the initial set of path points with the initial number of path points less than the burr point threshold to obtain a target skeleton image.

[0129] Optionally, the device further includes a component pose fitting module, and the component pose fitting module is used for:

[0130] After obtaining the component point cloud data of each of the fan components, optimize the component point cloud data of each of the fan components to determine the component pose information of each of the fan components.

[0131] Embodiment IV

[0132] Figure 4 It is a structural block diagram of a computer device provided in Embodiment IV of the present invention, asFigure 4 As shown, the computer device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the computer device can be one or more, Figure 4 and one processor 410 is taken as an example herein; the processor 410, the memory 420, the input device 430, and the output device 440 in the computer device can be connected through a bus or other means, Figure 4 and taking connection through a bus as an example herein.

[0133] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the component recognition method based on fan point cloud data in the embodiments of the present invention (for example, the skeleton image determination module 310, the component skeleton segmentation module 320, and the point cloud data recognition module 330 in the component recognition device based on fan point cloud data). The processor 410 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 420, that is, realizes the above-mentioned component recognition method based on fan point cloud data.

[0134] The memory 420 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 420 may further include a memory remotely set relative to the processor 410, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The input device 430 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the computer device. The output device 440 may include a display device such as a display screen.

[0136] Embodiment Five

[0137] Embodiment Five of the present invention further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute a component recognition method based on fan point cloud data when executed by a computer processor. The method includes:

[0138] Obtain fan point cloud data, project the fan point cloud data onto a blade plane and perform skeleton extraction to obtain a fan skeleton image, and the fan skeleton image is a binary image;

[0139] Perform skeleton segmentation on the fan skeleton image using a preset pathfinding method, and combine the characteristics of fan components to obtain the component fitting functions of each fan component;

[0140] Divide the fan point cloud data according to each of the component fitting functions to obtain the component point cloud data of each fan component.

[0141] Of course, for a storage medium containing computer-executable instructions provided in an embodiment of the present invention, the computer-executable instructions are not limited to the method operations described above, and can also execute related operations in the component recognition method based on fan point cloud data provided in any embodiment of the present invention.

[0142] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk, or optical disc of a computer, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0143] It should be noted that in the embodiments of the above-mentioned component recognition device based on fan point cloud data, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0144] Note that the above is only a preferred embodiment of the present invention and the applied technical principle. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A component recognition method based on fan point cloud data, characterized in that Including: Obtain the wind turbine point cloud data, project the wind turbine point cloud data onto the blade plane and perform skeleton extraction to obtain a wind turbine skeleton image, where the wind turbine skeleton image is a binary image; Perform skeleton segmentation on the wind turbine skeleton image by using a preset path finding method, and combine the characteristics of wind turbine components to obtain the component fitting functions of each wind turbine component; Divide the wind turbine point cloud data according to each of the component fitting functions to obtain the component point cloud data of each wind turbine component; Among them, the step of performing skeleton segmentation on the wind turbine skeleton image by using a preset path finding method and combining the characteristics of wind turbine components to obtain the component fitting functions of each wind turbine component includes: Perform skeleton division on the initial skeleton in the wind turbine skeleton image by using a preset path finding method to obtain at least one initial path point set, and filter out the burrs of the wind turbine skeleton image according to each initial path point set to obtain a target skeleton image; Perform skeleton division on the target skeleton in the target skeleton image by using a preset path finding method to obtain at least one component path point set; Perform linear fitting on each of the component path point sets respectively to obtain the corresponding component fitting functions, and perform component name annotation on each of the component fitting functions according to the characteristics of wind turbine components; The steps of the preset path finding method include: Determine all the end points of the skeleton in the image, where the image is the wind turbine skeleton image or the target skeleton image, and the skeleton is the initial skeleton or the target skeleton; For each end point, establish a path point set, add the end point to the path point set, use the end point as the path finding starting point, determine the number of connected points of the path point connected to the path finding starting point, if the number of connected points is equal to 1, then add the path point to the path point set, and re-determine the path point as the new path finding starting point to search for path points, where the path point set is the initial path point set or the component path point set.

2. The component recognition method based on fan point cloud data according to claim 1, wherein The step of projecting the wind turbine point cloud data onto the blade plane and performing skeleton extraction to obtain a wind turbine skeleton image includes: Perform coordinate transformation on the wind turbine point cloud data to obtain the large earth point cloud data in the earth coordinate system; Determine the blade plane according to the large earth point cloud data, and project the large earth point cloud data onto the blade plane to obtain an initial wind turbine image; Perform dilation operation and erosion operation on the initial wind turbine image to obtain a wind turbine skeleton image.

3. The component recognition method based on fan point cloud data according to claim 1, characterized in that, The method for determining the end points includes: For each non-edge pixel point in the image, divide a nine-pixel point set with the non-edge pixel point as the center, determine the total pixel value of the nine-pixel point set, and determine the non-edge pixel point corresponding to the nine-pixel point set with the total pixel value of 2 as the end point; For each edge pixel point in the image, divide a six-pixel point set with the edge pixel point as the center, determine the total pixel value of the six-pixel point set, and determine the edge pixel point corresponding to the six-pixel point set with the total pixel value of 2 as the end point; For each corner pixel point in the image, divide a four-pixel point set with the corner pixel point as the center, determine the total pixel value of the four-pixel point set, and determine the corner pixel point corresponding to the four-pixel point set with the total pixel value of 2 as the end point.

4. The component recognition method based on the fan point cloud data according to claim 1, wherein Performing burr filtering on the fan skeleton image according to each of the initial path point sets to obtain a target skeleton image, including: Determining the initial number of path points of the initial path points included in each of the initial path point sets; Filtering out the initial path point sets with the initial number of path points less than the burr point number threshold to obtain a target skeleton image.

5. The component recognition method based on fan point cloud data according to claim 1, characterized in that After obtaining the component point cloud data of each of the fan components, it further includes: Optimizing the component point cloud data of each of the components to determine the component attitude information of each of the fan components.

6. A component recognition device based on fan point cloud data, characterized in that Including: A skeleton image determination module, configured to obtain fan point cloud data, project the fan point cloud data onto a blade plane and perform skeleton extraction to obtain a fan skeleton image; A component skeleton segmentation module, configured to perform skeleton segmentation on the fan skeleton image by using a preset path finding method, and combine the characteristics of the fan components to obtain the component fitting functions of each fan component; A point cloud data recognition module, configured to divide the fan point cloud data according to each of the component fitting functions to obtain the component point cloud data of each of the fan components; Wherein, the component skeleton segmentation module includes: A skeleton burr filtering unit, configured to perform skeleton division on the initial skeleton in the fan skeleton image by using a preset path finding method to obtain at least one initial path point set, and perform burr filtering on the fan skeleton image according to each of the initial path point sets to obtain a target skeleton image; A skeleton path division unit, configured to perform skeleton division on the target skeleton in the target skeleton image by using a preset path finding method to obtain at least one component path point set; A component characteristic fitting unit, configured to perform linear fitting on each of the component path point sets respectively to obtain corresponding component fitting functions, and perform component name labeling on each of the component fitting functions according to the characteristics of the fan components; The steps of the preset path finding method include: Determining all endpoints of the skeleton in the image, where the image is the fan skeleton image or the target skeleton image, and the skeleton is the initial skeleton or the target skeleton; For each endpoint, establishing a path point set, adding the endpoint to the path point set, using the endpoint as the path finding starting point, determining the number of connected points of the path point connected to the path finding starting point, if the number of connected points is equal to 1, adding the path point to the path point set, and re-determining the path point as a new path finding starting point to search for path points, where the path point set is the initial path point set or the component path point set.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the component recognition method based on fan point cloud data as described in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the component recognition method based on fan point cloud data as described in any one of claims 1-5 when executed by a computer processor.

Images