A method, system and device for detecting the wear of a runner blade of a hydraulic turbine

Abstract

The application discloses a kind of wear amount detection method, system and equipment of water turbine runner blade, belong to wear amount detection technical field, by obtaining the real-time multi-modal data of runner blade, and using the wear area identification model after training to the real-time multi-modal data identification, to comprehensively identify the wear area of runner blade, and by generating wear depth map, wear depth map and the wear area identified are masked, to obtain the depth data of wear area, to obtain the depth and volume of wear, to quantify wear degree, and can realize automatic acquisition, identification and processing, compared with artificial processing, significantly improve detection efficiency and detection accuracy, facilitate application and promotion.

Description

Technical Field

[0001] This invention belongs to the field of wear detection technology, specifically relating to a method, system and equipment for detecting wear on turbine runner blades. Background Technology

[0002] A water turbine is a power machine that converts the energy of flowing water into rotational mechanical energy. It plays a crucial role in hydropower generation and is the core energy conversion equipment of a hydroelectric power station. It primarily converts the kinetic energy of flowing water into mechanical energy to drive generator sets and produce electricity. The performance and efficiency of the water turbine directly affect the economic benefits of the hydroelectric power plant and the stability of the power supply. The turbine runner blades, as one of the core components of the water turbine, bear enormous water flow impacts and mechanical loads. Their condition has a vital impact on the operating efficiency and safety of the entire system. During long-term operation, especially under conditions of sediment-laden water flow, corrosion and cavitation damage inevitably occur, forming wear zones. Wear of the runner blades not only significantly reduces the hydraulic efficiency of the water turbine and increases operating energy consumption, but in severe cases, it can lead to catastrophic accidents such as blade breakage, threatening the safety of the power grid.

[0003] However, traditional turbine blade inspection methods typically rely on manual visual inspection and contact inspection. These methods usually require professional personnel on-site, are cumbersome and time-consuming, and are easily affected by the inspector's experience level and varying inspection conditions. They are highly subjective, struggle to provide comprehensive blade condition data, and are prone to missed or false detections, resulting in an inability to fully assess turbine blade wear and low inspection accuracy and efficiency. With technological advancements, inspection methods based on two-dimensional image analysis have been applied to blade damage identification. These methods use image processing algorithms to identify color, texture, or grayscale changes caused by wear. However, they lack three-dimensional depth information and cannot obtain key quantitative indicators such as wear depth and volume.

[0004] Therefore, how to provide an effective technical solution to address the problems of low detection accuracy and efficiency, and the inability to obtain key quantitative indicators such as wear depth and volume in existing technologies, has become an urgent problem to be solved in existing technologies. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and equipment for detecting the wear of turbine runner blades, in order to solve the above-mentioned problems existing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for detecting the wear of turbine runner blades, comprising:

[0008] The system acquires real-time multimodal data of turbine runner blades collected by a multimodal sensing unit. The real-time multimodal data includes real-time raw point cloud data and real-time image texture data. The system performs feature extraction on the real-time raw point cloud data to obtain high-level semantic features, and performs feature extraction on the real-time image texture data to obtain local binary pattern features and directional gradient histogram features.

[0009] The extracted high-level semantic features, local binary pattern features, and directional gradient histogram features are concatenated and fused to obtain fused features. The fused features are then subjected to dimensionality reduction to obtain the main features. The main features are then input into a pre-trained wear area recognition model to output a binary mask image, which is used to identify wear areas.

[0010] The reference point cloud data of the turbine runner blades in the pre-built database is obtained, and the real-time raw point cloud data is three-dimensionally registered with the reference point cloud data so that the real-time raw point cloud data and the reference point cloud data are aligned in the spatial coordinate system to obtain the registered point cloud data.

[0011] Based on the registered point cloud data, the distance difference between points in the real-time raw point cloud data and points in the reference point cloud data is calculated to generate a wear depth map based on the distance difference. The wear depth map is then masked based on a binarized mask image to extract the depth data of the wear area. Based on the depth data of the wear area, the wear volume and / or average wear depth are calculated, and the wear volume and / or average wear depth are used as the final wear amount.

[0012] In one possible design, feature extraction is performed on real-time raw point cloud data to obtain high-level semantic features, and feature extraction is performed on real-time image texture data to obtain local binary pattern features and histogram of oriented gradients features, including:

[0013] A filtering algorithm is used to identify noise points and outliers in the real-time raw point cloud data, and the identified noise points and outliers are removed to obtain denoised real-time point cloud data.

[0014] The moving least squares method is used to smooth the denoised real-time point cloud data to obtain real-time point cloud data.

[0015] Contrast enhancement is performed on real-time image texture data to obtain enhanced real-time image texture data. The enhanced real-time image texture data is then converted from a color image to a grayscale image to obtain grayscale image texture data. Denoising is performed on the grayscale image texture data based on median filtering to obtain smooth image texture data.

[0016] Capsule networks are used to extract features from real-time point cloud data to obtain advanced semantic features;

[0017] The LBP feature extraction algorithm is used to extract features from the smooth image texture data to obtain local binary pattern features, and the HOG feature extraction algorithm is used to extract features from the smooth image texture data to obtain directional gradient histogram features.

[0018] In one possible design, capsule networks are used to extract features from real-time point cloud data to obtain high-level semantic features, including:

[0019] The real-time point cloud data is projected based on a pre-constructed parameterized weight matrix to obtain multiple prediction vectors;

[0020] The attention coefficients among multiple prediction vectors are calculated using a self-attention mechanism, and the multiple prediction vectors are weighted and summed based on the attention coefficients to obtain the output vector.

[0021] High-level semantic features are obtained by performing a nonlinear transformation on the output vector based on a compression function.

[0022] In one possible design, the LBP feature extraction algorithm is used to extract features from the smooth image texture data to obtain local binary pattern features, and the HOG feature extraction algorithm is used to extract features from the smooth image texture data to obtain histogram of oriented gradients features, including:

[0023] Obtain a preset radius and a preset number of sampling points, extract each pixel from the smooth image texture data, use each pixel as a center point, randomly select a center point based on the preset number of sampling points to obtain multiple sampling center points, and calculate the LBP value of the pixels in the neighborhood of each sampling center point based on the preset radius and the sampling center point.

[0024] Calculate the histogram of LBP values ​​of pixels in the neighborhood of each sampling center point, and normalize the histogram to obtain the normalized histogram. Use the normalized histogram as the local binary pattern feature.

[0025] Calculate the gradient strength and direction of smooth image texture data;

[0026] The smooth image texture data is divided into multiple cells, and the gradient direction histogram is calculated in each cell to obtain the unidirectional gradient histogram feature of each cell. Multiple cells are combined to obtain a block, and the unidirectional gradient histogram features of each cell in the block are spliced ​​together to obtain the oriented gradient histogram feature.

[0027] In one possible design, the fused features are dimensionality-reduced to obtain the main features, which are then input into a pre-trained wear region recognition model to output a binarized mask image, including:

[0028] Principal component analysis was used to reduce the dimensionality of the fusion features to obtain the main features;

[0029] The synthetic minority oversampling algorithm is used to oversample the main features to balance the categories of the main features, resulting in the oversampled main features.

[0030] The oversampled main features are input into a pre-trained wear area recognition model, which outputs a binarized mask image.

[0031] In one possible design, the pre-trained wear region recognition model includes an encoder and a decoder, the encoder comprising an input layer, multiple convolutional layers, and multiple pooling layers; the step of inputting the oversampled main features into the pre-trained wear region recognition model and outputting a binarized mask image includes:

[0032] The oversampled main features are input into the input layer, and a convolutional layer is used to perform a first convolution on the oversampled main features to obtain a deep feature map. A pooling layer is used to perform a downsampling operation on the deep feature map to obtain a dimensionality-reduced feature map. A convolutional layer is used again to perform a second convolution on the dimensionality-reduced feature map to obtain a second deep feature map.

[0033] The above-described first convolution process, downsampling operation, and second convolution process are repeated in the encoder until each convolutional layer and pooling layer are used to obtain multiple deep feature maps, which include the final encoded feature map.

[0034] The final encoded feature map is upsampled twice to obtain the processed final encoded feature map. The processed final encoded feature map is then fused with the corresponding deep feature map to obtain the feature fusion map. A convolutional layer is used to convolve the feature fusion map to obtain the deep feature fusion map. The deep feature map is then upsampled twice again to obtain the processed deep feature fusion map.

[0035] The above-mentioned first doubling upsampling process, feature fusion operation, convolution process, and second doubling upsampling process are repeated in the decoder until the output image has the same size as the main features after oversampling, thus obtaining the binarized mask image.

[0036] In one possible design, the real-time raw point cloud data is 3D registered with the reference point cloud data so that the real-time raw point cloud data and the reference point cloud data are aligned in a spatial coordinate system, resulting in registered point cloud data, including:

[0037] The real-time raw point cloud data and the reference point cloud data are filtered, and the filtered real-time raw point cloud data and the filtered reference point cloud data are downsampled to obtain the preprocessed real-time raw point cloud data and the preprocessed reference point cloud data.

[0038] Find the nearest point in the preprocessed reference point cloud data for each point in the preprocessed real-time raw point cloud data, and calculate the rigid transformation parameters that minimize the distance error between the corresponding nearest point and the point. The rigid transformation parameters include the rotation matrix and the translation vector.

[0039] The preprocessed real-time raw point cloud data is translated and rotated based on the rotation matrix and translation vector to obtain the transformed real-time raw point cloud data.

[0040] The average distance between the transformed real-time raw point cloud data and the preprocessed reference point cloud data is calculated to obtain the average distance of the point set. The average distance of the point set is compared with the preset distance threshold. If the average distance of the point set is less than or equal to the preset distance threshold, the transformed real-time raw point cloud data is used as the registered point cloud data.

[0041] In one possible design, a wear depth map is generated based on distance differences. The wear depth map is then masked using a binary mask image to extract depth data of the wear region. Based on the depth data of the wear region, the wear volume and / or average wear depth are calculated. The wear volume and / or average wear depth are then used as the final wear amount, including:

[0042] Define a projection plane and divide the projection plane into multiple pixel grids. Project the distance difference onto the multiple pixel grids. For each pixel grid, find all three-dimensional points projected into the pixel grid and set the value of the pixel grid to the maximum or average value of the distance difference corresponding to the three-dimensional point. The maximum value is used to characterize the wear point with the deepest wear at the blade position corresponding to the pixel grid, and the average value is used to characterize the average wear at the blade position corresponding to the pixel grid, so as to obtain a wear depth map.

[0043] Spatially align the wear depth map and the binarized mask image to extract the depth data of the wear region in the binarized mask image based on the wear depth map;

[0044] The average wear depth is calculated by averaging the depth data corresponding to all pixels within the wear area. The wear volume is then calculated by performing volume element analysis on all pixels within the wear area. The wear volume and / or average wear depth are used as the final wear amount.

[0045] Secondly, the present invention provides a wear detection system for turbine runner blades, comprising:

[0046] The feature extraction module is used to acquire real-time multimodal data of turbine runner blades collected by the multimodal sensing unit. The real-time multimodal data includes real-time raw point cloud data and real-time image texture data. Feature extraction is performed on the real-time raw point cloud data to obtain high-level semantic features, and feature extraction is performed on the real-time image texture data to obtain local binary pattern features and directional gradient histogram features.

[0047] The fusion recognition module is used to concatenate and fuse the extracted high-level semantic features, local binary pattern features, and directional gradient histogram features to obtain fused features. The fusion features are then subjected to dimensionality reduction processing to obtain the main features. The main features are then input into a pre-trained wear area recognition model to output a binary mask image, which is used to identify wear areas.

[0048] The registration and alignment module is used to acquire the reference point cloud data of the turbine runner blades in the pre-built database, and to perform three-dimensional registration between the real-time raw point cloud data and the reference point cloud data so that the real-time raw point cloud data and the reference point cloud data are aligned in the spatial coordinate system to obtain the registered point cloud data.

[0049] The wear calculation module is used to calculate the distance difference between points in the real-time raw point cloud data and points in the reference point cloud data based on the registered point cloud data, so as to generate a wear depth map according to the distance difference, perform masking processing on the wear depth map based on the binarized mask image to extract the depth data of the wear area, calculate the wear volume and / or average wear depth according to the depth data of the wear area, and use the wear volume and / or average wear depth as the final wear amount.

[0050] Thirdly, the present invention provides a computer device comprising a memory, a processor, and a transceiver connected in sequence and communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the wear detection method for turbine runner blades as described in the first aspect above.

[0051] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, perform the method for detecting the wear of turbine runner blades as described in the first aspect above.

[0052] Fifthly, the present invention provides a computer program product containing instructions that, when the instructions are executed on a computer, cause the computer to perform the method for detecting the wear of turbine runner blades as described in the first aspect above.

[0053] The beneficial effects of this invention are as follows: This invention discloses a method, system, and device for detecting the wear of turbine runner blades. First, real-time point cloud data and real-time image texture data of the turbine runner blades are acquired. Features are extracted from the real-time point cloud data and real-time image texture data to obtain high-level semantic features, local binary pattern features, and histogram of directional gradients features. These features are then spliced ​​and fused, and the fused features are subjected to dimensionality reduction. A pre-trained wear area recognition model is used to process the main features to obtain a binary mask image, which is used to identify the wear area. Subsequently, the real-time point cloud data and the reference point cloud data are aligned to calculate the distance difference between the real-time point cloud data and the reference point cloud data based on the registered point cloud data. A wear depth map is generated based on the distance difference. The wear depth map is masked based on the binary mask image to extract the depth data of the wear area. The wear volume and / or average wear depth are calculated based on the depth data of the wear area, and the wear volume and / or average wear depth are used as the final wear amount. This invention acquires real-time multimodal data of rotor blades and identifies the wear areas of the rotor blades by using a wear area identification model. By generating a wear depth map and masking the identified wear areas, the depth data of the wear areas is obtained, thus determining the wear depth and volume, and quantifying the degree of wear. This invention achieves automated acquisition, identification, and processing, significantly improving detection efficiency and accuracy compared to manual processing, and facilitating application and promotion. Attached Figure Description

[0054] Figure 1 A flowchart illustrating the method for detecting the wear of turbine runner blades provided in an embodiment of the present invention;

[0055] Figure 2 A block diagram of a wear detection system for turbine runner blades provided in an embodiment of the present invention;

[0056] Figure 3 A structural diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0058] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0059] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0060] Example:

[0061] like Figure 1 As shown, the first aspect of this embodiment provides a method for detecting the wear of turbine runner blades, which can be executed, but is not limited to, by a computer device or virtual machine with certain computing resources, such as a personal computer or smartphone, or by a virtual machine; the wear detection method includes, but is not limited to, the following steps:

[0062] S1. Acquire real-time multimodal data of turbine runner blades collected by the multimodal sensing unit. The real-time multimodal data includes real-time raw point cloud data and real-time image texture data. Perform feature extraction on the real-time raw point cloud data to obtain high-level semantic features. Perform feature extraction on the real-time image texture data to obtain local binary pattern features and orientation gradient histogram features.

[0063] It should be noted that the multimodal sensing unit mentioned in this embodiment includes, but is not limited to, a pulsed laser scanner, a structured light projector, and a high-speed camera. The acquisition process using the multimodal sensing unit is as follows: the pulsed laser scanner, the structured light projector, and the high-speed camera are installed on the inner wall of the turbine flow channel through a pressure-resistant light-transmitting window. The inner wall of the flow channel includes the inner wall of the volute, the inner wall of the seat ring, or the inner wall of the tailrace pipe. An angle encoder connected to the turbine runner shaft is used as a trigger source. When the turbine runner is working, the angle encoder sends a signal to synchronously trigger the multimodal sensing unit to scan and image the surface of the runner blades passing through its field of view, and acquire real-time multimodal data, namely real-time raw point cloud data and real-time image texture data. The real-time raw point cloud data includes three-dimensional coordinate information.

[0064] Specifically, in step S1, feature extraction is performed on the real-time raw point cloud data to obtain high-level semantic features, and feature extraction is performed on the real-time image texture data to obtain local binary pattern features and histogram of oriented gradients features, including:

[0065] S11. Use a filtering algorithm to identify noise points and outliers in the real-time raw point cloud data, and remove the identified noise points and outliers to obtain denoised real-time point cloud data.

[0066] S12. Use the moving least squares method to smooth the denoised real-time point cloud data to obtain real-time point cloud data;

[0067] S13. Perform contrast enhancement on the real-time image texture data to obtain enhanced real-time image texture data, and convert the enhanced real-time image texture data from a color image to a grayscale image to obtain grayscale image texture data. Perform denoising processing on the grayscale image texture data based on median filtering to obtain smooth image texture data.

[0068] S14. Use capsule networks to extract features from real-time point cloud data to obtain advanced semantic features;

[0069] S15. Use the LBP feature extraction algorithm to extract features from the smooth image texture data to obtain local binary pattern features, and use the HOG feature extraction algorithm to extract features from the smooth image texture data to obtain directional gradient histogram features.

[0070] It should be noted that this embodiment uses a filtering algorithm to identify and remove noise points and outliers, and smooths the point cloud data to reduce the impact of noise points and outliers on subsequent processing and improve the quality of the point cloud data. The filtering algorithm used in this embodiment is a statistical outlier filtering algorithm, which calculates the average distance from each point to its neighboring points and removes points whose mean distance exceeds a specified standard deviation threshold. Moving least squares is a point cloud smoothing technique based on the least squares framework, which reconstructs the point cloud surface by fitting a smooth surface in the local neighborhood of each point. Real-time image texture data is enhanced to highlight the texture features in the data, and the enhanced real-time image texture data is converted from a color image to a grayscale image to reduce the impact of subsequent noise on gradient calculation.

[0071] Furthermore, in step S14, a capsule network is used to extract features from the real-time point cloud data to obtain high-level semantic features, including:

[0072] S14.1. Project the real-time point cloud data according to the pre-constructed parameterized weight matrix to obtain multiple prediction vectors;

[0073] S14.2. Use a self-attention mechanism to calculate the attention coefficients among multiple prediction vectors, and then perform a weighted summation of the multiple prediction vectors based on the attention coefficients to obtain the output vector;

[0074] S14.3. Perform a nonlinear transformation on the output vector based on the compression function to obtain advanced semantic features.

[0075] It should be noted that the capsule network consists of an input layer, a primary capsule layer, a higher-level capsule layer, and an output layer. Each capsule is a group of neurons that outputs a vector representing the existence and attributes of a specific feature. Through a dynamic routing mechanism, the output vectors of lower-level capsules are passed to higher-level capsules to effectively capture the hierarchical structure and spatial relationships of features. In this embodiment, attention mechanisms and parameterized weight matrices are used in the higher-level capsule layer for dynamic routing between capsules, which more effectively selects and weights important features and suppresses unimportant features, thereby extracting high-level semantic features and providing accurate feature representations for subsequent identification of wear areas. Furthermore, the combination of dynamic routing mechanisms and self-attention mechanisms makes this model more robust and stable.

[0076] Further, in step S15, the LBP (Local Binary Patterns) feature extraction algorithm is used to extract features from the smooth image texture data to obtain local binary pattern features, and the HOG (Histogram of Oriented Gradients) feature extraction algorithm is used to extract features from the smooth image texture data to obtain histogram of oriented gradient features, including:

[0077] S15.1. Obtain the preset radius and the preset number of sampling points, extract each pixel in the smooth image texture data, take each pixel as the center point, randomly select the center point based on the preset number of sampling points to obtain multiple sampling center points, and calculate the LBP value of the pixels in the neighborhood of each sampling center point based on the preset radius and the sampling center point.

[0078] S15.2. Calculate the histogram of LBP values ​​of pixels in the neighborhood of each sampling center point, and normalize the histogram to obtain the normalized histogram. Use the normalized histogram as the local binary pattern feature.

[0079] S15.3. Calculate the gradient intensity and direction of the smoothed image texture data;

[0080] S15.4. Divide the smooth image texture data into multiple cells, and calculate the histogram of gradient direction in each cell to obtain the unidirectional gradient histogram feature of each cell. Combine multiple cells to obtain a block, and stitch together the unidirectional gradient histogram features of each cell in the block to obtain the oriented gradient histogram feature.

[0081] It should be noted that in step S15.1, for each pixel, each pixel is taken as the center point, and the binary mode value of the pixels in the neighborhood of each sampling center point within a preset radius is calculated. The specific calculation expression is as follows: In the formula, ( , () represents the coordinates of the sampling center point. , () represents the coordinates of the sampling center point within the circular neighborhood. For the preset radius, The preset number of sampling points, Sampling center point ( , The binary pattern value of the pixels in the neighborhood. The value range of is [0, P-1], which represents the currently calculated sampling center point. It is a step function. The center pixel of the sampling center point in the circular neighborhood. The center pixel of the sampling center point is used to obtain the binary mode value of each center pixel through the above calculation expression. After obtaining a block, the histogram is normalized within the block, and the unidirectional gradient histogram features of all cells are concatenated together to form the final unidirectional gradient histogram feature description of the image, thereby improving the robustness of the features.

[0082] S2. The extracted high-level semantic features, local binary pattern features, and directional gradient histogram features are concatenated and fused to obtain fused features. The fused features are then subjected to dimensionality reduction to obtain the main features. The main features are then input into a pre-trained wear area recognition model to output a binary mask image, which is used to identify wear areas.

[0083] Specifically, in step S2, the fused features are dimensionality reduced to obtain the main features, and the main features are input into a pre-trained wear area recognition model to output a binarized mask image, including:

[0084] S21. Principal component analysis is used to reduce the dimensionality of the fusion features to obtain the main features;

[0085] S22. Use the synthetic minority oversampling algorithm to oversample the main features to balance the categories of the main features and obtain the oversampled main features;

[0086] S23. Input the oversampled main features into the pre-trained wear area recognition model and output a binarized mask image.

[0087] It should be noted that the synthetic minority oversampling algorithm balances the dataset by generating synthetic minority class samples. It interpolates the feature space based on the feature vectors of the minority class samples to create new synthetic samples, thereby reducing model bias caused by data imbalance. The specific process includes calculating the Euclidean distance of each sample in the minority class to other minority class samples, determining the K nearest neighbors, setting the sampling ratio according to the sample imbalance ratio, randomly selecting several samples from the K nearest neighbors, and generating new samples through a linear interpolation formula to balance the feature categories.

[0088] In a preferred embodiment, a wear region recognition model is constructed based on a semantic segmentation model. A labeled training dataset is obtained, and the labeled training dataset is input into the wear region recognition model to obtain a predicted recognition result. Based on the predicted recognition result, the label, the cross-entropy loss function, and the Dice Loss, a loss value is calculated. The wear region recognition model is updated according to the loss value to obtain an updated wear region recognition model. The labeled training dataset is input into the updated wear region recognition model again, and the above iterative operation is repeated until a preset iteration termination condition is reached. The preset iteration termination condition includes reaching a preset number of iterations and the loss value being less than a preset loss threshold, to obtain a trained wear region recognition model.

[0089] The pre-trained wear region recognition model includes an encoder and a decoder. The encoder includes an input layer, multiple convolutional layers, and multiple pooling layers.

[0090] In a preferred embodiment, step S23 involves inputting the oversampled main features into a pre-trained wear area recognition model and outputting a binarized mask image, including:

[0091] S23.1. Input the oversampled main features into the input layer, and use a convolutional layer to perform a first convolution on the oversampled main features to obtain a deep feature map. Use a pooling layer to perform a downsampling operation on the deep feature map to obtain a dimensionality-reduced feature map. Use a convolutional layer again to perform a second convolution on the dimensionality-reduced feature map to obtain a second deep feature map.

[0092] S23.2. Repeat the above-described first convolution process, downsampling operation and second convolution process in the encoder until each convolutional layer and pooling layer is used to obtain multiple deep feature maps, the multiple deep feature maps including the final encoded feature map;

[0093] S23.3. Perform a double upsampling process on the final encoded feature map to obtain the processed final encoded feature map. Perform feature fusion with the corresponding deep feature map to obtain the feature fusion map. Perform convolution processing on the feature fusion map using a convolutional layer to obtain the deep feature fusion map. Perform a double upsampling process on the deep feature map again to obtain the processed deep feature fusion map.

[0094] S23.4. Repeat the above-mentioned first doubling upsampling process, feature fusion operation, convolution process, and second doubling upsampling process in the decoder until the output image has the same size as the main features after oversampling, and obtain the binarized mask image.

[0095] It should be noted that the convolutional layer in this embodiment uses a size of 3. The 3-convolution kernel extracts deep features through the convolutional layers in this embodiment, covering various information in the image. Then, through layer-by-layer upsampling and convolution operations of the decoder, the spatial resolution of the feature map is gradually restored until it is the same size as the input image, resulting in the final binarized mask image, which is used to identify the wear area. The encoder in this embodiment includes 13 convolutional layers and 4 pooling layers. In specific applications, only 2 convolutional layers are used for each of the first and second convolutional dimensionality reduction processes. In subsequent convolutional dimensionality reduction processes, 3 convolutional layers are used for each convolutional process. The decoder includes 8 convolutional layers, and 2 convolutional layers are used for each convolutional process.

[0096] S3. Obtain the reference point cloud data of the turbine runner blades in the pre-built database, and perform three-dimensional registration between the real-time raw point cloud data and the reference point cloud data so that the real-time raw point cloud data and the reference point cloud data are aligned in the spatial coordinate system to obtain the registered point cloud data.

[0097] Specifically, in step S3, the real-time raw point cloud data and the reference point cloud data are three-dimensionally registered so that the real-time raw point cloud data and the reference point cloud data are aligned in the spatial coordinate system to obtain the registered point cloud data, including:

[0098] S31. Filter the real-time raw point cloud data and the reference point cloud data, and downsample the filtered real-time raw point cloud data and the filtered reference point cloud data to obtain the preprocessed real-time raw point cloud data and the preprocessed reference point cloud data.

[0099] S32. Find the corresponding nearest point in the preprocessed reference point cloud data for each point in the preprocessed real-time raw point cloud data, and calculate the rigid transformation parameters that minimize the distance error between the corresponding nearest point and the point. The rigid transformation parameters include the rotation matrix and the translation vector.

[0100] S33. Based on the rotation matrix and translation vector, the preprocessed real-time raw point cloud data is translated and rotated to obtain the transformed real-time raw point cloud data;

[0101] S34. Calculate the average distance between the transformed real-time raw point cloud data and the preprocessed reference point cloud data to obtain the average distance of the point set. Compare the average distance of the point set with the preset distance threshold. If the average distance of the point set is less than or equal to the preset distance threshold, then use the transformed real-time raw point cloud data as the registered point cloud data.

[0102] In specific implementation, iterative calculations are performed on the preprocessed real-time raw point cloud data and the reference point cloud data until a preset convergence condition is reached. Each iteration of the iterative calculation includes: selecting a point in the preprocessed real-time raw point cloud data; searching for the nearest neighbor of this point in the preprocessed reference point cloud data; constructing an initial set of corresponding point pairs based on this point and its nearest neighbors; filtering the initial set of corresponding point pairs based on a preset first distance threshold to obtain a set of valid corresponding point pairs; and calculating the rigid transformation parameter with the minimum distance error between corresponding points using the least squares method based on the valid set of corresponding point pairs. The rigid transformation parameters include a rotation matrix and a translation vector. The preprocessed real-time raw point cloud data is updated based on the rigid transformation parameters to obtain the transformed real-time raw point cloud data. The average distance between the transformed real-time raw point cloud data and the preprocessed reference point cloud data is calculated to obtain the average distance of the point set. The average distance of the point set is compared with a preset second distance threshold to determine whether the transformed real-time raw point cloud data is aligned. If it is not aligned, that is, the average distance of the point set is greater than the preset distance threshold, the rigid transformation parameters are recalculated to translate and rotate the point cloud data according to the rigid transformation parameters to obtain aligned point cloud data.

[0103] S4. Calculate the distance difference between points in the real-time raw point cloud data and points in the reference point cloud data based on the registered point cloud data, so as to generate a wear depth map based on the distance difference. Perform masking processing on the wear depth map based on the binarized mask image to extract the depth data of the wear area. Calculate the wear volume and / or average wear depth based on the depth data of the wear area, and use the wear volume and / or average wear depth as the final wear amount.

[0104] Specifically, in step S4, a wear depth map is generated based on the distance difference. The wear depth map is then masked using a binary mask image to extract the depth data of the wear area. Based on the depth data of the wear area, the wear volume and / or average wear depth are calculated. The wear volume and / or average wear depth are used as the final wear amount, including:

[0105] S41. Define a projection plane and divide the projection plane into multiple pixel grids. Project the distance difference onto the multiple pixel grids. For each pixel grid, find all three-dimensional points projected into the pixel grid and set the value of the pixel grid to the maximum or average value of the distance difference corresponding to the three-dimensional point. The maximum value is used to characterize the wear point with the deepest wear at the blade position corresponding to the pixel grid, and the average value is used to characterize the average wear at the blade position corresponding to the pixel grid, so as to obtain a wear depth map.

[0106] S42. Spatially align the wear depth map and the binarized mask image to extract the depth data of the wear region in the binarized mask image based on the wear depth map;

[0107] S43. Calculate the average depth of the depth data corresponding to all pixels in the wear area to obtain the average wear depth. Calculate the volume element of all pixels in the wear area to obtain the wear volume. Use the wear volume and / or the average wear depth as the final wear amount.

[0108] The projection plane is usually selected as the main observation plane for the wear of turbine blades. When the distance difference is projected onto multiple pixel grids, if no three-dimensional point is projected onto the pixel grid, the pixel is filled with a preset empty value.

[0109] It should be noted that the masking process in this embodiment uses pixel-wise multiplication. Pixel-wise multiplication involves multiplying the corresponding pixel values ​​of two images of the same size, so that the pixel value of the resulting image is equal to the product of the pixel values ​​at the same position in the input image. This completes the masking process for the binarized mask image and the wear depth map. Subsequently, all pixel values ​​and depth values ​​greater than 0 are extracted and used as the depth data of the wear region. The arithmetic mean of the depth data corresponding to all pixels is calculated to obtain the average wear depth. The projected area in the physical world corresponding to each pixel in the wear depth map is obtained, and the depth data of the wear region is multiplied by its corresponding projected area to obtain the volume element contributed by that point. The volume elements corresponding to all points are added together to obtain the wear volume, thus obtaining the final wear amount.

[0110] like Figure 2 As shown, the second aspect of this embodiment provides a wear detection system for turbine runner blades, comprising:

[0111] The feature extraction module is used to acquire real-time multimodal data of turbine runner blades collected by the multimodal sensing unit. The real-time multimodal data includes real-time raw point cloud data and real-time image texture data. Feature extraction is performed on the real-time raw point cloud data to obtain high-level semantic features, and feature extraction is performed on the real-time image texture data to obtain local binary pattern features and directional gradient histogram features.

[0112] The fusion recognition module is used to concatenate and fuse the extracted high-level semantic features, local binary pattern features, and directional gradient histogram features to obtain fused features. The fusion features are then subjected to dimensionality reduction processing to obtain the main features. The main features are then input into a pre-trained wear area recognition model to output a binary mask image, which is used to identify wear areas.

[0113] The registration and alignment module is used to acquire the reference point cloud data of the turbine runner blades in the pre-built database, and to perform three-dimensional registration between the real-time raw point cloud data and the reference point cloud data so that the real-time raw point cloud data and the reference point cloud data are aligned in the spatial coordinate system to obtain the registered point cloud data.

[0114] The wear calculation module is used to calculate the distance difference between points in the real-time raw point cloud data and points in the reference point cloud data based on the registered point cloud data, so as to generate a wear depth map according to the distance difference, perform masking processing on the wear depth map based on the binarized mask image to extract the depth data of the wear area, calculate the wear volume and / or average wear depth according to the depth data of the wear area, and use the wear volume and / or average wear depth as the final wear amount.

[0115] The working process, working details and technical effects of the turbine runner blade wear detection system provided in the second aspect of this embodiment can be found in the turbine runner blade wear detection method described in the first aspect, and will not be repeated here.

[0116] like Figure 3As shown, the third aspect of this embodiment provides a computer device, including a memory, a processor, and a transceiver connected in sequence. The memory stores a computer program, the transceiver sends and receives messages, and the processor reads the computer program to execute the wear detection method for turbine runner blades as described in the first aspect. Specifically, the memory may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; the processor may include, but is not limited to, an STM32F105 series microprocessor. Furthermore, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0117] The working process, working details and technical effects of the aforementioned computer equipment provided in the third aspect of this embodiment can be found in the method for detecting the wear of turbine runner blades described in the first aspect, and will not be repeated here.

[0118] This fourth aspect of the embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the wear detection method for turbine runner blades as described in the first aspect is performed. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0119] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be found in the method for detecting the wear of turbine runner blades as described in the first aspect, and will not be repeated here.

[0120] The fifth aspect of this embodiment provides a computer program product, including a computer program or instructions, which, when executed by a computer, are used to implement the method for detecting the wear of turbine runner blades as described in the first aspect.

[0121] The working process, working details and technical effects of the aforementioned computer program product provided in this embodiment can be found in the method for detecting the wear of turbine runner blades as described in the first aspect, and will not be repeated here.

[0122] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of detecting an abrasion amount of a runner blade of a hydraulic turbine, characterized by, The method comprises the following steps: acquiring real-time multi-modal data of a runner blade of a hydraulic turbine collected by a multi-modal sensing unit, wherein the real-time multi-modal data comprises real-time original point cloud data and real-time image texture data, performing feature extraction on the real-time original point cloud data to obtain high-level semantic features, and performing feature extraction on the real-time image texture data to obtain local binary pattern features and histogram of oriented gradient features; splicing and fusing the extracted high-level semantic features, local binary pattern features and histogram of oriented gradient features to obtain fused features, performing dimension reduction processing on the fused features to obtain main features, and inputting the main features into a pre-trained wear area recognition model to output a binary mask image, wherein the binary mask image is used for identifying a wear area; acquiring reference point cloud data of the runner blade of the hydraulic turbine in a pre-constructed database, and performing three-dimensional registration on the real-time original point cloud data and the reference point cloud data so that the real-time original point cloud data and the reference point cloud data are aligned in a spatial coordinate system to obtain registered point cloud data; calculating distance differences between points in the real-time original point cloud data and points in the reference point cloud data based on the registered point cloud data, generating a wear depth map according to the distance differences, performing mask processing on the wear depth map based on the binary mask image to extract depth data of the wear area, and calculating a wear volume and / or an average wear depth according to the depth data of the wear area, wherein the wear volume and / or the average wear depth are taken as a final wear amount.

2. The method of claim 1, wherein The method for performing feature extraction on the real-time original point cloud data to obtain high-level semantic features and performing feature extraction on the real-time image texture data to obtain local binary pattern features and histogram of oriented gradient features comprises the following steps: using a filtering algorithm to identify noise points and outlier points in the real-time original point cloud data, and removing the identified noise points and outlier points to obtain denoised real-time point cloud data; using a moving least squares method to perform smoothing processing on the denoised real-time point cloud data to obtain real-time point cloud data; performing contrast enhancement on the real-time image texture data to obtain enhanced real-time image texture data, converting the enhanced real-time image texture data from a color image to a grayscale image to obtain grayscale image texture data, and performing denoising processing on the grayscale image texture data based on a median filtering to obtain smoothed image texture data; using a capsule network to perform feature extraction on the real-time point cloud data to obtain high-level semantic features; using an LBP feature extraction algorithm to perform feature extraction on the smoothed image texture data to obtain local binary pattern features, and using an HOG feature extraction algorithm to perform feature extraction on the smoothed image texture data to obtain histogram of oriented gradient features.

3. A method of detecting the wear amount of a runner blade of a hydraulic turbine according to claim 2, characterized in that, The method for performing feature extraction on the real-time point cloud data to obtain high-level semantic features using a capsule network comprises the following steps: projecting the real-time point cloud data according to a pre-constructed parameterized weight matrix to obtain a plurality of prediction vectors; calculating attention coefficients between the plurality of prediction vectors using a self-attention mechanism, and performing weighted summation on the plurality of prediction vectors based on the attention coefficients to obtain an output vector; performing nonlinear conversion on the output vector based on a compression function to obtain high-level semantic features.

4. The method of claim 2, wherein the method is characterized by: The LBP feature extraction algorithm is used for feature extraction on the smooth image texture data to obtain a local binary pattern feature, and the HOG feature extraction algorithm is used for feature extraction on the smooth image texture data to obtain a histogram of oriented gradients feature, including: A preset radius and a preset number of sampling points are obtained, each pixel point in the smooth image texture data is extracted, each pixel point is taken as a center point, a plurality of sampling center points are obtained by randomly selecting the center point based on the preset number of sampling points, and an LBP value of a pixel in a neighborhood of each sampling center point is calculated based on the preset radius and the sampling center point; A histogram of the LBP value of the pixel in the neighborhood of each sampling center point is calculated, and the histogram is normalized to obtain a normalized histogram, and the normalized histogram is taken as the local binary pattern feature; Gradient intensity and direction of the smooth image texture data are calculated; The smooth image texture data is divided into a plurality of cells, and a histogram of the gradient direction is calculated in each cell to obtain a single direction histogram of oriented gradients feature of each cell, a plurality of cells are combined to obtain a block, and the single direction histogram of oriented gradients feature of each cell in the block is spliced to obtain the histogram of oriented gradients feature.

5. The method of claim 1, wherein the method is characterized by: The fusion feature is dimensionally reduced to obtain a main feature, and the main feature is input into a pre-trained wear area recognition model to output a binary mask image, including: The fusion feature is dimensionally reduced by principal component analysis to obtain the main feature; The main feature is oversampled by using a synthetic minority over-sampling technique to balance the classes of the main feature to obtain an oversampled main feature; The oversampled main feature is input into the pre-trained wear area recognition model to output the binary mask image.

6. A method of detecting the wear amount of a runner blade of a hydraulic turbine according to claim 5, characterized in that The pre-trained wear area recognition model includes an encoder and a decoder, the encoder includes an input layer, a plurality of convolutional layers and a plurality of pooling layers; the oversampled main feature is input into the input layer, and the convolutional layer is used to perform one convolutional processing on the oversampled main feature to obtain a deep feature map, the pooling layer is used to perform a down-sampling operation on the deep feature map to obtain a dimensionally reduced feature map, and the convolutional layer is used again to perform two convolutional processing on the dimensionally reduced feature map to obtain a second deep feature map; The one convolutional processing, the down-sampling operation and the two convolutional processing are repeated in the encoder until each convolutional layer and each pooling layer are used to obtain a plurality of deep feature maps, and the plurality of deep feature maps include a final encoding feature map; The final encoding feature map is one-time two times up-sampling processed to obtain a processed final encoding feature map, the processed final encoding feature map is fused with the corresponding deep feature map to obtain a feature fusion map, the convolutional layer is used to perform convolutional processing on the feature fusion map to obtain a deep feature fusion map, and the deep feature fusion map is two times up-sampling processed again to obtain a processed deep feature fusion map; ​ The above one-time two times upsampling processing, feature fusion operation, convolution processing and second two times upsampling processing are repeated in the decoder until the size of the output image is the same as that of the main feature after oversampling, so as to obtain a binary mask image.

7. The method of claim 1, wherein the method is characterized by: The real-time original point cloud data and the reference point cloud data are three-dimensionally registered so that the real-time original point cloud data and the reference point cloud data are aligned under a spatial coordinate system to obtain registered point cloud data, including: The real-time original point cloud data and the reference point cloud data are filtered, and the filtered real-time original point cloud data and the filtered reference point cloud data are down-sampled to obtain pre-processed real-time original point cloud data and pre-processed reference point cloud data; The corresponding near point of each point in the pre-processed real-time original point cloud data in the pre-processed reference point cloud data is searched, and a rigid transformation parameter that minimizes the distance error between the corresponding near point and the point is calculated, the rigid transformation parameter including a rotation matrix and a translation vector; The pre-processed real-time original point cloud data is translated and rotated based on the rotation matrix and the translation vector to obtain transformed real-time original point cloud data. The transformed real-time original point cloud data and the pre-processed reference point cloud data are subjected to average distance calculation to obtain a point set average distance, and a preset distance threshold is compared with the point set average distance, if the point set average distance is less than or equal to the preset distance threshold, the transformed real-time original point cloud data is taken as the registered point cloud data.

8. The method of claim 1, wherein the method is characterized by: A projection plane is defined, and the projection plane is divided into a plurality of pixel grids, the distance difference is projected onto the plurality of pixel grids, for each pixel grid, all three-dimensional points projected into the pixel grid are searched, and the value of the pixel grid is set to the maximum value or the average value of the distance value in the distance difference corresponding to the three-dimensional points, the maximum value is used to represent the deepest wear point of the blade position corresponding to the pixel grid, and the average value is used to represent the average wear condition of the blade position corresponding to the pixel grid, to obtain a wear depth map; The wear depth map and the binary mask image are spatially aligned to extract the depth data of the wear region in the binary mask image according to the wear depth map; The depth data corresponding to all pixel points in the wear region is subjected to average value calculation to obtain an average wear depth, and the volume element calculation is performed on all pixel points in the wear region to obtain a wear volume, and the wear volume and / or the average wear depth are taken as the final wear amount. including:

9. A system for detecting an abrasion amount of a runner blade of a hydraulic turbine, for implementing the method according to any one of claims 1 to 8, characterized by ​ The feature extraction module is configured to acquire real-time multi-modal data of a runner blade of a hydraulic turbine collected by a multi-modal sensing unit, the real-time multi-modal data including real-time original point cloud data and real-time image texture data, perform feature extraction on the real-time original point cloud data to obtain high-level semantic features, and perform feature extraction on the real-time image texture data to obtain local binary pattern features and histogram of oriented gradient features. The fusion recognition module is configured to splice and fuse the extracted high-level semantic features, local binary pattern features and histogram of oriented gradient features to obtain fusion features, perform dimension reduction processing on the fusion features to obtain main features, and input the main features into a pre-trained wear region recognition model to output a binary mask image, the binary mask image being used to identify a wear region. The registration alignment module is configured to acquire reference point cloud data of the runner blade of the hydraulic turbine in a pre-constructed database, perform three-dimensional registration on the real-time original point cloud data and the reference point cloud data, so that the real-time original point cloud data and the reference point cloud data are aligned under a spatial coordinate system to obtain registered point cloud data. The wear calculation module is configured to calculate a distance difference between points in the real-time original point cloud data and points in the reference point cloud data based on the registered point cloud data, generate a wear depth map according to the distance difference, perform mask processing on the wear depth map based on the binary mask image to extract depth data of the wear region, calculate a wear volume and / or an average wear depth according to the depth data of the wear region, and take the wear volume and / or the average wear depth as a final wear amount.

10. A computer device, comprising: The device comprises a memory, a processor and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to transmit and receive messages, and the processor is configured to read the computer program and execute the method for detecting a wear amount of a runner blade of a hydraulic turbine according to any one of claims 1 to 8.

Images