Method for measuring roughness of aero-engine blade of continuum robot based on multi-modal fusion

Abstract

The present application relates to a kind of roughness measurement methods of continuum robot navigation engine blade based on multi-modal fusion, aero-engine blade surface roughness measurement field, comprising the following steps: step 1, construct multi-modal input data;Step 2, adopt dynamic graph convolution network to carry out adaptive topological modeling to point cloud;Step 3, the blade body point cloud is projected to parameter domain, and according to local point cloud density and surface curvature, multi-scale uniform grid division is carried out;Step 4, the spatial context relationship between grid units is modeled by introducing graph attention mechanism, and the arithmetic mean height S a of optimized surface is calculated;Step 5, compare the measurement result with the preset average value tolerance, standard deviation range, judge whether the blade surface quality is qualified or not.The present application method can meet the high precision, high stability demand in the online detection of aero-engine blade, has the advantages such as high segmentation precision, good measurement repeatability, strong anti-interference ability.

Description

Technical Field

[0001] This invention relates to the field of aero-engine blade surface roughness measurement technology, and in particular to a method for measuring aero-engine blade roughness based on a multimodal fusion continuum robot. The method utilizes multimodal sensors mounted on a continuum robot to collect blade point cloud and image data, and achieves high-precision segmentation and quantitative analysis of the complex curved surface of the blade through dynamic graph convolution and graph attention mechanisms, so as to improve the accuracy, stability and online detection capability of aero-engine blade surface quality inspection. Background Technology

[0002] As a core hot-end component of an aero-engine, the blade operates for extended periods in high-temperature, high-pressure, and corrosive gas environments. Its surface quality directly determines the engine's aerodynamic efficiency, fatigue life, and operational reliability. Surface roughness is a key parameter for evaluating blade performance; excessive roughness can lead to surface airflow separation, stress concentration, and fatigue crack initiation, seriously threatening flight safety. Therefore, achieving high-precision online detection of blade surface roughness is of significant engineering importance for engine manufacturing and maintenance. However, blades possess complex free-form surface structures, with significant differences in geometric characteristics and dramatic curvature variations in parts such as the blade body, tenon, crown, and transition area. Traditional measurement methods struggle to achieve stable and accurate roughness assessments on complex surfaces.

[0003] Existing surface roughness measurement methods are mainly divided into two categories: contact and non-contact. While contact profilometers can provide high-precision local measurements, they are limited by probe size and the complex geometry of the blade, making it difficult to reach deep cavities such as the blade root and junction. Furthermore, the probe is prone to damaging the blade surface during measurement, and the scanning efficiency is low, failing to meet the needs of large-scale online inspection. Non-contact optical methods (such as white light interferometry and confocal microscopy) avoid surface damage, but the equipment is expensive and the measurement is time-consuming, mainly limiting them to laboratory environments. In recent years, intelligent inspection technologies based on machine vision and 3D point clouds have been initially applied, but still face two major bottlenecks: First, single-modal data (point cloud only or image only) cannot simultaneously capture macroscopic geometric structures and microscopic texture features, resulting in insufficient segmentation accuracy in weakly textured and high-curvature areas, making it impossible to accurately locate the blade area to be measured. Second, traditional roughness measurement methods directly statistically analyze the entire point cloud, lacking effective separation of macroscopic morphology, microscopic texture, and measurement noise in the surface signal, making them susceptible to interference from outliers and regional heterogeneity, resulting in poor repeatability of measurement results. Therefore, there is an urgent need for a technical solution that can integrate multimodal information, adaptively process complex surfaces, and achieve high-precision and high-stability online roughness measurement. Summary of the Invention

[0004] This invention proposes a method for measuring the roughness of aero-engine blades using a continuum robot based on multimodal fusion. It aims to solve the problems of low accuracy, poor efficiency, and easy surface damage in traditional contact measurement methods, as well as the problems of blurred segmentation boundaries, susceptibility to abnormal interference, and poor repeatability in existing non-contact methods on complex curved surfaces of blades.

[0005] The technical solution adopted to achieve the above objectives is:

[0006] A method for measuring the roughness of aero-engine blades using a continuum robot based on multimodal fusion includes the following steps:

[0007] Step 1: Simultaneously acquire 3D point cloud data and grayscale images of aero-engine blades using a 3D structured light camera and a grayscale camera mounted on the end of the continuum robot to construct multimodal input data;

[0008] Step 2: Adaptive local topology modeling of the point cloud is performed using a dynamic graph convolutional network, and the texture features of the grayscale image are fused with the geometric features of the point cloud. Combining channel attention mechanism and spatial attention mechanism, semantic segmentation is performed on the leaf point cloud to extract the point cloud of the leaf body region.

[0009] Step 3: Project the point cloud of the blade region onto the parameter domain, divide it into a multi-scale uniform grid based on the local point cloud density and surface curvature, traverse each grid cell and analyze the statistical distribution and frequency characteristics of the internal elevation signal, and remove interfering grid cells with discontinuous curvature and abnormal morphology.

[0010] Step 4: Construct a graph structure based on the retained effective mesh cells, use a graph attention mechanism to model the spatial context relationships between meshes, perform global optimization on the mesh height features to suppress outlier interference, and calculate the arithmetic mean height S of the optimized surface. a The roughness values ​​are recorded, along with their mean and standard deviation.

[0011] Step 5: Compare the calculated roughness values ​​and their mean and standard deviation with the preset average tolerance and standard deviation range to determine whether the blade surface quality is qualified. If all are within the preset range, it is considered a qualified product; otherwise, it is considered an unqualified product. This invention directly uses non-contact multimodal fusion point cloud as the calculation basis, combined with the flexible pose adjustment capability of the continuum robot, reducing human error and accessibility limitations of complex curved surfaces in traditional contact measurement processes.

[0012] Step 1 includes the following steps:

[0013] 1.1 Control the end effector motion of the continuum robot, adjust the onboard 3D structured light camera and grayscale camera to the preset scanning start pose, and ensure that the angle between the camera optical axis and the normal direction of the blade surface to be measured is within the set threshold range.

[0014] 1.2 Adjust the brightness and illumination angle of the blue LED linear light source to form uniform illumination on the leaf surface, suppress overexposure in highly reflective areas and signal loss in shadow areas, and improve the clarity of grayscale image texture and point cloud acquisition quality;

[0015] 1.3 Based on the geometry and size of the blade, plan the scanning path of the continuum robot so that the camera sequentially covers key areas such as the crown, blade, edge plate and root of the blade, and automatically adjust the camera tilt angle in the transition area where the curvature changes drastically to ensure data integrity.

[0016] 1.4 During the scanning process, the end effector is kept stable by the active compliant control of the continuum robot, and the blade vibration and displacement are eliminated in conjunction with the fixing device. Simultaneously, the 3D structured light camera and grayscale camera are triggered to acquire multiple frames of data.

[0017] 1.5 The collected raw point cloud data is filtered and denoised, and outlier points are removed. The grayscale image is copied to generate a pseudo-RGB image to meet the input requirements of the pre-trained feature extractor.

[0018] 1.6 Based on camera calibration parameters, the point cloud data and grayscale image are spatially aligned, background noise outside the blades is cropped and removed, the complete blade point cloud and corresponding image area are retained, and the processed data is saved as a specified format file to provide input for subsequent segmentation and measurement.

[0019] Specifically, step 2 is as follows:

[0020] 2.1 Leaf point cloud collected and preprocessed in step 1 The corresponding grayscale image and its generated pseudo-RGB image are used as input, where N is the total number of point clouds, and each point... Based on its three-dimensional coordinates and color features obtained by sampling corresponding pixels from pseudo-RGB images Together, they constitute and initialize the point features of layer 0 of the network. It is a concatenated vector of the three-dimensional coordinates of a point and its color features, i.e. ,

[0021] 2.2 In the l-th layer, l≥1, for each point and its current features Based on the feature space, a K-nearest neighbor search is performed to dynamically construct a local directed graph, resulting in a set of neighbor indices.

[0022]

[0023] in, Indicates the first Midpoint of layer The set of neighbor indices, where K is the preset number of neighbors. For point In the The feature vector of the layer, For the first The set of feature vectors of all N points in the layer. The function returns the result based on the Euclidean distance in the feature space. The indexes of the K most similar neighbors.

[0024] 2.3 For each point and its dynamic neighbors Calculate directed edge features and encode them using a shared multilayer perceptron:

[0025]

[0026] in, This represents a vector concatenation operation. For the multilayer perceptron shared by layer l, the parameters are: ,

[0027] 2.4 Max pooling is used to aggregate the edge features of all neighbors, and the features of the point are updated:

[0028]

[0029] 2.5 Set of edge features output from convolution of dynamic graphs The input channel attention module first compresses the channel descriptors along the spatial dimensions using global average pooling:

[0030]

[0031] Where C is the number of feature channels, K is the number of neighbors for each point, and channel attention weights are generated through a two-layer fully connected network and a sigmoid activation function:

[0032]

[0033] in, , , For the reduction rate, It is the ReLU activation function. For the Sigmoid function,

[0034] Finally, the original edge features are multiplied by the channel weights channel by channel:

[0035]

[0036] in, This represents element-wise multiplication. This represents the edge feature vector after channel attention calibration.

[0037] 2.6 Side feature vectors after channel attention calibration The input spatial attention module calculates an attention score for each neighboring point using a lightweight multilayer perceptron.

[0038]

[0039] in, This represents a lightweight multilayer perceptron used to map the edge features of the input to this scalar score and perform Softmax normalization on the scores of all neighbors:

[0040]

[0041] in, Represents the normalized spatial attention weights, quantizing the neighbors. Point Based on the relative importance of each element, the edge features are weighted and aggregated using normalized spatial weights to obtain preliminary aggregated point features:

[0042]

[0043] Then The input channel attention module performs secondary calibration and outputs the final l-th layer point features.

[0044] The core improvement in this step is that existing methods typically input all points directly into the network in point cloud segmentation, lacking cross-modal fusion design for input features. Furthermore, after static neighborhood division, the same convolutional kernel is used for feature extraction in each neighborhood, making it impossible to adaptively adjust the feature aggregation method according to local geometry. This step incorporates three targeted improvements at the technical implementation level: First, it concatenates point cloud coordinates with corresponding pixel color features as the input to layer 0, enabling the fusion of geometric and texture information early in the feature space, rather than simply overlaying features in deeper layers of the network, thus enhancing the efficiency of multimodal information interaction. Second, it re-searches for neighbors based on the current features at each layer, allowing the receptive field to dynamically change with network depth. Shallow neighbors reflect local geometric details, while deep neighbors focus on semantically relevant regions, resolving the contradiction that a single fixed neighborhood cannot simultaneously consider local details and global semantics. Third, channel attention applies a uniform global weight to each channel, while spatial attention applies independent local weights to each neighbor. This forms a progressive "global-local" selection process, first finding semantically relevant feature dimensions through channel selection, and then locating geometrically critical neighbors through spatial selection, avoiding redundant computations that might result from parallel or sequential exchanges of these two approaches. These improvements allow the feature update of each point to adaptively adjust the aggregation strategy according to its own geometric region (flat blade area, tenon abrupt change area, transition area), improving the discrimination accuracy of the segmentation network on complex blade surfaces.

[0045] 2.7 Stack multi-layer dynamic graph convolution and two-dimensional attention modules, and connect a fully connected classifier after the last layer to output the semantic label of each point belonging to the leaf body, tenon, leaf crown and other parts, and extract the point cloud of the leaf body region accordingly.

[0046] The core improvement in this step lies in: deeply fusing the texture features of grayscale images with the geometric features of point clouds, enabling each point cloud point to simultaneously possess three-dimensional coordinates and RGB color information, thus solving the problem of insufficient segmentation capability of single-modal data in weakly textured and high-curvature regions; adaptively constructing local topology in the feature space through dynamic graph convolution, overcoming the defect of poor adaptability of static neighborhood partitioning to changes in point cloud density; and achieving channel-space dual-dimensional attention mechanism to realize channel-level recalibration of edge features and normalized weighted aggregation of neighboring points, and introducing secondary calibration to optimize feature representation, thereby enhancing the boundary perception capability of complex transition regions such as blade transition areas and leaf roots. Existing point cloud segmentation methods in roughness measurement often rely on single geometric information, static neighborhood partitioning, and equal feature processing, resulting in inaccurate segmentation of weakly textured and high-curvature regions and blurred complex transition boundaries. Compared with existing methods, this step effectively suppresses over-segmentation and under-segmentation by using multimodal feature fusion, dynamic graph convolution adaptive topology modeling, and channel-space dual-dimensional attention mechanism. This achieves higher overall segmentation accuracy in the blade point cloud segmentation task and provides accurate blade region masks for subsequent roughness measurement.

[0047] Specifically, step 3 is as follows:

[0048] 3.1 Project the 3D point cloud of the blade region extracted in step 2 onto the least squares fitting plane, establish a 2D parameter domain, calculate the bounding box of the point cloud in the parameter domain, and record the elevation values ​​of each point. (i.e., the signed distance from the point to the fitted plane);

[0049] 3.2 A multi-scale uniform grid partitioning strategy is adopted, based on the local point cloud density. With average surface curvature The size of each mesh cell is adaptively determined, with the base mesh size constant set to . The global average point cloud density is The global mean curvature is The weighting coefficient is , Then it is located at the parameter coordinates Grid side length at the location Calculated by the following formula:

[0050]

[0051] according to The parameter domain is divided into a series of uniform square micro-element grids, each grid... It contains a set of three-dimensional points and their corresponding elevations;

[0052] 3.3 For each grid The statistical distribution and frequency characteristics of the internal elevation signals are analyzed, and the comprehensive confidence weight of each point is calculated to obtain the representative height value of the grid.

[0053] 3.3.1 Statistically analyze the elevation values ​​of all points within the grid, plot an elevation histogram, identify the dominant mode through peak detection, and obtain the elevation values ​​for each point. Statistical weights This represents the probability that the point belongs to the true surface morphology;

[0054] 3.3.2 After sorting the points within the grid by elevation to form a sequence, perform a Fast Fourier Transform to calculate the power spectral density. Components with concentrated spectral energy in the mid-to-high frequency range are identified as valid texture signals, thus obtaining the power spectral density for each point. Frequency domain weights ;

[0055] 3.3.3 Integrate statistical weights and frequency domain weights to calculate the weights for each point. The overall confidence weight;

[0056] 3.3.4 Computational Grid The representative height value;

[0057] 3.4 Traverse all mesh elements and calculate the local curvature of each mesh. Let the mean curvature of all grids be... The standard deviation is The coefficients were determined through cross-validation. (k ranges from 1 to 3), define the dynamic noise threshold:

[0058]

[0059] If the curvature of a certain grid satisfies Or the number of valid points in the grid is less than the preset minimum number of points threshold. If the mesh is found to be a non-interference mesh element with discontinuous curvature or abnormal shape, it is discarded; all mesh elements that meet the requirements are retained. And a sufficient number of grid cells and its representative height value The output is then sent to step 4 as a valid mesh.

[0060] The core improvement in this step lies in the fact that traditional roughness measurement methods typically involve directly statistically analyzing the entire point cloud or using fixed-size meshes, making it difficult to separate macroscopic surface morphology, microscopic texture, and measurement noise, and are susceptible to interference from high-curvature regions and outliers. This step first uses a multi-scale uniform mesh, adaptively adjusting the mesh size based on local point cloud density and curvature to accommodate geometric variations in different regions of the blade. Then, it integrates the statistical weights of the elevation histogram and the frequency domain weights of the power spectral density to calculate the comprehensive confidence level of each point, effectively distinguishing between true morphology signals and noise. Finally, it dynamically sets thresholds based on the mean and standard deviation of curvature to automatically eliminate interfering mesh cells with curvature discontinuities and abnormal morphology. This process effectively suppresses the amplification effect of outlier clusters on roughness calculations, providing clean, reliable, and effective mesh data for subsequent global optimization. Existing methods often directly statistically analyze the entire point cloud or use fixed meshes, making it difficult to separate true texture and noise, and susceptible to interference from high-curvature regions and outliers. Compared with existing methods, this step effectively suppresses the bias amplification effect of abnormal point clusters on roughness calculation by using multi-scale adaptive mesh generation, statistical-frequency domain fusion weighting, and dynamic curvature threshold removal, thus significantly improving the stability and anti-interference capability of the measurement.

[0061] Specifically, step 4 is as follows:

[0062] 4.1 Retain the valid mesh cells from step 3 Viewed as nodes in a graph structure, each node's initial feature vector Based on its representative height value Grid center coordinates and local curvature It is pieced together;

[0063] 4.2 Constructing a graph edge structure based on the spatial adjacency relationship of the mesh: If two mesh cells and If two neighbors are adjacent within the parameter domain (including 4-neighborhoods or 8-neighborhoods), then an undirected edge is established between them. Finally, a graph structure is obtained. ,in For a set of nodes, It is an edge set;

[0064] 4.3 A graph attention mechanism is used to update node features. For the target node m and its neighboring nodes, the shared weight matrix is ​​first used to update the node features. Perform a linear transformation on the input features, and then calculate the attention coefficients:

[0065]

[0066] in, For attention vectors, Let m be the feature vector of node m in the t-th iteration; perform Softmax normalization on the attention coefficients:

[0067]

[0068] Normalized attention weights The importance of neighbor node n to node m was quantified;

[0069] 4.4 The neighbor features are weighted and summed based on attention weights, and a multi-head attention mechanism is used to enhance the stability of the representation. The updated features of node m are:

[0070]

[0071] Where H represents the number of attention heads. Let h be the attention weight of the h-th head. This is the corresponding weight matrix. The activation function is non-linear; after multiple iterations, the final optimized node features are obtained. Take its first dimension as the optimized grid height value. This forms the optimized height matrix. ;

[0072] 4.5 Based on the optimized height matrix Fit a least-squares reference plane with a center point over the entire measurement area: Let the plane equation be... By minimizing all grids Corresponding optimization height Solve for plane parameters by summing the squared distances to the plane; calculate the optimal height for each mesh cell. Vertical deviation from the reference plane:

[0073]

[0074] 4.6 Calculation of the arithmetic mean height S of surface roughness a :

[0075]

[0076] And record S a The mean and standard deviation; mean and standard deviation; mean and standard deviation of S, according to industry standards. a Convert to the corresponding R a The numerical value outputs the overall roughness measurement results of the blade surface.

[0077] The core improvement in this step lies in the fact that traditional coarsening methods typically perform simple statistical analysis or local filtering on grid height, lacking modeling of the spatial context relationships between grids and making it difficult to effectively suppress the global impact of outliers. This step constructs effective grid cells as a graph structure, introduces a graph attention mechanism to adaptively learn the spatial dependencies between adjacent grids, and employs multi-head attention to enhance representation stability, achieving global optimization of height features. Based on this, a least-squares reference plane is fitted and the arithmetic mean height is calculated. This effectively suppresses the interference of local abnormal point clusters, improving the repeatability and noise resistance of roughness measurement.

[0078] Specifically, step 5 is as follows:

[0079] 5.1 Calculate the mean roughness value of all effective mesh elements obtained in step 4. and standard deviation A normal distribution plot was drawn to show the distribution of roughness values, and the range of three standard deviations was calculated. The corresponding roughness values ​​are used to analyze the concentration and dispersion of the roughness distribution;

[0080] mean The calculation formula is:

[0081]

[0082] Standard deviation The calculation formula is:

[0083]

[0084] in, The total number of effective grid cells. Let be the roughness value of the m-th mesh element;

[0085] 5.2 Compare the mean roughness values ​​before and after removing grid cells with curvature discontinuities and abnormal shapes in step 3: Before removal, the roughness values ​​are relatively scattered due to interference from high curvature regions and noise points; after removal, the remaining point cloud data tends to be closer to the mean. Furthermore, the standard deviation decreased significantly, indicating that noise and high curvature mesh cells had a significant impact on the roughness measurement results, verifying that the interference mesh removal operation in step 3 is a necessary preprocessing step to ensure measurement accuracy.

[0086] 5.3 Based on the calculated mean roughness value and standard deviation The tolerance range between the preset average value and the actual value. and standard deviation threshold Compare and evaluate the surface quality of the blades:

[0087] like and If the surface roughness of the blade is qualified, it is considered a qualified product.

[0088] like Exceeding the preset average tolerance range, or If the value exceeds the standard deviation threshold, the product is deemed unqualified, the reason for non-compliance is output (mean value exceeds the standard deviation), and rework or further testing is recommended.

[0089] The core improvement in this step lies in the fact that traditional roughness acceptance criteria typically only compare measured values ​​with a fixed threshold, lacking in-depth analysis of the statistical distribution of measurement results and failing to verify the effectiveness of preprocessing steps. This step, by calculating the roughness mean and standard deviation and plotting a normal distribution diagram and three standard deviation ranges, quantitatively assesses the uniformity of surface processing. Then, by comparing the roughness mean before and after removing interfering meshes, the significant reduction in standard deviation quantitatively verifies the necessity and effectiveness of the interference removal operation in step 3. Finally, a comprehensive judgment is made based on the preset tolerance range and standard deviation threshold, distinguishing between two reasons for non-compliance: mean deviation exceeding the tolerance and standard deviation exceeding the tolerance. This process not only provides a conclusion on whether the surface is acceptable or not, but also offers a basis for the confidence level of measurement quality and rework suggestions, significantly improving the interpretability and engineering practicality of the inspection system.

[0090] The present invention has the following beneficial effects and advantages:

[0091] 1. This invention employs a continuum robot equipped with multimodal sensors (3D structured light camera and grayscale camera) for non-contact data acquisition. Utilizing the flexible pose adjustment capability of the continuum robot, it can adaptively conform to the complex curved surface of the blade for scanning, effectively overcoming the problems of traditional contact measurement methods that are difficult to reach deep cavity areas, easily damage surfaces, and have large human errors, thus achieving high degree of freedom and high-fidelity data acquisition.

[0092] 2. This invention proposes a structure-aware segmentation network based on dynamic graph convolution and two-dimensional attention mechanism. By dynamically constructing local topology in the feature space and combining it with channel-space attention, it significantly enhances the boundary perception capability of complex transition regions such as blade transition areas and blade roots. It solves the problems of blurred boundaries, undersegmentation or oversegmentation in traditional segmentation methods at points of drastic surface changes, and provides accurate region masks for subsequent roughness measurement.

[0093] 3. This invention proposes a robust roughness quantization method based on multi-scale uniform grid division and graph attention context fusion. By using statistical-frequency domain weighted fusion within the grid and global optimization of graph attention between grids, it effectively suppresses the interference of local curvature abrupt changes and abnormal point clusters, and achieves micron-level high-precision and high-stability roughness measurement.

[0094] 4. This invention constructs an integrated measurement framework covering the entire process from multimodal data acquisition and structure-aware segmentation to robust roughness measurement. Combined with the online deployment capability of a continuum robot, it can meet the real-time, accurate, and non-destructive testing requirements of in-service inspection and large-scale production of aero-engine blades, providing reliable technical support for blade surface quality assessment, process closed-loop control, and life prediction.

[0095] 5. This invention utilizes the active compliant control of a continuum robot to maintain end-effector stability and eliminate blade vibration during scanning, while simultaneously achieving safe rigid-flexible coupling contact with a fixing device. Compared to traditional rigid industrial robots, continuum robots possess inherent flexibility and passive compliance, avoiding collision damage caused by path deviations in deep blade cavity regions or complex curved surface transition sections. This makes them particularly suitable for non-destructive online inspection of high-value components such as aero-engine blades, expanding the application boundaries of robots in the field of precision measurement.

[0096] 6. This invention not only outputs a judgment on whether the surface roughness is acceptable, but also provides specific reasons for non-compliance, such as the mean deviation or standard deviation exceeding the tolerance. Furthermore, it quantifies the uniformity of blade surface processing by plotting a normal distribution and three standard deviation ranges. This information can be directly fed back to upstream manufacturing processes (such as grinding, polishing, and coating) for adaptive adjustment of process parameters and closed-loop quality control, providing a data-driven process improvement method for intelligent manufacturing of aero-engine blades. Attached Figure Description

[0097] Figure 1 This is a flowchart of a method for measuring the roughness of aero-engine blades using a continuum robot based on multimodal fusion, as described in this invention. Detailed Implementation

[0098] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0099] Example: Figure 1 As shown, a method for measuring the roughness of aero-engine blades of a continuum robot based on multimodal fusion includes the following steps;

[0100] Step 1: Multimodal data acquisition: Simultaneously acquire 3D point cloud data and grayscale images of aero-engine blades using a 3D structured light camera and a grayscale camera mounted on the end effector of the continuum robot to construct multimodal input data.

[0101] Step 1.1: Control the end effector movement of the continuum robot, adjust the onboard 3D structured light camera and grayscale camera to the preset scanning start pose, and ensure that the angle between the camera optical axis and the normal direction of the blade surface to be measured is within the set threshold range (e.g., within ±15°).

[0102] Step 1.2: Adjust the brightness and illumination angle of the blue LED linear light source to form uniform illumination on the leaf surface, suppress overexposure in highly reflective areas and signal loss in shadow areas, and improve the clarity of grayscale image texture and point cloud acquisition quality.

[0103] Step 1.3: Based on the geometry and size of the blade, plan the scanning path of the continuum robot so that the camera sequentially covers key areas such as the crown, blade body, edge plate and root of the blade, and automatically adjust the camera tilt angle in the transition area where the curvature changes drastically to ensure data integrity.

[0104] Step 1.4: During the scanning process, the end effector is kept stable by the active compliant control of the continuum robot, and the blade vibration and displacement are eliminated in conjunction with the fixing device. Simultaneously, the 3D structured light camera and grayscale camera are triggered to collect multiple frames of data.

[0105] Step 1.5: Filter and denoise the collected raw point cloud data and remove outliers. Copy the grayscale image channels to generate a pseudo-RGB image to meet the input requirements of the pre-trained feature extractor.

[0106] Step 1.6: Based on the camera calibration parameters, spatially align the point cloud data with the grayscale image, crop and remove background noise outside the blades, retain the complete blade point cloud and corresponding image area, and save the processed data as a specified format file (such as .pcd, .ply) to provide input for subsequent segmentation and measurement.

[0107] Step 2, Multimodal Structure-Aware Segmentation: Adaptive local topology modeling of the point cloud is performed using a dynamic graph convolutional network, and the texture features of the grayscale image are fused with the geometric features of the point cloud. Combining channel attention mechanism and spatial attention mechanism, semantic segmentation of the leaf point cloud is performed to extract the point cloud of the leaf body region.

[0108] Step 2.1: Using the leaf point cloud collected and preprocessed in Step 1. The corresponding grayscale image and its generated pseudo-RGB image are used as input, where N is the total number of point clouds, and each point... And initialize the point features of layer 0 of the network. It is a concatenated vector of the three-dimensional coordinates of a point and its color features, i.e. .

[0109] Step 2.2, in the l-th layer (l≥1), for each point and its current features Based on the feature space, a K-nearest neighbor search is performed to dynamically construct a local directed graph, resulting in a set of neighbor indices.

[0110]

[0111] in, Indicates the first Midpoint of layer The set of neighbor indices, where K is the preset number of neighbors. For point In the The feature vector of the layer, For the first The set of feature vectors of all N points in the layer. The function returns the result based on the Euclidean distance in the feature space. Indexes of the K most similar neighbor points.

[0112] Step 2.3, for each point and its dynamic neighbors Calculate directed edge features and encode them using a shared multilayer perceptron:

[0113]

[0114] in, This represents a vector concatenation operation. For the multilayer perceptron shared by layer l, the parameters are: .

[0115] Step 2.4: Aggregate the edge features of all neighbors using max pooling, and update the features of the point:

[0116]

[0117] Step 2.5: Input the set of edge features output by the dynamic graph convolution into the channel attention module. First, compress it along the spatial dimension (K neighbors) through global average pooling to generate channel descriptors. Then, generate channel attention weights through two fully connected layers and the Sigmoid activation function. Multiply the original edge features and channel weights channel by channel to achieve adaptive recalibration of feature channels.

[0118] Step 2.6: Input the edge features after channel attention calibration into the spatial attention module. Calculate the attention score for each neighbor point using a lightweight multilayer perceptron. After Softmax normalization, obtain the spatial attention weight. Use this weight to perform weighted aggregation on the edge features to obtain the preliminary aggregated point features. Then input these features into the channel attention module for secondary calibration and output the final l-th layer point features.

[0119] Step 2.7: Stack multiple layers (usually 3 to 4 layers) of dynamic graph convolution and two-dimensional attention modules, and connect a fully connected classifier after the last layer to output the semantic label of each point belonging to the leaf body, tenon, leaf crown and other parts, and extract the point cloud of the leaf body region accordingly.

[0120] Step 3: Multi-scale mesh generation and interference removal: Project the point cloud of the blade region onto the parameter domain, perform multi-scale uniform mesh generation based on the local point cloud density and surface curvature, traverse each mesh cell and analyze the statistical distribution and frequency characteristics of the internal elevation signal, and remove interference mesh cells with discontinuous curvature and abnormal morphology.

[0121] Step 3.1: Using the point cloud of the leaf region extracted in Step 2. The 3D point cloud is projected onto the least squares fitting plane to establish a 2D parameter domain, and the bounding box of the point cloud within the parameter domain is calculated. Record the elevation values ​​of each point. (i.e., the signed distance from the point to the fitted plane).

[0122] Step 3.2: Adopt a multi-scale uniform grid division strategy, based on local point cloud density. With average surface curvature The size of each mesh cell is adaptively determined, with the base mesh size constant set to . The global average point cloud density is The global mean curvature is The weighting coefficient is , Then it is located at the parameter coordinates Grid side length at the location Calculated by the following formula:

[0123]

[0124] according to The parameter domain is divided into a series of uniform square micro-grids. Each grid It contains a set of three-dimensional points and its corresponding elevation .

[0125] Step 3.3, for each grid The statistical distribution and frequency characteristics of the internal elevation signals are analyzed, and the comprehensive confidence weight of each point is calculated to obtain the representative height value of the grid.

[0126] Step 3.3.1: Calculate the elevation values ​​of all points within the grid, plot an elevation histogram, identify the dominant mode through peak detection, and obtain the statistical weight of each point j. This represents the probability that the point belongs to the true surface morphology.

[0127] Step 3.3.2: Sort the points in the grid by elevation to form a sequence, perform a Fast Fourier Transform, calculate the power spectral density, and determine the components with concentrated spectral energy in the mid-to-high frequency part as valid texture signals to obtain the frequency domain weight of each point j. .

[0128] Step 3.3.3: Combine statistical weights and frequency domain weights to calculate the overall confidence weight for each point j:

[0129]

[0130] Step 3.3.4: Calculate the mesh Representative height values:

[0131]

[0132] in, This represents the overall confidence weight of point j. Represents a grid The elevation value of the j-th point (i.e., the signed distance from that point to the fitted plane).

[0133] Step 3.4: Traverse all mesh elements and calculate the local curvature of each mesh. Let the mean curvature of all grids be . The standard deviation is The coefficients were determined through cross-validation. Define the dynamic noise threshold:

[0134]

[0135] If the curvature of a certain grid satisfies Or the number of valid points in the grid is less than the preset minimum number of points threshold. If the mesh is found to be a non-interference mesh element with discontinuous curvature or abnormal shape, it is discarded; all mesh elements that meet the requirements are retained. And a sufficient number of grid cells and its representative height value The output is then sent to step 4 as a valid mesh.

[0136] Step 4: Global Optimization and Roughness Calculation with Graph Attention: A graph structure is constructed based on the retained effective mesh cells. A graph attention mechanism is used to model the spatial context relationships between meshes. Global optimization of the mesh height characteristics is performed to suppress outlier interference. The arithmetic mean height S of the optimized surface is calculated. a The roughness values ​​are recorded, along with their mean and standard deviation.

[0137] Step 4.1: The valid mesh cells retained in Step 3 are... Viewed as nodes in a graph structure, each node's initial feature vector Based on its representative height value Grid center coordinates and local curvature It is pieced together:

[0138]

[0139] Where D is the initial feature dimension. This indicates a vector concatenation operation.

[0140] Step 4.2: Construct the graph edge structure based on the spatial adjacency relationship of the mesh: If two mesh cells and If two neighbors are adjacent within the parameter domain (including 4-neighborhoods or 8-neighborhoods), then an undirected edge is established between them. Finally, a graph structure is obtained. ,in For a set of nodes, It is an edge set.

[0141] Step 4.3: Update node features using a graph attention mechanism for the target node m and its neighboring nodes. (Including self-loops), firstly through the shared weight matrix Perform a linear transformation on the input features, and then calculate the attention coefficients:

[0142]

[0143] in, Represents the attention vector The transpose of is used to map the concatenated feature vector (2D dimension) to a scalar. Let be the feature vector of node m in the t-th iteration. It is a non-linear activation function.

[0144] Softmax normalization is applied to the attention coefficient:

[0145]

[0146] in, This represents the unnormalized attention coefficient between node m and its neighbor node n. This indicates that the attention coefficient is exponentially applied, which is used in Softmax. The summation range of the summation index k includes all neighbors of node m as well as the node itself (i.e., self-loop attention).

[0147] Normalized attention weights The importance of neighbor node n to node m was quantified.

[0148] Step 4.4: The neighbor features are weighted and summed based on attention weights, and a multi-head attention mechanism is used to enhance representation stability. The updated features of node m are:

[0149]

[0150] Where H represents the number of attention heads. Let h be the attention weight of the h-th head. This is the corresponding weight matrix. The activation function is non-linear; multiple iterations are performed to obtain the final optimized node features. Take its first dimension as the optimized grid height value. This forms the optimized height matrix. .

[0151] Step 4.5: Based on the optimized height matrix Fit a least-squares reference plane over the entire measurement area: Let the plane equation be... By minimizing all grid center points Corresponding optimization height The plane parameters are solved by summing the squared distances to the plane. Here, A, B, and C are the three components of the plane's normal vector, determining the plane's direction. D is a constant term determining the signed distance from the plane to the origin. x, y, and z represent the three-dimensional coordinates of a point in space.

[0152] Calculate the optimal height for each grid cell Vertical deviation from the reference plane:

[0153]

[0154] Step 4.6: Calculate the arithmetic mean height S of the surface roughness.a :

[0155]

[0156] in, This represents the total number of valid grid cells. The vertical deviation of the optimized height of the m-th mesh cell from the reference plane. This means summing up the absolute values ​​of the deviations corresponding to each grid in the index.

[0157] And record S a The mean and standard deviation; and according to industry standards, S a Convert to the corresponding R a The numerical value outputs the overall roughness measurement results of the blade surface.

[0158] Step 5, Acceptance Judgment: Compare the calculated roughness values ​​and their mean and standard deviation with the preset average tolerance and standard deviation range to determine whether the blade surface quality is acceptable.

[0159] Step 5.1: Calculate the roughness values ​​of all effective mesh elements obtained in Step 4. That is, to perform statistical analysis on the arithmetic mean height of each grid cell and calculate its mean. and standard deviation And a normal distribution plot is drawn to show the distribution of roughness values.

[0160] mean The calculation formula is:

[0161]

[0162] Standard deviation The calculation formula is:

[0163]

[0164] Step 5.2: Compare the mean roughness values ​​before and after removing grid cells with curvature discontinuities and abnormal shapes in Step 3. Before removal, the roughness values ​​are relatively scattered due to interference from high curvature regions and noise points; after removal, the remaining point cloud data tends towards the mean. Furthermore, the standard deviation decreased significantly, indicating that noise and high curvature mesh cells had a significant impact on the roughness measurement results, verifying that the interference mesh removal operation in step 3 is a necessary preprocessing step to ensure measurement accuracy.

[0165] Step 5.3: Based on the calculated average roughness value and standard deviation The tolerance range between the preset average value and the actual value. and standard deviation threshold Compare and evaluate the surface quality of the blades:

[0166] like and If the surface roughness of the blade is acceptable, it is considered a qualified product; otherwise... Exceeding the preset average tolerance range, or If the value exceeds the standard deviation threshold, the product is deemed unqualified, the reason for non-compliance is output (mean value exceeds the standard deviation), and rework or further testing is recommended.

[0167] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for measuring the roughness of aero-engine blades using a continuum robot based on multimodal fusion, characterized in that, Includes the following steps: Step 1: Simultaneously acquire 3D point cloud data and grayscale images of aero-engine blades using a 3D structured light camera and a grayscale camera mounted on the end effector of the continuum robot to construct multimodal input data; Step 2: Adaptive local topology modeling of the point cloud is performed using a dynamic graph convolutional network, and the texture features of the grayscale image are fused with the geometric features of the point cloud. Combining channel attention mechanism and spatial attention mechanism, semantic segmentation is performed on the leaf point cloud to extract the point cloud of the leaf body region. Step 3: Project the point cloud of the blade region onto the parameter domain, divide it into a multi-scale uniform grid based on the local point cloud density and surface curvature, traverse each grid cell and analyze the statistical distribution and frequency characteristics of the internal elevation signal, and remove interfering grid cells with discontinuous curvature and abnormal morphology. Step 4: Construct a graph structure based on the retained effective mesh cells, use a graph attention mechanism to model the spatial context relationships between meshes, perform global optimization on the mesh height features to suppress outlier interference, and calculate the arithmetic mean height S of the optimized surface. a The roughness values ​​are recorded, along with their mean and standard deviation. Step 5: Compare the calculated roughness values ​​and their mean and standard deviation with the preset mean tolerance and standard deviation range to determine whether the blade surface quality is qualified. If they are all within the preset range, the product is considered qualified; otherwise, it is considered unqualified.

2. The method for measuring the roughness of aero-engine blades based on multimodal fusion for a continuum robot according to claim 1, characterized in that, Step 1 includes the following steps: 1.1 Control the end effector movement of the continuum robot, adjust the onboard 3D structured light camera and grayscale camera to the preset scanning start pose, and ensure that the angle between the camera optical axis and the normal direction of the blade surface to be measured is within the set threshold range. 1.2 Adjust the brightness and illumination angle of the blue LED linear light source to form uniform illumination on the leaf surface, suppress overexposure in highly reflective areas and signal loss in shadow areas, and improve the clarity of grayscale image texture and point cloud acquisition quality. 1.

3. Based on the geometry and size of the blade, plan the scanning path of the continuum robot so that the camera sequentially covers the key areas of the leaf crown, blade body, edge plate and leaf root, and automatically adjust the camera tilt angle in the transition area where the curvature changes drastically to ensure data integrity. 1.4 During the scanning process, the end effector is kept stable by the active compliant control of the continuum robot, and the blade vibration and displacement are eliminated in conjunction with the fixing device. Simultaneously, the 3D structured light camera and grayscale camera are triggered to acquire multiple frames of data. 1.

5. Filter and denoise the collected raw point cloud data and remove outliers. Duplicate the grayscale image channels to generate a pseudo-RGB image to meet the input requirements of the pre-trained feature extractor. 1.6 Based on the camera calibration parameters, spatially align the point cloud data with the grayscale image, crop and remove background noise outside the blades, retain the complete blade point cloud and corresponding image area, and save the processed data as a specified format file to provide input for subsequent segmentation and measurement.

3. The method for measuring the roughness of aero-engine blades based on multimodal fusion for a continuum robot according to claim 2, characterized in that, Step 2 is as follows: 2.1 Leaf point cloud collected and preprocessed in step 1 The corresponding grayscale image and its generated pseudo-RGB image are used as input, where N is the total number of point clouds, and each point... Based on its three-dimensional coordinates and color features obtained by sampling corresponding pixels from pseudo-RGB images Together, they constitute and initialize the point features of layer 0 of the network. It is a concatenated vector of the three-dimensional coordinates of a point and its color features, i.e. , 2.2 In the l-th layer, l≥1, for each point and its current features Based on the feature space, a K-nearest neighbor search is performed to dynamically construct a local directed graph, resulting in a set of neighbor indices. in, Indicates the first Midpoint of layer The set of neighbor indices, where K is the preset number of neighbors. For point In the The feature vector of the layer, For the first The set of feature vectors of all N points in the layer. The function returns the result based on the Euclidean distance in the feature space. The indexes of the K most similar neighbors. 2.3 For each point and its dynamic neighbors Calculate directed edge features and encode them using a shared multilayer perceptron: in, This represents a vector concatenation operation. For the multilayer perceptron shared by the l-th layer, the parameters are: , 2.4 Max pooling is used to aggregate the edge features of all neighbors, and the features of the point are updated: 2.5 Set of edge features output from convolution of dynamic graphs The input channel attention module first compresses the channel descriptors along the spatial dimensions using global average pooling: Where C is the number of feature channels, K is the number of neighbors for each point, and channel attention weights are generated through a two-layer fully connected network and a sigmoid activation function: in, , , For the reduction rate, It is the ReLU activation function. For the Sigmoid function, Finally, the original edge features are multiplied by the channel weights channel by channel: in, This represents element-wise multiplication. This represents the edge feature vector after channel attention calibration. 2.6 Side feature vectors after channel attention calibration The input spatial attention module calculates an attention score for each neighboring point using a lightweight multilayer perceptron. in, This represents a lightweight multilayer perceptron used to map the edge features of the input to this scalar score and perform Softmax normalization on the scores of all neighbors: in, Represents the normalized spatial attention weights, quantizing the neighbors. Point Based on the relative importance of each element, the edge features are weighted and aggregated using normalized spatial weights to obtain preliminary aggregated point features: Then The input channel attention module performs secondary calibration and outputs the final l-th layer point features. 2.7 Stack multi-layer dynamic graph convolution and two-dimensional attention modules, and connect a fully connected classifier after the last layer to output the semantic label of each point belonging to the leaf body, tenon and crown components, and extract the point cloud of the leaf body region accordingly.

4. The method for measuring the roughness of aero-engine blades based on multimodal fusion for a continuum robot according to claim 3, characterized in that, Step 3 specifically involves: 3.1 Point cloud of leaf region extracted in step 2 ,in The 3D point cloud is projected onto the least squares fitting plane, a 2D parameter domain is established, and the bounding box of the point cloud in the parameter domain is calculated. Record the elevation values ​​of each point. , that is, the signed distance from the point to the fitted plane; 3.2 A multi-scale uniform grid partitioning strategy is adopted, based on the local point cloud density. With average surface curvature The size of each mesh cell is adaptively determined, with the base mesh size constant set to . The global average point cloud density is The global mean curvature is The weighting coefficient is , Then it is located at the parameter coordinates Grid side length at the location Calculated by the following formula: according to The parameter domain is divided into a series of uniform square micro-grids. Each grid It contains a set of three-dimensional points and its corresponding elevation ; 3.3 For each grid The statistical distribution and frequency characteristics of the internal elevation signals are analyzed, and the comprehensive confidence weight of each point is calculated to obtain the representative height value of the grid. The elevation values ​​of all points within the grid are statistically analyzed, an elevation histogram is plotted, and the dominant mode is identified through peak detection to obtain the statistical weight of each point j. The probability that a point belongs to the true surface topography is represented; the points in the grid are sorted by elevation to form a sequence, and a fast Fourier transform is performed to calculate the power spectral density. The components with spectral energy concentrated in the mid-to-high frequency part are determined as valid texture signals, and the frequency domain weight of each point j is obtained. By combining statistical weights and frequency domain weights, the overall confidence weight for each point j is calculated. Computational grid Representative height values: in, This represents the overall confidence weight of point j. Represents a grid The elevation value of the j-th point is the signed distance from that point to the fitted plane. 3.4 Traverse all mesh elements and calculate the local curvature of each mesh. Global mean curvature The standard deviation is The coefficients were determined through cross-validation. Define the dynamic noise threshold: If the curvature of a certain grid satisfies Or the number of valid points in the grid is less than the preset minimum number of points threshold. If the mesh is found to be a non-interference mesh element with discontinuous curvature or abnormal shape, it is discarded; all mesh elements that meet the requirements are retained. And a sufficient number of grid cells and its representative height value The output is then sent to step 4 as a valid mesh.

5. The method for measuring the roughness of aero-engine blades of a continuum robot based on multimodal fusion according to claim 4, characterized in that, Step 4 specifically involves: 4.1 Retain the valid mesh cells from step 3 Viewed as nodes in a graph structure, each node's initial feature vector Based on its representative height value Grid center coordinates and local curvature It is pieced together: Where D is the initial feature dimension. This represents a vector concatenation operation; 4.2 Constructing a graph edge structure based on the spatial adjacency relationship of the mesh: If two mesh cells and If neighbors are adjacent within the parameter domain, including 4-neighborhoods or 8-neighborhoods, then an undirected edge is established between them. Finally, a graph structure is obtained. ,in For a set of nodes, It is an edge set; 4.3 A graph attention mechanism is used to update node features for the target node m and its neighboring nodes. First, through the shared weight matrix Perform a linear transformation on the input features, and then calculate the attention coefficients: in, Represents the attention vector The transpose of is used to map the concatenated feature vectors to a scalar. Let be the feature vector of node m in the t-th iteration. It is a non-linear activation function. Softmax normalization is applied to the attention coefficient: in, This represents the unnormalized attention coefficient between node m and its neighbor node n. This indicates that the attention coefficient is exponentially applied, which is used in Softmax. The summation range of the summation index k includes all neighbors of node m and the node itself, i.e., the self-loop attention, and the normalized attention weights. The importance of neighbor node n to node m was quantified. 4.4 The neighbor features are weighted and summed based on attention weights, and a multi-head attention mechanism is used to enhance the stability of the representation. The updated features of node m are: Where H represents the number of attention heads. Let h be the attention weight of the h-th head. This is the corresponding weight matrix. The activation function is non-linear; multiple iterations are performed to obtain the final optimized node features. Take its first dimension as the optimized grid height value. This forms the optimized height matrix. ; 4.5 Based on the optimized height matrix Fit a least-squares reference plane over the entire measurement area: Let the plane equation be... By minimizing all grid center points Corresponding optimization height Solve for plane parameters by summing the squared distances to the plane; calculate the optimized height for each mesh cell. Vertical deviation to the reference plane In this context, A, B, and C are the three components of the plane's normal vector, determining the plane's direction; D is a constant term determining the signed distance from the plane to the origin; and x, y, and z represent the three-dimensional coordinates of a point in space. 4.6 Calculation of the arithmetic mean height S of surface roughness a : in, This represents the total number of valid grid cells. The vertical deviation of the optimized height of the m-th mesh cell from the reference plane. This means summing up the absolute values ​​of the deviations corresponding to each grid in the index. And record S a The mean and standard deviation, and S according to industry standards a Convert to the corresponding R a The numerical value outputs the overall roughness measurement results of the blade surface.

6. The method for measuring the roughness of aero-engine blades using a continuum robot based on multimodal fusion according to claim 5, characterized in that, Step 5 specifically involves: 5.1 Calculate the roughness values ​​of all effective mesh elements obtained in step 4. That is, the arithmetic mean height of each grid is statistically analyzed to calculate its mean. and standard deviation : Plot a normal distribution of roughness values, label the probability density curve, and calculate the roughness values ​​within three standard deviations. It is used to analyze the concentration and dispersion of roughness distribution and to evaluate the uniformity of blade surface processing; 5.2 Compare the mean roughness values ​​before and after removing the grid cells with curvature discontinuities and abnormal morphology in step 3, and calculate the relative deviation of the mean values ​​before and after removal. If the mean value tends to be stable and the standard deviation is significantly reduced after removal, it indicates that the interference grid removal operation in step 3 has eliminated the deviation amplification effect of high curvature regions and noise points on the measurement results, which is a necessary preprocessing step to ensure the accuracy of roughness measurement. 5.3 Calculate the mean roughness value and its standard deviation Tolerance range from the preset average value and standard deviation threshold Compare: If and If the surface roughness of the blade is satisfactory, a qualified mark will be output; if any item fails to meet the requirements, the product will be deemed unqualified, and the reason for non-compliance will be output, such as the mean value exceeding the tolerance or the standard deviation exceeding the tolerance, and rework or further testing is recommended.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a method for measuring the roughness of aero-engine blades of a continuum robot based on multimodal fusion, as described in any one of claims 1-6.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement a method for measuring the roughness of aero-engine blades of a continuum robot based on multimodal fusion as described in any one of claims 1-6.

Images