Aero-engine blade binocular three-dimensional reconstruction system based on perspective difference

Abstract

This invention relates to the field of industrial precision inspection technology, specifically addressing the problems of reflection noise interference and physical scale distortion in binocular 3D reconstruction of aero-engine blades. Strong reflections and carbon buildup on the blade surface lead to voids in feature extraction during reconstruction, and depth prediction suffers from nonlinear scale bias. To address this, this invention proposes a binocular 3D reconstruction system for aero-engine blades based on viewpoint difference. This system employs a viewpoint difference algorithm to eliminate reflection noise; enhances feature extraction of damaged areas through a dual attention mechanism of spatial and matching approaches; identifies natural feature anchor points on the blade surface for depth refinement; and constructs a high-precision compensation matrix to achieve physical scale calibration. Experiments show that this system can effectively eliminate reflection interference and significantly improve the fidelity and physical scale consistency of complex surface reconstruction. This invention is applicable to high-precision 3D morphology inspection and evaluation of aero-engine blades in service status.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial precision measurement, specifically to a binocular three-dimensional reconstruction system for aero-engine blades based on perspective difference. Background Technology

[0002] Aero-engine blades are core components of aero-engines. During long-term service, their surfaces often accumulate carbon deposits, spalling pits, and complex oxide scale. Currently, 3D reconstruction of in-service blades mainly relies on manual visual inspection or contact measurement, which is inefficient and has limited accuracy. In recent years, non-contact reconstruction technologies, represented by binocular vision, have been gradually introduced into the field of industrial inspection. However, in practical applications, the reconstruction of aero-engine blades still faces many severe challenges: the surface of aero-engine blades has typical strong reflective metallic characteristics, which easily produce high-brightness glare under industrial light sources, causing a large number of false matching points to be generated during feature matching in the left and right views, thus resulting in errors in the generated depth map. Severe abnormal noise or local data gaps; the surface of blades after service is often covered with carbon deposits of varying thicknesses, which not only alter the original geometric properties of the blades but also conceal the underlying micro-cracks or peeling features; traditional local feature extraction algorithms are extremely poorly robust when processing such low-contrast, weak-texture areas, making it difficult to extract true morphological details; when conventional vision systems acquire 3D coordinates, their depth predictions often exhibit complex nonlinear mapping deviations from the actual physical spatial scale; limited by camera distortion, environmental disturbances, and algorithmic approximations, the reconstruction results often lack a stable physical benchmark, making it difficult to meet the high-precision measurement requirements of industrial applications;

[0003] The purpose of 3D reconstruction is to achieve quantitative and precise characterization of carbon distribution, spalling damage, and geometric deformation on the surface of serviced aero-engine blades by acquiring high-precision 3D point cloud data. Simultaneously, it provides a high-precision digital morphological benchmark for aero-engine health monitoring and life prediction, thereby replacing traditional, inefficient manual inspection methods and improving the intelligence level of aviation maintenance. This paper proposes a binocular 3D reconstruction system for aero-engine blades based on perspective difference to effectively solve the above problems. The system controls a displacement stage to drive a camera for micro-displacement bias sampling, and uses a perspective difference algorithm to remove the unique reflection noise of the metal surface. It also employs a dual architecture of spatial attention and matched attention, enabling high-precision extraction of blade morphological details even in carbon-covered, damaged areas. Furthermore, by identifying natural feature anchor points and combining them with the ground truth displacement of the moving platform to construct a compensation matrix, a high-precision conversion from visual prediction to spatial physical scale is achieved. This system provides a high-precision digital morphological benchmark for the quantitative characterization of carbon accumulation and damage on the surface of aero-engine blades. Summary of the Invention

[0004] The purpose of this invention is to address the problems of high surface reflection noise, carbon buildup obscuring damage details, and inaccurate visual depth prediction on in-service aero-engine blades, by proposing a binocular 3D reconstruction system based on viewpoint difference. This system aims to improve the integrity of the reconstructed point cloud and ensure that the reconstruction results possess high-precision physical spatial scale information. To achieve the above objectives, this invention adopts the following technical solution: A binocular 3D reconstruction system for aero-engine blades based on viewpoint difference, comprising:

[0005] S1, Image Acquisition Module: Through industrial software operator flow, two industrial cameras located on the gantry crane-type precision moving platform are synchronously triggered; multiple sequence images of the aero-engine blades are sampled at multiple preset static points.

[0006] S2, Displacement Control Module and Viewpoint Difference Preprocessing Module: Controls the displacement stage to drive the camera to perform small viewpoint offset sampling; uses the viewpoint difference algorithm to compare the light intensity changes at the same point under the micro-difference viewpoint, and identifies and removes the high-brightness reflection noise on the metal surface.

[0007] S3. The feature extraction module uses a dual feature extraction network of spatial attention and matching attention to extract the morphological features of the blades in service.

[0008] (1) This module uses Outlook Attention to capture minute damage details within a local window and combines the moving window self-attention mechanism to encode the global geometric structure at the full-image scale, through non-linear fusion of local details and global context;

[0009] Local self-attention LSA compensation:

[0010] Using Outlook Attention to capture minute damage details within a local window of a leaf image. The calculation formula is as follows:

[0011] (1);

[0012] in, Attention weights within a local window. For the extracted value characteristics;

[0013] Global Self-Attention Propagation: SWSA (Simultaneous Self-Attention Window) is used to encode the global geometry across the entire leaf scale, encoding global geometric features. The formula is expressed as:

[0014] (2);

[0015] LN represents layer normalization, and SWSA represents moving window self-attention operator. This mechanism ensures that even if the leaf root is completely covered by carbon deposits, the system can infer the reasonable depth of the region based on the overall bending trend of the blade.

[0016] (2) This module stitches the features of the left and right views along the epipolar direction and uses a bidirectional cross-attention mechanism to align global features;

[0017] First, the features from the left and right views are stitched together along the epipolar direction to obtain the fused features. :

[0018] (3);

[0019] In the formula and The left and right views represent the fused features below the 4th layer of the pyramid, respectively; then, cross-attention is used for feature alignment:

[0020] (4);

[0021] In the formula, CrossAttn represents a one-dimensional cross-attention operator used to search for matching points in the epipolar direction.

[0022] S4. Deep Refinement Based on Natural Feature Anchor Points: The system automatically identifies the edges of carbon deposit peeling pits on the blade surface as natural geometric anchor points, and corrects the initial cost volume through a volume attention mechanism to improve the accuracy of the depth map.

[0023] (1) This module first identifies carbon peeling pits or oxidation points on the surface of the blade by gradient threshold and generates a natural feature anchor mask; then it uses volume attention technology to directly inject the strong geometric information carried by the anchor into the 4D cost volume, and uses a pixel-level excitation guidance algorithm to find the precise depth value in the neighborhood of the anchor.

[0024] The formula for generating the natural feature anchor mask is:

[0025] (5);

[0026] Extracting semantic feature maps of blade surfaces using lightweight convolution operators By setting a threshold Identify gradient abrupt change regions and generate binary feature masks. In the formula, The volume represents the gradient value at pixel (i,j), and the cost volume after volumetric attention activation. The calculation is as follows:

[0027] (6);

[0028] (7);

[0029] In the formula, To guide the feature vector, MLP stands for Multilayer Perceptron, which is used to adjust the weight response in the disparity search space;

[0030] S5. Physical Truth Value Closed-Loop Calibration: Extract the true physical displacement of the moving platform and compare it with the predicted depth value generated by the vision system to construct a high-precision compensation matrix for nonlinear mapping. The core is to establish a closed-loop feedback mechanism between the visual coordinate system and the physical space coordinate system. By eliminating the nonlinear deviation of the vision algorithm in depth prediction, it ensures that the reconstruction result has a true physical scale. Synchronous extraction of physical truth value and visual prediction value: During system operation, the differential preprocessing module S2 controls the gantry crane-type precision moving platform to generate a known displacement. At this time, the closed-loop calibration module synchronously reads the true physical stroke value Y of the precision servo system. At the same time, the degradation feature enhancement modules S3 and S4 calculate the visual predicted depth change y of the aero-engine blade at the corresponding point based on the binocular vision algorithm. By performing the above synchronous sampling at multiple static points, the system obtains a series of interrelated physical-visual displacement pairs.

[0031] (1) This module synchronously reads the true value of the physical displacement of the mobile platform and the visual prediction depth value, and calculates the average absolute error by comparing the residuals; then it constructs a sub-pixel level compensation matrix to perform real-time mapping correction for the twisted edge and complex surface of the blade;

[0032] The expression for calculating the calibration matrix is:

[0033] Y (8);

[0034] In the formula, Y represents the true value of the physical travel of the precision moving platform, and y represents the depth change predicted by the vision algorithm. These are correction coefficients for local curvature and distortion; the final output is a corrected disparity map. as follows:

[0035] (9);

[0036] This is the original disparity map. This indicates element-wise multiplication, which is a pixel-by-pixel multiplication operation.

[0037] S6. 3D Point Cloud Reconstruction and Output: The disparity map is corrected in real time using a compensation matrix to generate a depth map with spatial physical scale information, and a high-precision 3D point cloud model of the aero-engine blade is reconstructed based on this map. As the final output stage of the system, this stage aims to transform the physically scale-calibrated disparity data into an intuitive, high-precision 3D geometric model. The spatial physical scale depth map generation system receives the disparity map corrected in step S5. Using the intrinsic parameter matrix focal length f and the extrinsic parameter baseline distance B of the binocular camera, pixel parallax is converted into the true depth distance Z; the calculation formula is as follows:

[0038] (9);

[0039] because The generated depth map, which includes subpixel-level compensation information, not only has high-precision morphological features but also spatial physical scale consistent with the travel of the precision displacement stage. This depth map can reflect the undulations of the blade surface in real time, including the trace accumulation of carbon deposits and the actual depth of the spalling pits.

[0040] The remapping of primitive coordinates to the 3D world coordinate system utilizes the principle of perspective projection to remap each effective pixel (u, v, Z) in the depth map to the 3D spatial coordinate system (X, Y, Z); the remapping formula is expressed as:

[0041] (10);

[0042] In the formula, The coordinates of the camera's optical center are used; through this nonlinear transformation, the two-dimensional pixels on the image plane are given a clear physical position, forming the original point cloud data of the engine blade surface; the system performs filtering and surface smoothing processing on the remapped spatial points, and finally constructs a three-dimensional point cloud model of the engine blade.

[0043] Invention Effects

[0044] Compared with the prior art, the present invention has the following significant advantages:

[0045] Strong anti-interference capability: The interference of light reflection noise on the metal surface of the aero-engine blade to the matching algorithm is solved by the perspective difference technology; High detail reproduction: The dual attention architecture can simultaneously take into account the details of minor damage and the macroscopic morphology of the blade. Even when the blade root is severely covered by carbon deposits, it can infer the reasonable depth based on the overall trend.

[0046] Precise scale alignment: The platform displacement true value is introduced for closed-loop calibration, which transforms the reconstruction result from a simple pixel coordinate system to a high-precision physical space coordinate system. Attached Figure Description

[0047] Figure 1 : Overall flowchart of the three-dimensional reconstruction system for aero-engine blades based on perspective difference;

[0048] Figure 2 : Computational architecture diagram of local detail extraction and global structure encoding in the spatial attention module;

[0049] Figure 3 A comparative diagram showing the difference between the viewpoint difference algorithm before and after removing metal reflection noise. Specific implementation methods Specific implementation method one:

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] like Figure 1 As shown, the binocular 3D reconstruction system for aero-engine blades based on viewpoint difference includes the following steps:

[0053] S1. Image acquisition: Two high-resolution area array cameras are mounted on a gantry crane-type precision mobile platform. Under the algorithm operator flow control written in Hikvision VM industrial vision software, images of the aero-engine blades are acquired synchronously at multiple preset points. Micro-displacement offset sampling is performed additionally for subsequent view difference preprocessing to obtain the original image sequence containing blade surface details and reflection noise.

[0054] S2. Viewpoint Differential Preprocessing: Multi-view images of the same blade position are obtained using micro-displacement offset sampling. By calculating the pixel differences between images from different viewpoints, images such as... are accurately identified and removed. Figure 3 The transient strong reflection noise shown on the left ultimately outputs a clean image sequence that preserves the true surface morphology, providing high-quality input for subsequent feature extraction.

[0055] S3, Dual Attention Feature Extraction: Receives the denoised image from S2 and processes it collaboratively through parallel spatial attention and matching attention modules. Spatial attention utilizes the Outlook Attention operator to extract local features of minor damage, while simultaneously encoding the global geometric structure through moving window self-attention. The fusion of these two processes enhances degraded features. Matching attention, through a bidirectional epipolar cross-attention mechanism, performs feature alignment between the left and right views, improving matching robustness in weakly textured regions and providing high-quality feature input for the deep refinement in S4.

[0056] S4, Natural Feature Anchor Point Depth Refinement, receives the enhanced feature map output by S3. It extracts inherent high-gradient features on the blade surface, such as groove edges and carbon deposit contours, as natural feature anchor points and generates a mask. It uses a volume attention mechanism to guide the cost value to be adaptively weighted and aggregated in the neighborhood of the anchor point, effectively suppressing the matching ambiguity in weak texture regions, thereby outputting a more accurate and robust visual prediction depth value y.

[0057] S5, the closed-loop calibration module, serves as the core error correction link. It compares and models the visual predicted depth value y output by the S4 module with the true physical displacement value Y fed back by the grating ruler of the high-precision displacement platform in real time. It calculates a sub-pixel-level high-precision compensation matrix that can correct the system's scale error and nonlinearity, and corrects all subsequent visual depth measurement results in real time, thereby ensuring that the final 3D point cloud has accurate physical scale and sub-pixel-level accuracy.

[0058] S6. High-precision physical-scale depth map generation: Receives the visual depth and sub-pixel compensation matrix calibrated by S5, and generates the final model through coordinate mapping and 3D reconstruction. Specifically, the compensation matrix is ​​applied to the original depth map to obtain a depth map with high physical accuracy; then, based on camera calibration parameters, the coordinates of each pixel in the image are mapped to the 3D world coordinate system using the principle of triangulation, and the corresponding 3D point coordinates are calculated; finally, all mapped point cloud data are integrated to output a high-precision 3D point cloud model that accurately characterizes the geometric morphology and damage details of the blade surface.

[0059] The embodiments of the present invention will be described in detail below:

[0060] The specific implementation of this invention is as follows:

[0061] S1. The steps for image acquisition via the gantry are as follows:

[0062] (1) Hardware system setup: A gantry crane-type high-precision three-axis electric displacement platform is used as the motion support system. The platform integrates Heidenhain grating rulers to achieve closed-loop control, with a positioning accuracy of ±1µm. The imaging system consists of two Basler ace 2 series area array industrial cameras and Computar dual telecentric lenses. The camera resolution is set to 2448×2048, and they are installed horizontally in a standard binocular architecture. The baseline distance B is calibrated to 200.0mm.

[0063] (2) Control and Acquisition Process: An industrial software operator flow was written and executed on the Hikvision VM algorithm platform in the industrial control computer. The operator flow controls the displacement platform to move along a preset path, and simultaneously triggers two cameras at each sampling point to acquire the original binocular image pair {Left,Right}. To support S2 noise suppression, at each main sampling point, the operator flow additionally controls the platform to perform a micro-displacement of ±0.3mm in the direction perpendicular to the baseline, and simultaneously acquires the offset view image, thereby generating a data packet containing three sets of images for each physical point: the main view, the positive offset view, and the negative offset view.

[0064] S2. Viewpoint difference preprocessing to suppress strong specular reflection noise on the metal surface and obtain a clean image sequence. The steps are as follows:

[0065] (1) Image acquisition: Read the three left-side views (main view, positive offset, negative offset) of the same physical point in S1, and the corresponding three right-side views;

[0066] (2) Reflection noise removal: Calculate the pixel difference between the offset images for the left and right views respectively. Since the real surface texture is stable while the specular highlight changes drastically with the viewing angle, set the threshold T=30, and determine the pixel region where ΔI > T as instantaneous reflection noise, and generate a binary noise mask;

[0067] (3) Image restoration and output: Using the normal pixels around the noise area in the main view image, a bilinear interpolation algorithm is used for restoration to generate a clean left view with specular interference removed. The right view is processed in the same way. Finally, a clean stereo image sequence is output as the input to S3.

[0068] S3. The specific implementation steps for dual-attention morphological feature extraction are as follows:

[0069] (1) Simultaneously feed the input feature map into the local path and the global path.

[0070] Local Path: After the feature map is segmented, it is input into the Outlook Attention module. This module calculates the attention weights within the local window, as shown in Equation (1), efficiently aggregating the contextual information of minor damages and outputting local features with enhanced details;

[0071] Global path: After layer normalization, the feature map is input into the moving window self-attention module. This module establishes long-range dependencies through the window moving mechanism, as shown in formula (2), encoding the overall macroscopic geometric features of the blade, such as bending and twisting;

[0072] (2) Input the enhanced feature maps of the left and right views into the bidirectional epipolar cross attention module. This module performs attention calculation along the epipolar direction, using the features of the left image as the query and the features of the right image as the key and value, to realize the feature information flow from the right image to the left image; at the same time, it performs reverse calculation; this process enables the network to achieve robust feature alignment in weak texture regions and outputs the final feature pair optimized for the matching task.

[0073] S4. The steps for refining the depth of natural feature anchor points are as follows:

[0074] (1) Anchor point mask generation: Calculate the spatial gradient magnitude on the enhanced feature map of the left view output by S3. Set the gradient threshold τ = 0.15 × max(Gradient), and determine the pixels with gradient values ​​higher than τ as natural feature anchor points to generate a binary anchor point mask. .

[0075] (2) Construct the initial 4D cost volume based on the features of the left and right views.

[0076] (3) Anchor point mask The encoding is an attention-guided weight, which reweights the cost volume through a volumetric attention module consisting of a lightweight 3D convolutional layer and two MLP layers. This mechanism enables the network to significantly focus on the anchor point and its neighborhood when calculating the matching cost, thereby obtaining more accurate and stable cost values ​​at these feature-defined locations and effectively suppressing fuzzy matching in smooth regions.

[0077] (4) After aggregation optimization, the depth value y predicted by vision is output.

[0078] The steps for S5 closed-loop calibration module are as follows:

[0079] (1) Error comparison data acquisition: During the system calibration stage, the control displacement platform drives the blade to perform a series of known physical displacements along the depth direction. At each position i, record the true value of the physical displacement fed back by the platform's grating ruler. The S1-S4 algorithm is then run to obtain the corresponding visual prediction depth change. .

[0080] (2) Compensation matrix calculation: Collect multiple sets of { , For the data pairs, a least-squares method was used to fit a visual system scale and nonlinear error model. Based on this model, a high-precision compensation matrix related to the image coordinates (u,v) was calculated and generated. This matrix incorporates all systematic errors, including lens distortion and mounting deviations.

[0081] (3) Real-time correction: In subsequent measurements, this compensation matrix will be used. The original visual depth map output by S4 is applied in real time to complete scale alignment and error correction.

[0082] S6. The steps for generating a high-precision physical-scale depth map and reconstructing a 3D model are as follows:

[0083] (1) Depth Map Correction and Coordinate Mapping: The high-precision depth map calibrated by S5, combined with the intrinsic parameter matrix K, distortion coefficients, and extrinsic parameter [R|T] calibrated by the binocular camera, is back-projected onto the 3D world coordinate system based on the principle of triangulation. Its physical coordinates (X, Y, Z) are then calculated. The formula is as follows:

[0084] (11);

[0085] (2) Point cloud generation and output: Integrate the three-dimensional point data of all sampling points, and use statistical outlier removal filtering and moving least squares surface smoothing for post-processing. Finally, output a high-precision three-dimensional point cloud model of the aero-engine blade with accurate physical scale and can be directly used for quantitative damage analysis.

Claims

1. A binocular 3D reconstruction system for aero-engine blades based on perspective difference, characterized in that, include: S1. The image acquisition module synchronously triggers two industrial cameras on the precision mobile platform through industrial software operator streams to sample image sequences at static points. S2, the displacement control module controls the displacement stage to drive the camera to perform micro-displacement offset sampling, and the view difference preprocessing module uses the view difference algorithm to remove reflection noise; S3. The feature extraction module uses a dual feature extraction network of spatial attention and matching attention to extract the morphological features of the blades in service. S4. The natural feature anchor point recognition module identifies the edges of carbon deposit peeling pits on the blade surface as natural feature anchor points, and the volume enhancement module uses volume attention for in-depth refinement. S5. The closed-loop calibration module compares the true value of the physical displacement of the mobile platform with the visually predicted depth value to construct a sub-pixel level compensation matrix. S6. The depth reconstruction module corrects the disparity map in real time and outputs a three-dimensional morphological depth map of the blade with spatial physical scale information, and reconstructs a three-dimensional point cloud model of the aero-engine blade based on the depth map.

2. The binocular three-dimensional reconstruction system for aero-engine blades based on viewpoint difference as described in claim 1, characterized in that, The calculation method for the dual feature extraction network described in step S3 is as follows: S31. This module uses Outlook Attention to capture minute damage details within a local window and combines a moving window self-attention mechanism to encode the global geometric structure at the full-image scale, through non-linear fusion of local details and global context. Local self-attention LSA compensation: Using Outlook Attention to capture minute damage details within a local window of a leaf image. The calculation formula is as follows: (1)； in, Attention weights within a local window. For the extracted value characteristics; Global Self-Attention Propagation: SWSA (Simultaneous Self-Attention Window) is used to encode the global geometry across the entire leaf scale, encoding global geometric features. The formula is expressed as: (2)； LN represents layer normalization, and SWSA represents moving window self-attention operator. This mechanism ensures that even if the leaf root is completely covered by carbon deposits, the system can infer the reasonable depth of the region based on the overall bending trend of the blade. S32. This module stitches the features of the left and right views along the epipolar direction and uses a bidirectional cross-attention mechanism to align global features. First, the features from the left and right views are stitched together along the epipolar direction to obtain the fused features. : (3)； In the formula and The left and right views represent the fused features below the 4th layer of the pyramid, respectively; then, cross-attention is used for feature alignment: (4)； In the formula, CrossAttn represents a one-dimensional cross-attention operator used to search for matching points in the epipolar direction.

3. The binocular three-dimensional reconstruction system for aero-engine blades based on viewpoint difference as described in claim 1, characterized in that, The natural feature anchor point mask described in step S4 The identification and volume excitation layer calculation methods are as follows: S41. This module first identifies carbon deposits or oxidation points on the blade surface through gradient thresholding and generates a natural feature anchor mask. Then, it uses volume attention technology to directly inject the strong geometric information carried by the anchor into the 4D cost volume and uses a pixel-level excitation guidance algorithm to find the precise depth value in the neighborhood of the anchor. The formula for generating the natural feature anchor mask is: (5)； Extracting semantic feature maps of blade surfaces using lightweight convolution operators By setting a threshold Identify gradient abrupt change regions and generate binary feature masks. In the formula, The volume represents the gradient value at pixel (i,j), and the cost volume after volumetric attention activation. The calculation is as follows: (6)； (7)； In the formula, To guide the feature vector, the MLP is a multilayer perceptron used to adjust the weight response of the disparity search space.

4. The binocular three-dimensional reconstruction system for aero-engine blades based on viewpoint difference as described in claim 1, characterized in that, The sub-pixel level compensation matrix and correction described in step S5 The formula is as follows: S51. This module synchronously reads the true value of the physical displacement of the mobile platform and the visual predicted depth value, calculates the average absolute error by comparing the residuals, and then constructs a sub-pixel level compensation matrix to perform real-time mapping correction for blade twist edges and complex surfaces. The expression for calculating the calibration matrix is: Y (8)； In the formula, Y represents the true value of the physical travel of the precision moving platform, and y represents the depth change predicted by the vision algorithm. These are correction coefficients for local curvature and distortion; the final output is a corrected disparity map. as follows: (9)； This is the original disparity map. This indicates element-wise multiplication, which is a pixel-by-pixel multiplication operation.

Images