3D imaging for engine component inspection

Abstract

Systems and methods for engine imaging are provided. An imaging system includes an optical sensor coupled to an insertion tool and a light source configured to output a light projection onto a component of an engine assembly. The system includes a processor configured to: receive data including a plurality of frames from an optical sensor, the data captured as a component of an engine assembly rotates; determining an angular displacement of the component between the frames; and forming a 3D point cloud of the component based on combining data in the frames based on the angular displacement of the component between the frames.

Description

Technical Field

[0001] The present subject matter relates generally to imaging tools, and more particularly, to three-dimensional (3D) imaging tools for engine component inspection. Background Art

[0002] Complex devices, such as jet engines, are subject to wear during normal use, which can lead to performance degradation and failure. Wear can occur at various locations on the device, including internal areas that are difficult to access without disassembling the device. Insertion imaging systems, such as borescopes, can be inserted through inspection ports to perform internal inspections of jet engines. However, the form factor and field of view of insertion imaging systems can be limited due to the insertion path and space constraints within the engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] A full and enabling disclosure of the present disclosure, including the best mode thereof, to one of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0004] Figure 1 is a schematic cross-sectional view of a high bypass turbofan jet engine according to some embodiments of the present disclosure.

[0005] Figure 2 is a block diagram of an inspection system according to some embodiments of the present disclosure;

[0006] Figure 3 is a flow chart of a method according to some embodiments of the present disclosure;

[0007] Figure 4 is a flow chart of another method according to some embodiments of the present disclosure;

[0008] Figure 5A , 5B and 5C is a diagram of an inspection device according to some embodiments of the present disclosure; and

[0009] Figure 6 is an illustration of a process for combining points in a point cloud according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0010] Reference will now be made in detail to embodiments of the present disclosure, one or more examples of which are illustrated in the accompanying drawings. Each example is provided by way of explanation of the present disclosure rather than limitation thereof. In fact, it will be apparent to those skilled in the art that various modifications and variations may be made in the present disclosure without departing from the scope or spirit of the present disclosure. For example, a feature illustrated or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such modifications and variations that fall within the scope of the appended claims and their equivalents.

[0011] As used herein, the terms "first," "second," "third," etc. may be used interchangeably to distinguish one component from another, and are not intended to indicate the position or importance of each component.

[0012] Unless otherwise indicated herein, the terms "coupled," "fixed," "attached to," and the like refer to both direct coupling, fixing, or attachment as well as indirect coupling, fixing, or attachment via one or more intermediate components or features.

[0013] The singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise.

[0014] As used herein throughout the specification and claims, approximate language can be applied to modify any quantitative representation that allows variation without causing the basic function associated therewith to change. Therefore, the value modified by terms such as "about", "approximately", "almost" and "substantially" is not limited to the specified exact value. In some cases, approximate language can correspond to the accuracy of the instrument used to measure the value. For example, approximate language can refer to within a margin of 1%, 2%, 4%, 10%, 15% or 20%. These approximate margins can be applied to a single value, any endpoint or two endpoints of a limited numerical range, and / or the margin of the range between the endpoints. Here and throughout the specification and claims, range restrictions are combined and interchanged, unless otherwise specified by the context or language, such ranges are identified and include all sub-ranges contained therein. For example, all ranges disclosed herein include endpoints, and the endpoints can be combined independently of each other.

[0015] Typically, borescope inspections (BSI), or more generally, inspections using insertion tools, are performed regularly to assess the extent of damage to components inside fully assembled aircraft engines. This inspection can be performed under continuous lighting while the component is in motion. Once the inspector identifies damage on the component, the inspector can stop the movement of the component and move the component and the BSI device or other insertion tool to obtain the desired damage view. At this point, the inspector can also change the BSI from a 2D assessment to a 3D point cloud (PC) generation and perform 3D measurements around the point of interest. 3D measurements allow comparison of the measured size of the damage and the tolerance threshold associated with a specific component. 3D measurements can be performed via 3D stereo (dual camera system or a single camera with a prism view) and / or structured light pattern projection in combination with a visual system to establish a 3D point cloud. However, typically, conventional 3D scanning techniques require that the inspected component is static. Therefore, inspectors typically stop component movement (e.g., rotation of a compressor section) during 3D scanning.

[0016] In some embodiments, the systems and methods described herein provide imaging tools that are constructed to provide consistent images regardless of operator subjectivity or skill level. In some embodiments, the systems and methods described herein provide light projection combined with consistent imaging tools to generate 3D point clouds when parts are moving. The imaging devices described herein can reduce inspection time and increase the consistency of data capture by different operators and across inspection locations. In some embodiments, the system includes a multi-color (e.g., red, green, blue) light projection system (e.g., laser line, structured light) that is offset from the camera when the part is in the field of view of the camera. In some embodiments, the RGB channels in each frame are processed to obtain a laser projection pattern that is combined into a point cloud. In some embodiments, as the video of the moving part is captured with the laser line, the points from each color line are converted into one or more 3D point clouds. In some embodiments, a 3D point cloud is formed based on the position of the imaging system relative to the part from a computer-aided design (CAD) model. In some embodiments, a 3D point cloud of the part is reconstructed by appropriate correction of the magnification. In some embodiments, the motion of the part may be estimated through video analysis or through external sensors (optical, roller, ultrasonic, infrared, etc.) and used by a point cloud construction algorithm to combine the captured data points.

[0017] Reference is now made to the drawings, wherein like numerals refer to like elements throughout. Figure 1is a schematic cross-sectional view of a conventional gas turbine engine 10 for an aircraft in which the imaging and inspection systems described herein may operate. The engine 10 has a generally longitudinally extending axis or centerline 12 extending from a front portion 14 to an aft portion 16. The engine 10 includes, in downstream serial flow relationship: a fan section 18 including a fan 20; a compressor section 22 including a supercharger or low pressure (LP) compressor 24 and a high pressure (HP) compressor 26; a combustion section 28 including a combustor 30; a turbine section 32 including an HP turbine 34 and an LP turbine 36; and an exhaust section 38.

[0018] Fan section 18 includes a fan housing 40 surrounding fan 20. Fan 20 includes a plurality of fan blades 42 radially disposed about centerline 12.

[0019] HP compressor 26, combustor 30, and HP turbine 34 form a combustion gas generating core 44 of engine 10. Core 44 is surrounded by a core housing 46 which may be coupled to fan housing 40.

[0020] An HP shaft or spool 48 disposed coaxially about the centerline 12 of the engine 10 drivingly connects the HP turbine 34 to the HP compressor 26. An LP shaft or spool 50 disposed coaxially about the centerline 12 of the engine 10 within the larger diameter annular HP spool 48 drivingly connects the LP turbine 36 to the LP compressor 24 and the fan 20.

[0021] The LP compressor 24 and the HP compressor 26 each include a plurality of compressor stages 52, 54, wherein a set of compressor blades 56, 58 rotate relative to a set of corresponding static compressor vanes 60, 62 (also referred to as nozzles) to compress or pressurize a fluid flow passing through the stage. In a single compressor stage 52, 54, the plurality of compressor blades 56, 58 may be arranged in a ring and extend radially outward from a blade platform to a blade tip relative to the centerline 12, while the corresponding static compressor vanes 60, 62 are positioned downstream of and adjacent to the rotating blades 56, 58. It should be noted that Figure 1 The numbers of blades, buckets, and compressor stages shown in FIG. 5 are selected for illustration purposes only, and other numbers are possible.

[0022] The HP turbine 34 and the LP turbine 36 each include a plurality of turbine stages 64, 66 in which a set of turbine blades 68, 70 rotate relative to a set of corresponding static turbine vanes 72, 74 (also referred to as nozzles) to extract energy from the fluid flow passing through the stage. In a single turbine stage 64, 66, the plurality of turbine blades 68, 70 may be arranged in a ring and extend radially outward from the blade platform to the blade tip relative to the centerline 12, while the corresponding static turbine vanes 72, 74 are positioned upstream of and adjacent to the rotating blades 68, 70. It should be noted that Figure 1 The number of blades, buckets, and turbine stages shown in FIG. 5 are selected for illustration purposes only, and other numbers are possible.

[0023] In operation, the rotating fan 20 supplies ambient air to the LP compressor 24, which then supplies the pressurized ambient air to the HP compressor 26, which further pressurizes the ambient air. The pressurized air from the HP compressor 26 is mixed with fuel and ignited in the combustor 30, thereby generating combustion gases. Some work is extracted from these gases by the HP turbine 34 that drives the HP compressor 26. The combustion gases are exhausted into the LP turbine 36, which extracts additional work to drive the LP compressor 24, and the exhaust gas is ultimately exhausted from the engine 10 via the exhaust section 38. The drive of the LP turbine 36 drives the LP spool 50 to rotate the fan 20 and the LP compressor 24.

[0024] It will be appreciated that the engine 10 may further define a plurality of openings that allow for inspection of various components within the engine 10. For example, the engine 10 may define a plurality of insertion tool openings at various axial locations within the compressor section, the combustion section 28, and the turbine section 32. Additionally, as will be discussed below, the engine 10 may include, for example, one or more igniter ports within the combustion section 28 of the engine 10 that may allow for inspection of the combustion section 28.

[0025] It should be further understood that Figure 1 The exemplary engine 10 depicted in the drawings is merely an example, and in other exemplary embodiments, the engine 10 may have any other suitable configuration, including, for example, any other suitable number of shafts or spools, turbines, compressors, etc. Additionally or alternatively, in other exemplary embodiments, any other suitable turbine engine may be inspected using the systems and methods described herein. For example, in other exemplary embodiments, the turbine engine may not be a turbofan engine, but may be configured as a turboshaft engine, a turboprop engine, a turbojet engine, etc., or may be an industrial gas turbine engine for power generation, fluid pumping, etc.

[0026] Reference now Figure 2, an engine inspection system 200 is shown. The system 200 includes an inspection controller 210 and an imaging device 215. The imaging device 215 includes a light source 220, an optical sensor 230, and in some embodiments, a component sensor 240 coupled to one or more insertion tools 250. The component sensor 240 can be a sensor that detects the position, motion flow, and / or speed of a rotating component being inspected. Typically, the component sensor 240 can output a signal that can be used by the processor 212 to determine / estimate the angular displacement of the rotating component. In some embodiments, the component sensor 240 can generate a signal based on the movement of the component, the movement of another component, and / or a signal from another device (such as a rotating tool).

[0027] The inspection controller 210 includes a processor 212 coupled to a memory 213. In some embodiments, the inspection controller 210 may also include a data and / or network interface device for communicating with the light source 220, the optical sensor 230, the component sensor 240, and / or one or more databases for image processing and data storage as described herein. In some embodiments, the inspection controller 210 may include one or more of a control circuit, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), etc., and may be configured to execute computer readable instructions stored on a computer readable storage memory 213. The computer readable storage memory 213 may include volatile and / or non-volatile memory and have computer readable code stored thereon, which, when executed by the processor, causes the inspection controller 210 to receive and process data received from the optical sensor 230 and / or the component sensor 240. In some embodiments, the processor 212 may be further configured to control the operation of the light source 220, the optical sensor 230, and the component sensor 240. In some embodiments, the inspection controller 210 can be further configured to control the movement of one or more automatic or semi-automatic insertion tools 250 (such as a snake-arm robot or a robotic arm) of the imaging device 215. In some embodiments, the inspection controller 210 can be further configured to control the actuation of one or more mechanical components of the imaging device 215 (such as a joint and tip extender for spacing and / or angled the light source 220, optical sensor 230, and / or component sensor 240 relative to each other and / or relative to the body of the insertion tool 250 within the engine interior). In some embodiments, the inspection controller 210 is configured to perform the operations described herein. Figure 3 and Figure 4Inspection controller 210 may be located locally or remotely from the engine being inspected and communicate with light source 220, optical sensor 230, and / or component sensor 240 via a wired connection, a wireless LAN connection, and / or through a wide area network. In some embodiments, inspection controller 210 may communicate with imaging devices 215 in one or more inspection spaces for inspecting one or more aircraft components.

[0028] Insertion tool 250 generally refers to a tool configured to insert light source 220 and / or optical sensor 230 through a port on the housing of the engine to capture an image of the engine interior. In some embodiments, insertion tool 250 may be an insertion tool and / or a borescope insertion tool. In some embodiments, insertion tool 250 may include a flexible insertion tool, a rigid insertion tool, a mechanical arm or a snake-shaped arm robot. In some embodiments, insertion tool 250 may be a manually operated tool, a semi-automatic tool or an automatic tool. In some embodiments, insertion tool 250 may be configured to position optical sensor 230 and / or light source 220 in a predefined position and orientation in the engine when inserted. In some embodiments, insertion tool 250 may include one or more channels for guiding the wire of a transmission signal between one or more of inspection controller 210 and light source 220, optical sensor 230 and / or component sensor 240. In some embodiments, light source 220 is coupled to a second insertion tool separated from optical sensor 230, and the second insertion tool may be inserted through different ports of an engine assembly.

[0029] The light source 220 generally includes one or more light emitters for projecting one or more light patterns onto the inspected component. In some embodiments, the light source 220 may include one or more of a laser, a light emitting diode (LED), etc. In some embodiments, the light source 220 and the optical sensor 230 may be spaced apart and positioned along the length of the insertion tool 250. In some embodiments, the light source 220 is configured to project a line pattern onto the component. In some embodiments, the light source 220 is configured to project a structured light pattern, a grid, a horizontal bar, and / or a vertical bar. In some embodiments, the light source 220 may be configured to sequentially project a series of different patterns. For example, the light source 220 may project a first line at a first angle during a first rotation of the component, and project a second line at a second angle during a second rotation of the component. In some embodiments, the insertion tool 250 may include two or more distal tips configured to extend separately when inserted into an engine assembly, and the optical sensor 230 is positioned on the first distal tip, and the light source 220 is positioned on the second distal tip.

[0030] In some embodiments, the light source 220 includes a plurality of light emitters configured to project light of different wavelengths. In some embodiments, the light emitters are configured to simultaneously project different patterns of different wavelengths onto the component. For example, the light source 220 may include a blue laser, a green laser, and a red laser that simultaneously project patterns onto the component. In some embodiments, the light emitters project lines that are angled relative to each other. In some embodiments, the emitters of the light source 220 may be configured to project patterns that overlap at least partially with each other and are within the field of view of the optical sensor 230. In some embodiments, the light source 220 includes a plurality of light emitters positioned around the optical sensor 230 and coupled to the insertion tool 250. In some embodiments, the light source 220 includes a plurality of light emitters positioned along the length of the insertion tool 250. In some embodiments, the light source 220 includes a blue laser, a red laser, and a green laser. In some embodiments, the light source 220 may include an emitter that emits at a wavelength (such as infrared or ultraviolet) inside or outside the visible spectrum.

[0031] The optical sensor 230 generally includes one or more light sensors configured to capture an image of light projection on a component. In some embodiments, the optical sensor 230 may include one or more sensor elements for detecting light inside or outside the visible spectrum. In some embodiments, the optical sensor 230 may include a sensor device such as a MOS, CCD, CMOS sensor, etc. In some embodiments, the optical sensor 230 is configured to capture images in multiple color channels. In some embodiments, the optical sensor 230 includes a single sensor having an optical filter (e.g., a linear or Bayer filter, or any other color filter array) that separates incident light into a series of colors. In some embodiments, the optical sensor 230 includes a separate sensor for each color / wavelength channel, such as a three-chip color CCD camera. For example, the optical sensor 230 may include a monochromatic sensor for a first wavelength and a second monochromatic sensor for a second wavelength that differ by 50, 80, or 100 nm. In some embodiments, the color channels of the optical sensor 230 correspond to the wavelengths (e.g., red, green, blue) of the light emitters of the light source 220.

[0032] In some embodiments, the imaging device 215 also includes a component sensor 240 configured to detect the rotation speed of the inspected component. In some embodiments, the component sensor 240 is spaced apart from the optical sensor 230 and / or the light source 220. In some embodiments, the component sensor 240 is coupled to an insertion tool 250 that is the same or different from the light source 220 and / or the optical sensor 230. In some embodiments, the component sensor 240 can be positioned outside the housing of the inspected engine assembly. For example, the component sensor 240 can monitor the rotation of the fan blades 42, turbine blades 70, or rotator of the engine from the outside of the engine assembly. In some embodiments, monitoring can be based on the detection of optical marks on these components. In some embodiments, the component sensor 240 can be coupled to a rotating device that rotates the component of the inspected engine assembly, and the output of the rotating device is used to estimate the speed of the component rotation. In some embodiments, the component sensor 240 measures the rotation speed of the component based on the image of the captured component and / or the visual encoding on the component or shaft of the engine assembly. In some embodiments, component sensor 240 may include a rangefinder configured to determine the rotational velocity based on oscillations in the distance measurement corresponding to the airfoil moving past the measurement point of the rangefinder. In some embodiments, the velocity may be measured based on detecting the lowest point in the distance measurement, which corresponds to the tip of the airfoil passing in front of component sensor 240. The component rotational velocity measured by component sensor 240 may be used to determine the angular displacement between image frames captured by optical sensor 230 for combining the frames in a 3D point cloud. In some embodiments, component sensor 240 may be omitted, and the angular displacement may be determined by other means, such as based on the images captured by optical sensor 230.

[0033] Next reference Figure 3 , a method for performing an engine inspection is shown. In some embodiments, Figure 3 One or more steps can be referenced Figure 2 The engine inspection system 200 is described.

[0034] In step 310, an insertion tool (such as insertion tool 250) is inserted into the engine assembly. In some embodiments, the insertion tool can be manually operated, semi-automatic, or automatic. In some embodiments, the insertion tool can include a rigidizable tool that is inserted and rigidified to position the optical sensor 230 and / or light source 220 of the inspection tool in a predefined position and / or orientation. In some embodiments, the insertion tool can be mechanically driven into place by a signal from a controller.

[0035] In step 330, a component, such as component 550, is rotated. In some embodiments, the component may include a compressor airfoil, a turbine airfoil, or a shaft of an engine assembly. In some embodiments, the component may include a reference Figure 1 Airfoils and other structures of the LP compressor 24, HP compressor 26, HP turbine 34, LP turbine 36 and / or fan blades 42 described. In some embodiments, the component can be a cylindrical section of the engine, which is configured to rotate around the centerline 12 of the engine assembly on its axis. In some embodiments, the component can be rotated by a rotating device configured to drive rotation, such as an auxiliary power unit, an external power source, a device configured to apply mechanical torque to the fan blades 42 or the rotator of the engine assembly, or a device configured to rotate the HP shaft or spool 48 via a gearbox (e.g., an accessory gearbox). In some embodiments, the engine assembly can be inspected while it is installed on the aircraft. In some embodiments, the component can be manually rotated, such as by hand or via a crank. In some embodiments, due to the physical limitations of the rotating device and the engine assembly, the rotation speed of the component may vary when the image is captured. For example, the rotation speed of the component can increase and decrease within a single rotation.

[0036] In step 320, light source 220 outputs a light projection onto a component of an engine assembly within the field of view of an optical sensor, such as optical sensor 230. In some embodiments, the light projection is a laser projection. In some embodiments, the light projection is a line. In some embodiments, the light projection is a structured light pattern, a grid, horizontal bars, and / or vertical bars. In some embodiments, light source 220 includes multiple emitters that output multiple projections, each at a different peak wavelength. Figure 4 Details of embodiments with projection of multiple wavelengths are described in more detail.

[0037] In step 340, an inspection controller (such as inspection controller 210) receives data from the optical sensor. In some embodiments, the data from the optical sensor includes a plurality of image frames captured while components of the engine assembly are rotated and light projections are projected onto the engine assembly. In some embodiments, the data received in step 340 may be processed in real time or near real time to form a 3D point cloud in accordance with steps 350, 360, 370, and 380 while the insertion tool is still inserted into the engine and the optical sensor is still capturing data. In some embodiments, the data received in step 340 may be stored for later processing.

[0038] In step 350, the inspection controller determines the angular displacement of the component between the first frame and the second frame of the plurality of frames received in step 340. In some embodiments, the angular displacement can be measured based on a signal from the component sensor 240 described herein. For example, a rangefinder can be used to measure the rotational speed of the component. In some embodiments, the angular displacement of the component is determined based on the detection of the visual encoding on the component or shaft of the engine assembly in the image captured by the optical sensor 230 and / or the component sensor 240. For example, the angular displacement can be measured based on the pixel displacement of the visual encoding between frames. In some embodiments, the angular displacement of the component is determined based on illuminating the component with broadband light (e.g., white light) and recording the distance and / or orientation of the component in the frame. In some embodiments, the angular displacement of the component is determined by estimating the rotational speed of the component based on the plurality of frames captured by the optical sensor. In some embodiments, the light projection includes a line positioned to be tangent to the point on the leading edge of the compressor airfoil closest to the optical sensor or light source 220 when the compressor airfoil is in a rotational position closest to the optical sensor or light source 220, and the angular displacement of the component is determined based on detecting the point on the compressor airfoil via the light projection. In some embodiments, the angular displacement of the component between the first frame and the second frame is determined based on comparing the first frame and the second frame to a stored 3D model of the component. In some embodiments, the angular displacement between the two frames can be estimated based on determining the rotation of the component over a period of time (e.g., 2 rpm, 10 rpm, etc.) and dividing the total angular displacement measured by the number of frames acquired during the period of time (e.g., 60 fps, 120 fps, etc.).

[0039] In step 360, the inspection controller determines a set of points based on a first frame of the image captured by the optical sensor 230. In some embodiments, the first set of points includes 3D coordinates determined based on triangulating the distance between the optical sensor 230 and the point on the part based on (1) a known distance and angle between the light source 220 and the optical sensor 230, and (2) a displacement of projected pixels associated with the point in the first image of the light projection. In some embodiments, the first set of points may include points on a local coordinate system relative to a reference point or plane of the imaging tool. In some embodiments, the first set of points is determined based on a magnification correction of the first image of the light projection. In some embodiments, the distance (H) between the point captured by the optical sensor 230 and the reference plane is determined based on the displacement (vH) of pixels in the first image of the light projection in the first frame, where in, is the angular offset between the optical sensor 230 and the light source 220, m is the magnification factor, and And where d is the distance between the optical sensor 230 and the light source 220, and u is the distance between the optical sensor 230 and the reference plane.

[0040] In step 370, the inspection controller determines a second set of points based on the second frame of images captured by the optical sensor 230. Typically, the second set of points is determined based on the same or similar process as described in step 360.

[0041] In step 380, the inspection controller forms a 3D point cloud based on combining at least the plurality of groups of points determined in steps 360 and 370. In some embodiments, the 3D point cloud can be based on any number of frames. The 3D point cloud of the component can be formed based on combining the first group of points and the second group of points based on the angular displacement of the component between the first frame and the second frame determined in step 350. In some embodiments, forming the 3D point cloud of the component includes converting local coordinates of the first group of points and the second group of points into global coordinates of the 3D point cloud based on the angular displacement. Figure 6 An example of this conversion is shown in Figure 6 In the embodiment of the present invention, points from a first frame 610 and a second frame 620 are positioned relative to each other based on an angular displacement (Δ) between the two frames. In some embodiments, the displacement between the two frames can include a y-axis (tangential to the direction of rotation of the component) and a z-axis (perpendicular to the plane of the frame) shift. In some embodiments, the displacement between the set of points can be determined based on the measured angular displacement and the distance between a reference plane or point and the optical sensor 230 and / or the known distance between the optical sensor 230 and the axis of rotation of the component. The system can combine any number of frames based on the same or similar processing to form a 3D model of the inspected component. For example, with a full rotation of the compressor section, the system can form a 360 degree 3D model of the compressor section around the axis of rotation.

[0042] In some embodiments, the 3D point cloud is generated based on averaging and / or applying a smoothing function between adjacent points on the first set of points and the second set of points. In some embodiments, the 3D point cloud of the component can be further determined based on removing outliers from the first set of points or the second set of points, or from the formed 3D point cloud. In some embodiments, the system is configured to determine points in the first set of points and the second set of points associated with a stationary component of the engine (e.g., a casing wall, a blade). In some embodiments, the system can remove points associated with the stationary component in the 3D point cloud of the component. In some embodiments, the system can process points associated with the stationary component separately and combine the moving component and the stationary component in the 3D point cloud model. For example, points associated with the stationary component can be combined without performing a frame shift based on an angular displacement. In some embodiments, points associated with the stationary component of the engine are determined based on comparing images of light projections in multiple frames to identify static / unchanging points. In some embodiments, points associated with the stationary component of the engine are determined based on comparing the 3D model with a stored computer model of the rotating component.

[0043] In some embodiments, the system can be configured to determine that enough data has been captured to form a 3D point cloud. For example, the system can detect that the part has made a full rotation and / or that data has been captured around the entire perimeter of the part. The system can then signal the inspection tool and / or the user interface to terminate the inspection.

[0044] In some embodiments, after step 380, the system can be further configured to detect anomalies on the component based on the 3D point cloud in step 390. In some embodiments, the anomaly can be detected based on comparing the 3D point cloud to a stored 3D model of the component of the engine assembly. In some embodiments, the anomaly can be detected based on local surface outlier detection.

[0045] Next reference Figure 4 , shows a method for engine inspection using multiple color channels. In some embodiments, Figure 4 One or more steps can be referenced Figure 2 In some embodiments, Figure 4 The steps in Figure 3 The steps sharing the same reference numerals are the same or similar.

[0046] exist Figure 4 In the illustrated embodiment, in step 320B, the light source 220 projects two or more patterns, each having a different peak wavelength / color, onto the component. In some embodiments, the peak wavelengths of the first wavelength and the second wavelength differ by at least 50 nanometers. In some embodiments, the wavelengths differ by 80 nanometers, 100 nanometers, or more. In some embodiments, the light projection includes a first line of a first wavelength and a second line of a second wavelength, the second line being angled relative to the first line. In some embodiments, each pattern may include a line, a structured light pattern, a grid, a horizontal bar, and / or a vertical bar. In some embodiments, the patterns overlap at least partially in the field of view of the optical sensor 230. In some embodiments, the first pattern and the second pattern are projected simultaneously or sequentially. In some embodiments, the light source 220 projects three patterns of three wavelengths (e.g., red, green, blue). In some embodiments, the pattern may be a line, a structured light pattern, or the like.

[0047] In step 340B, an inspection controller (such as inspection controller 210) receives data from optical sensor 230. In some embodiments, the data from optical sensor 230 includes multiple color channels. In some embodiments, optical sensor 230 includes multiple monochromatic sensors, such as a monochromatic sensor for a first wavelength and a second monochromatic sensor for a second wavelength. In some embodiments, the optics include linear or Bayer filters or some other color filter arrays for separating the color channels.

[0048] In step 340A, the system filters the frame for the first wavelength. In some embodiments, step 340A can be based on isolating a color channel associated with the first wavelength from the data received from the optical sensor 230. In some embodiments, step 340A can include applying a software color filter on the captured color image. In step 360A, the system determines a first subset of points in the first frame associated with the first pattern having the first wavelength. The determination of the first subset of points can be the same or similar to the process described with reference to 360. In step 370A, the system determines a first subset of points in the second frame associated with the first pattern having the first wavelength. The determination of the second subset of points can be the same or similar to the process described with reference to step 370.

[0049] In step 340B, the inspection controller filters the frame for the second wavelength. In some embodiments, step 340B can be based on isolating the color channel associated with the second wavelength from the data received from the optical sensor 230. In some embodiments, step 340B can include applying a software color filter on the captured color image. In step 360B, the system determines a first subset of points in the first frame associated with the second pattern having the second wavelength. The determination of the second subset of points of the first frame can be the same or similar to the process described in reference 360. In step 370B, the system determines a second subset of points in the second frame associated with the second pattern having the second wavelength. The determination of the second subset of points of the second frame can be the same or similar to the process described in reference 370. Generally, steps 360A, 370A, 360B, and 370B can be performed in any order and / or simultaneously.

[0050] In step 380B, the inspection controller forms a 3D point cloud based on at least a subset of the points determined in steps 360A, 370A, 360B, and 370B. Figure 3 As discussed, subsets of points can be combined based on angular displacements between frames. In some embodiments, points from different color channels can be combined or stitched by taking an average or weighted average of adjacent points. In some embodiments, the overall resolution of the 3D point cloud can be increased based on combining data from different color channels of an optical sensor compared to triangulation based on a monochrome image. In some embodiments, the projections can have different angles of incidence relative to the surface of the part and / or relative to each other in the plane of the frame to enhance detection of different surface features / anomalous geometries. For example, a line projection at an angle to the linear damage can better detect linear damage than a line projection parallel to the damage line.

[0051] Although the reference Figure 4Filtering of two wavelengths is shown and described, but the systems and methods described herein can utilize any number of projected color channels, such as three (e.g., red, green, blue), four, five, or six discrete color channels. Figure 3 and Figure 4 The channels are filtered, processed, and combined using the processing described in .

[0052] Next reference Figure 5A -C, shows an example of an imaging device configuration according to some embodiments. Figure 5A , light source 220A, optical sensor 230A and component sensor 240 are connected to the distal tip of insertion tool 250. Insertion tool 250 is inserted through the port of the engine assembly to provide a line of sight to the inspected component 550. In some embodiments, component 550 can be an airfoil of a fan or compressor of a turbine engine 10. Light source 220A is positioned to project light pattern 510A onto the surface of component 550, and optical sensor 230A is positioned to capture the projected image when component 550 rotates around the axis of the engine assembly section. As indicated by arrow 551, the airfoil of component 550 is shown as moving downward. However, it should be understood that the image shows a planar representation of a group of objects (e.g., blades) mounted on a cylindrical support (e.g., a shaft or disk) rotating around the centerline of the engine.

[0053] exist Figure 5A In the illustrated embodiment, component sensor 240, light source 220A and optical sensor 230A are positioned along the length of insertion tool 250. In some embodiments, one or more of component sensor 240, insertion tool 250 and optical sensor 230A can be coupled to segments of insertion tool 250 that are pivotable and / or rigidifiable relative to each other. In some embodiments, insertion tool 250 can include a distal tip, wherein light source 220A and optical sensor 230A are mounted on different tips that are configured to pivot independently relative to previous segments. An actuator can actuate two distal tips apart when insertion tool 250 is inserted.

[0054] exist Figure 5B In the embodiment of the present invention, the light source 220 includes three emitters - a red emitter 220B-1, a blue emitter 220B-2, and a green emitter 220B-3 positioned around the optical sensor 230B at the tip of the insertion tool 250. In some embodiments, the emitters may include an actuator configured to retract the emitters when the insertion tool 250 is inserted into the engine assembly and to extend the emitters when the tool is in position for data capture. Figure 5BThe position shown. In some embodiments, the transmitter can be retracted again to withdraw the insertion tool 250 from the interior of the engine. In some embodiments, the transmitter actuator can be powered, line-controlled and / or spring-loaded. The red transmitter 220B-1, the blue transmitter 220B-2 and the green transmitter 220B-3 each project a pattern of discrete wavelengths, including a red line projection 510B-1, a blue line projection 510B-2 and a green line projection 510B-3. The optical sensor 230B may include a filter for separating out three color channels and / or may include multiple monochrome sensors.

[0055] exist Figure 5C In the embodiment of the present invention, the light source 220 includes three emitters, a red emitter 220C-1, a blue emitter 220C-2, and a green emitter 220C-3, positioned along the length of the insertion tool 250. The optical sensor 230C is positioned at the tip of the component sensor 240. The emitters 220C-1, 220C-2, and 220C-3 are connected to the first segment of the component sensor 240, and the optical sensor 230C is connected to the second segment angled relative to the first segment. In some embodiments, the two segments are configured to pivot relative to each other. In some embodiments, each of the emitters 220C-1, 220C-2, and 220C-3 can be located on different segments of the insertion tool 250 that can pivot relative to each other and / or be angled. The red emitter 220C-1, the blue emitter 220C-2, and the green emitter 220C-3 each project a pattern of discrete wavelengths, including a red line projection 510C-1, a blue line projection 510C-2, and a green line projection 510C-3. The optical sensor 230C may include a filter for separating three color channels and / or may include a plurality of monochrome sensors.

[0056] Figure 5A The imaging tool configuration shown in -C is provided by way of example only, and it should be understood that the systems and methods described herein utilize imaging tools of many different configurations having at least the optical sensor 230 and light source 220 described herein.

[0057] Further aspects of the disclosure are provided by the subject matter of the following clauses:

[0058] An engine inspection system includes: an optical sensor connected to an insertion tool, the insertion tool being configured to be inserted into an engine component for inspection; a light source being configured to output a light projection onto a component of the engine component within a field of view of the optical sensor; and a processor being configured to: receive data including a plurality of frames from the optical sensor, the data being captured when the component of the engine component is rotated and the light projection is projected onto the engine component; determine an angular displacement of the component between a first frame and a second frame of the plurality of frames; determine a first set of points based on a first image of the light projection in the first frame; determine a second set of points based on a second image of the light projection in the second frame; and form a 3D point cloud of the component based on combining the first set of points and the second set of points based on the angular displacement of the component between the first frame and the second frame.

[0059] A system as in any preceding clause wherein the light projection is a line.

[0060] The system of any preceding clause, wherein the light projection comprises a structured light pattern, a grid, horizontal bars and / or vertical bars.

[0061] A system as claimed in any preceding clause, wherein the angular displacement of the component is determined based on detection of a visual encoding on a shaft of the component or the engine assembly.

[0062] A system as in any preceding clause, wherein the angular displacement of the component is determined based on illuminating the component and the light source with broadband light and recording the distance and / or orientation of the component.

[0063] A system as in any preceding clause, further comprising a component sensor spaced apart from the optical sensor, wherein the angular displacement of the component is determined based on a signal from the component sensor.

[0064] A system as claimed in any preceding clause, wherein the component sensor is coupled to the insertion tool or a separate insertion tool.

[0065] A system as claimed in any preceding clause, wherein the component sensor is positioned external to a housing of the engine assembly.

[0066] A system as in any preceding clause, wherein the angular displacement of the component is determined by estimating a rotational velocity of the component based on the plurality of frames captured by the optical sensor.

[0067] A system as described in any preceding clause, wherein the light projection includes a line positioned to be tangent to a point on a leading edge of the compressor airfoil closest to the optical sensor or the light source when the compressor airfoil is in a rotational position closest to the optical sensor or the light source, and wherein the processor is configured to determine the angular displacement of the component based on detecting the point on the compressor airfoil via the light projection.

[0068] A system as in any preceding clause, wherein the angular displacement of the component between the first frame and the second frame is determined based on comparing the first frame and the second frame to a stored 3D model of the component.

[0069] A system as in any preceding clause, wherein the first set of points comprises 3D coordinates determined based on triangulation based on a known distance between the light source and the optical sensor and a displacement of pixels in the first image of the light projection.

[0070] A system as in any preceding clause, wherein the first set of points is determined based on magnification correction of the first image of the light projection.

[0071] A system as described in any preceding clause, wherein the distance (H) between a point captured by the optical sensor and a reference plane is determined based on a displacement (vH) of a pixel in the first image of the light projection in the first frame, wherein, in, is the angular offset between the optical sensor and the light source, m is the magnification factor, and And wherein d is the distance between the optical sensor and the light source, and u is the distance between the optical sensor and the reference plane.

[0072] A system as in any preceding clause, wherein the 3D point cloud of the component is further determined based on performing outlier removal.

[0073] The system of any preceding clause, wherein the processor is further configured to determine points in the first and second sets of points that are associated with stationary components of the engine and remove the points in the 3D point cloud of the components.

[0074] A system as in any preceding clause, wherein the point associated with the stationary component of the engine is determined based on comparing images of the light projection in the plurality of frames to identify static points.

[0075] A system as in any preceding clause, wherein forming the 3D point cloud of the part comprises transforming local coordinates of the first and second sets of points into global coordinates of the 3D point cloud based on the angular displacement.

[0076] A system as described in any preceding clause, wherein the light projection comprises a plurality of patterns projected sequentially.

[0077] A system as described in any preceding clause, wherein the insertion tool comprises a flexible insertion tool, a rigidifiable insertion tool, a robotic arm, or a snake-arm robot.

[0078] A system according to any preceding clause, wherein the insertion tool includes two distal tips, at least one of which is configured to extend apart when inserted into the engine assembly, and the optical sensor is positioned on a first distal tip and the light source is positioned on a second distal tip.

[0079] The system of any preceding clause, wherein the light source is coupled to an insertion tool separate from the insertion tool, and wherein the insertion tool and the insertion tool are inserted through different ports of the engine assembly.

[0080] A system as described in any preceding clause, wherein the light source comprises one or more laser emitters.

[0081] The system of any preceding clause, wherein the light source comprises a plurality of light emitters positioned about the optical sensor and coupled to the insertion tool.

[0082] The system of any preceding clause, wherein the light source comprises a plurality of light emitters positioned along the length of the insertion tool.

[0083] The system of any preceding clause, wherein the light source is coupled to a first segment of the insertion tool and the optical sensor is coupled to a second segment of the insertion tool, the second segment being angled relative to the first segment during capture of the plurality of frames.

[0084] A system as in any preceding clause, wherein the light source comprises two or more of blue light, red light, and green light.

[0085] A system as described in any preceding clause, wherein the light source includes a first light emitter that outputs a first pattern of the light projection at a first wavelength and a second light emitter that outputs a second pattern of the light projection at a second wavelength; and wherein the first set of points is determined based on: filtering the first frame for the first wavelength to determine a first subset of points associated with the first pattern; and filtering the first frame for the second wavelength to determine a second subset of points associated with the second pattern.

[0086] A system as in any preceding clause, wherein the optical sensor comprises a monochromatic sensor for the first wavelength and a second monochromatic sensor for the second wavelength.

[0087] A system as in any preceding clause, wherein the filtering of the first frame is based on a linear filter or a Bayer filter.

[0088] A system as in any preceding clause, wherein the peak wavelengths of the first wavelength and the second wavelength differ by at least 50 nanometers.

[0089] The system of any preceding clause, wherein the first pattern comprises first lines and the second pattern comprises second lines angled relative to the first lines.

[0090] The system of any preceding clause, wherein the first pattern or the second pattern comprises a structured light pattern, a grid, horizontal bars and / or vertical bars.

[0091] A system as in any preceding clause, wherein the first pattern and the second pattern at least partially overlap in the field of view of the optical sensor.

[0092] A system as in any preceding clause, wherein the first pattern and the second pattern are projected simultaneously.

[0093] A system as described in any preceding clause, wherein the first pattern and the second pattern are projected sequentially.

[0094] The system of any preceding clause, wherein the light source further comprises a third light emitter that outputs a third pattern at a third wavelength, and wherein determining the first set of points further comprises determining a third subset of points based on filtering the first frame for the third wavelength.

[0095] A system as recited in any preceding clause, wherein the component of the engine assembly comprises a compressor or turbine airfoil.

[0096] A system as in any preceding clause, wherein the 3D point cloud is generated based on averaging between neighboring points in the first set of points and the second set of points.

[0097] The system of any preceding clause, wherein the processor is further configured to detect anomalies on the component based on the 3D point cloud.

[0098] A system as claimed in any preceding clause, wherein the anomaly is detected based on comparing the 3D point cloud to a stored 3D model of the component of the engine assembly.

[0099] A system as in any preceding clause, wherein the anomaly is detected based on identifying outliers in a first set of points compared to a second set of points and by aligning two or more sets of points for combination into a point cloud.

[0100] An engine inspection method comprises: rotating a component of an engine assembly; projecting a light projection from a light source onto the component of the engine assembly within a field of view of an optical sensor inserted into the engine assembly via an insertion tool; receiving data including a plurality of frames from the optical sensor at a processor, the data being captured when the component of the engine assembly is rotated and the light projection is projected onto the component; determining, using the processor, an angular displacement of the component between a first frame and a second frame of the plurality of frames; determining, using the processor, a first set of points based on a first image of the light projection in the first frame; determining, using the processor, a second set of points based on a second image of the light projection in the second frame; and forming a 3D point cloud of the component by combining, using the processor, the first set of points and the second set of points based on the angular displacement of the component between the first frame and the second frame.

[0101] A method as in any preceding clause wherein the light projection is a line.

[0102] A method as in any preceding clause wherein the light projection comprises a structured light pattern, a grid, horizontal bars and / or vertical bars.

[0103] A method as in any preceding clause, wherein the angular displacement of the component is determined based on detection of a visual encoding on a shaft of the component or the engine assembly.

[0104] The method of any preceding clause, wherein the angular displacement of the component is determined based on illuminating the component and the light source with a broadband light source and recording the distance and / or orientation of the component relative to the optical sensor in a plurality of frames. The method of any preceding clause, wherein the angular displacement of the component is determined based on a signal from a component sensor spaced apart from the optical sensor.

[0105] A method as in any preceding clause, wherein the component sensor is coupled to the insertion tool or a separate insertion tool.

[0106] A method as in any preceding clause, wherein the component sensor is positioned external to a housing of the engine assembly.

[0107] A method as in any preceding clause, wherein the angular displacement of the component is determined by estimating a rotational velocity of the component based on the plurality of frames captured by the optical sensor.

[0108] A method according to any preceding clause, wherein the light projection includes a line positioned to be tangent to a point on the leading edge of the compressor airfoil closest to the optical sensor or the light source when the compressor airfoil is in a rotational position closest to the optical sensor or the light source, and determining the angular displacement of the component based on detecting the point on the compressor airfoil via the light projection.

[0109] A method as in any preceding clause, wherein the angular displacement of the component between the first frame and the second frame is determined based on comparing the first frame and the second frame to a stored 3D model of the component.

[0110] A method as in any preceding clause, wherein the first set of points comprises 3D coordinates determined based on triangulation based on a known distance between the light source and the optical sensor and a displacement of light points in the first image of the light projection.

[0111] A method as in any preceding clause, wherein the first set of points is determined based on magnification correction of the first image of the light projection.

[0112] A method as in any preceding clause, wherein the distance (H) between a point captured by the optical sensor and a reference plane is determined based on a displacement (vH) of a pixel in the first image of the light projection in the first frame, wherein, in, is the angular offset between the optical sensor and the light source, m is the magnification factor, and And wherein d is the distance between the optical sensor and the light source, and u is the distance between the optical sensor and the reference plane.

[0113] A method as in any preceding clause, wherein the 3D point cloud of the component is further determined based on performing outlier removal.

[0114] A method as in any preceding clause, further comprising: determining points in the first and second sets of points associated with a stationary component of the engine; and removing the points in the 3D point cloud of the component.

[0115] A method as in any preceding clause, wherein the point associated with the stationary component of the engine is determined based on comparing images of the light projection in the plurality of frames to identify static points.

[0116] A method as in any preceding clause, wherein forming the 3D point cloud of the part comprises transforming local coordinates of the first and second sets of points into global coordinates of the 3D point cloud based on the angular displacement.

[0117] A method as in any preceding clause, wherein the light projection comprises a plurality of patterns projected sequentially.

[0118] A method according to any preceding clause, wherein the insertion tool comprises a flexible insertion tool, a rigidifiable insertion tool, a robotic arm or a snake-arm robot.

[0119] A method according to any preceding clause, wherein the insertion tool includes two distal tips configured to extend apart when inserted into the engine assembly, and the optical sensor is positioned on a first distal tip and the light source is positioned on a second distal tip.

[0120] A method as in any preceding clause, wherein the light source is coupled to an insertion tool separate from the insertion tool, and wherein the insertion tool and the insertion tool are inserted through different ports of the engine assembly.

[0121] A method as in any preceding clause, wherein the light source comprises one or more laser emitters.

[0122] The method of any preceding clause, wherein the light source comprises a plurality of light emitters positioned about the optical sensor and coupled to the insertion tool.

[0123] A method as in any preceding clause, wherein the light source comprises a plurality of light emitters positioned along the length of the insertion tool.

[0124] The method of any preceding clause, wherein the light source is coupled to a first segment of the insertion tool and the optical sensor is coupled to a second segment of the insertion tool, the second segment being angled relative to the first segment during capture of the plurality of frames.

[0125] A method as in any preceding clause, wherein the light source comprises two or more of blue light, red light and green light.

[0126] A method according to any preceding clause, wherein the light source comprises a first light emitter that outputs a first pattern of the light projection at a first wavelength and a second light emitter that outputs a second pattern of the light projection at a second wavelength; and wherein the first set of points is determined based on: filtering the first frame for the first wavelength to determine a first subset of points associated with the first pattern; and filtering the first frame for the second wavelength to determine a second subset of points associated with the second pattern.

[0127] A method as in any preceding clause, wherein the optical sensor comprises a monochromatic sensor for the first wavelength and a second monochromatic sensor for the second wavelength.

[0128] A method as in any preceding clause, wherein the filtering of the first frame is based on a linear filter or a Bayer filter.

[0129] A method as in any preceding clause, wherein the peak wavelengths of the first wavelength and the second wavelength differ by at least 50 nanometers.

[0130] A method as in any preceding clause, wherein the first pattern comprises first lines and the second pattern comprises second lines angled relative to the first lines.

[0131] A method as in any preceding clause, wherein the first pattern or the second pattern comprises a structured light pattern, a grid, horizontal bars and / or vertical bars.

[0132] A method as in any preceding clause, wherein the first pattern and the second pattern at least partially overlap in the field of view of the optical sensor.

[0133] A method as in any preceding clause, wherein the first pattern and the second pattern are projected simultaneously.

[0134] A method as in any preceding clause, wherein the first pattern and the second pattern are projected sequentially.

[0135] A method as in any preceding clause, wherein the light source further comprises a third light emitter that outputs a third pattern at a third wavelength, and wherein determining the first set of points further comprises determining a third subset of points based on filtering the first frame for the third wavelength.

[0136] A method as described in any preceding clause, wherein the component of the engine assembly comprises a compressor or turbine airfoil.

[0137] A method as in any preceding clause, wherein the 3D point cloud is generated based on averaging between neighboring points in the first and second sets of points.

[0138] A method as in any preceding clause, further comprising detecting anomalies on the component based on the 3D point cloud.

[0139] A method as in any preceding clause, wherein the anomaly is detected based on comparing the 3D point cloud to a stored 3D model of the component of the engine assembly.

[0140] This written description uses examples to disclose the present disclosure, including the best mode, and also to enable any person skilled in the art to practice the present disclosure, including making and using any device or system and performing any combined methods. The patentable scope of the present disclosure is defined by the claims, and may include other examples that occur to one skilled in the art. These other examples are intended to fall within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements that do not differ substantially from the literal language of the claims.

Claims

1. An engine inspection system, characterized in that: include: an optical sensor coupled to an insertion tool configured to be inserted into an engine component for inspection; a light source configured to project light output onto a component of the engine assembly within a field of view of the optical sensor; as well as A processor, the processor being configured to: receiving data including a plurality of frames from the optical sensor, the data being captured while the component of the engine assembly is rotating and the light projection is projected onto the engine assembly; determining an angular displacement of the component between a first frame and a second frame in the plurality of frames; determining a first set of points based on a first image of the light projection in the first frame; determining a second set of points based on a second image of the light projection in the second frame; and A 3D point cloud of the component is formed based on combining the first set of points and the second set of points based on the angular displacement of the component between the first frame and the second frame.

2. The system according to claim 1, characterized in that in, The light projection comprises a line pattern, a structured light pattern, a grid, horizontal bars or vertical bars.

3. The system according to claim 1, characterized in that in, The angular displacement of the component is determined based on detection of a visual encoding on a shaft of the component or the engine assembly.

4. The system according to claim 1, characterized in that in, The angular displacement of the component is determined based on illuminating the component and the light source with a broadband light source and recording the distance and / or orientation of the component relative to the optical sensor in a plurality of frames.

5. The system according to claim 1, characterized in that Further included is a component sensor spaced apart from the optical sensor, wherein the angular displacement of the component is determined based on a signal from the component sensor.

6. The system according to claim 1, characterized in that in, The angular displacement of the component is determined by estimating a rotational velocity of the component based on the plurality of frames captured by the optical sensor.

7. The system according to claim 1, characterized in that in, The light projection includes a line projection positioned to be substantially tangent to a point on a leading edge of the compressor airfoil closest to the optical sensor or the light source when the compressor airfoil is in a rotational position closest to the optical sensor or the light source, and Wherein the processor is configured to determine the angular displacement of the component based on detecting the point on the compressor airfoil via the light projection.

8. The system according to claim 1, characterized in that in, The angular displacement of the component between the first frame and the second frame is determined based on comparing the first frame and the second frame to a stored 3D model of the component.

9. The system according to claim 1, characterized in that in, The first set of points includes 3D coordinates determined based on triangulation based on a known distance between the light source and the optical sensor and a displacement of light points in the first image of the light projection.

10. The system according to claim 1, characterized in that in, The processor is further configured to determine points in the first set of points and the second set of points that are associated with a stationary component of the engine and remove the points from the 3D point cloud of the component.

Images