System and method for zone-oriented repair range optimization

Abstract

The invention relates to a system and method for region-oriented repair range optimization. An inspection system (100) is disclosed. The inspection system (100) includes an image capture system (150) oriented to capture at least one component image (170) of a component (250) from at least one perspective, the at least one component image (170) including a set of features (171) selected such that features of the component (250) and features of defects (274) on a surface (260) of the component (250) can be identified (312). The processor (120) is configured to classify a type of defect (274) detected on a surface (260) of the component (250) in the at least one component image (170), and to assign (308) an area (280) to the surface (260) of the component (250) captured in the at least one component image (170).

Description

Background Art

[0001] The subject matter described herein relates to systems and methods for inspecting components, and more particularly to zone guided systems and methods for inspecting components.

[0002] Manufacturers and end users usually perform regular inspections on components to ensure that the components are free of mechanical and material defects and to determine that the components are in normal working condition. Known defects may typically include, for example, surface stress fractures, fatigue cracking and inelastic deformation, and surface finish defects such as pitting due to erosion or corrosion, and / or spalling.

[0003] Known inspections may include visual inspections for detecting surface defects, or measurements for determining tolerance variations. As a result of such inspections, the operator may determine that the component is free of defects. Alternatively, it may be determined that the component requires repair or refurbishment, that the component should be scrapped, or that a more detailed inspection and testing via specialized equipment and processes is required. In making such determinations, the operator may consider factors such as the type and size of the defect, the location of the defect on the component, the acceptable tolerance limits of the component, and / or other known factors that may affect the course of action recommended for the component, such as the amount of time required to perform repair or refurbishment, reliability analysis, and calculate the remaining useful life of the component. In addition, depending on the component and / or the amount of damage or defect, the operator may weigh the costs associated with further inspection and processing relative to replacing the component entirely.

[0004] Although operators are trained in inspection techniques and cost assessment, the process can be subjective and may vary from operator to operator and / or from one repair facility to another. Furthermore, the determination process itself is inefficient and can be inaccurate due to user error. Even if guidelines and training procedures are properly employed, operators typically do not have the necessary authority to approve scrapping and replacement parts, and the determination is typically handled by management, which can add inconsistency to the process and / or increase the time of the process.

[0005] In order to facilitate improved process consistency and reduce the time associated with the process, at least some known facilities use automated inspection systems. At least some known automated inspection systems implement component imaging and defect identification via machine learning algorithms, in which digital two-dimensional images of the surface of the component are analyzed. During surface analysis, surface defects are detected using traditional machine learning algorithms. Where high-resolution imaging is used, relatively small defects such as microcracks can be detected and classified by machine learning algorithms, however, high-resolution imaging of microcracks can be processor demanding and may require increased processing time.

[0006]

[0006] Therefore, there is a need to provide systems and methods for inspecting components in a manner that facilitates improved processing, accuracy, and consistency in the classification and repair of defects. Summary of the invention

[0007] In one aspect, an inspection system is disclosed. The inspection system includes an image capture system oriented to capture at least one component image of a component from at least one viewing angle, the at least one component image including a set of features selected to enable features of the component and features of defects on a surface of the component to be identified; and a processor configured to: classify a type of defect detected on the surface of the component in the at least one component image; and assign a region to the surface of the component captured in the at least one component image.

[0008] In another aspect, a method for region-guided repair extent optimization for a component is disclosed. The method includes providing an image captured by an image capture system, the image capture system being oriented to capture at least one component image of a component from at least one viewing angle, the at least one component image including a set of features selected to enable features of the component and features of a defect on a surface of the component to be identified; and classifying a type of defect detected on a surface of the component in the at least one component image; and assigning a region to a surface of the component captured in the at least one component image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The presently disclosed subject matter will be explained in more detail hereinafter with reference to exemplary embodiments shown in the drawings.

[0010] Figure 1 is a schematic diagram of an exemplary computer processing system;

[0011] Figure 2 is a schematic diagram of an exemplary inspection system including an imaging device for capturing an image of a component;

[0012] Figure 3A Shown by Figure 2 An exemplary component image captured by an imaging device of

[0013] Figure 3B showing a reference image stored in a memory;

[0014] Figure 4 Shows the storage Figure 1 exemplary data in a memory of a computer processing system;

[0015] Figure 5 Shown by Figure 1 an exemplary operational diagram of a classification module and a partitioning module operated by a processor of a computer processing system of; and,

[0016] FIG. 6A to FIG. 6C An exemplary flow chart of an exemplary method 300 for inspecting a component is shown.

[0017] The reference numerals used in the drawings and their meanings are listed in summary form in the reference numeral list. In principle, identical components are provided with the same reference numerals in the drawings. DETAILED DESCRIPTION

[0018] In the following specification and claims, reference will be made to a number of terms which shall be defined to have the following meanings.

[0019] As used herein, references to "an example or exemplary embodiment" or "one embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0020] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. The term "optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where the event occurs and instances where it does not occur.

[0021] Unless otherwise indicated, approximate language as used herein, such as "substantially", "substantially" and "about" indicate that as one of ordinary skill in the art will recognize, the terms so modified may apply only to an approximate degree, rather than an absolute or perfect degree. Therefore, values ​​modified by one or more terms (such as "about", "approximately" and "substantially") are not limited to the precise values ​​specified. In at least some cases, approximate language may correspond to the precision of the instrument used to measure the value. Here and throughout the specification and claims, range limitations may be identified. Unless otherwise indicated by context or language, these ranges may be combined and / or interchanged, and include all subranges contained therein.

[0022] In addition, unless otherwise indicated, the terms "first", "second", etc. are used herein merely as labels and are not intended to impose order, position, or hierarchical requirements on the items to which these terms refer. Furthermore, for example, reference to a "second" item does not require or exclude the existence of, for example, a "first" or lower numbered item or a "third" or higher numbered item.

[0023] In one embodiment, a computer program is provided, the computer program being embodied on a computer readable medium. In an exemplary embodiment, the system is executed on a single computer system without requiring connection to a server computer. In another embodiment, the system is executed on a single computer system. In another embodiment, the system is executed in a mainframe environment and UNIX is a registered trademark of X / Open Company Limited, located in Reading, Berkshire, United Kingdom. In another embodiment, the system is (iOS is a registered trademark of Cisco Systems, Inc., located in San Jose, CA). In another embodiment, the system is implemented on a Mac (Mac OS is a registered trademark of Apple Inc. located in Cupertino, CA). In another embodiment, the system In another embodiment, the system runs on an Android OS (Android is a registered trademark of Google, Inc., located in Mountain View, CA). The system is implemented as a computer program product (Linux is a registered trademark of Linus Torvalds of Boston, MA). The application is flexible and designed to run in a variety of different environments without affecting any primary functionality. In some embodiments, the system includes multiple components distributed among multiple computing devices. One or more components can take the form of computer-executable instructions contained in a computer-readable medium.

[0024] As used herein, the terms "software" and "firmware" are interchangeable and include any computer program stored in a memory for execution by a processor, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are exemplary only and are not limited to the types of memory that can be used to store computer programs.

[0025] As used herein, the terms "processor" and "computer" and related terms such as "processing device", "computing device" and "controller" are not limited to those integrated circuits known as computers in the art, but in a broad sense refer to microcontrollers, microcomputers, analog computers, programmable logic controllers (PLCs) and application-specific integrated circuits (ASICs), as well as other programmable circuits, and these terms are used interchangeably herein. In the embodiments described herein, "memory" may include, but is not limited to, computer-readable media, such as random access memory (RAM), computer-readable non-volatile media, such as flash memory. Alternatively, floppy disks, compact disk read-only memory (CD-ROM), magneto-optical disks (MODs) and / or digital versatile disks (DVDs) may also be used. In addition, in the embodiments described herein, additional input channels may be, but are not limited to, computer peripherals associated with an operator interface, such as a touch screen, a mouse, and a keyboard. Alternatively, other computer peripherals may also be used, which may include, for example, but are not limited to, a scanner. In addition, in exemplary embodiments, additional output channels may include, but are not limited to, an operator interface monitor or a head-up display. Some embodiments involve the use of one or more electronic devices or computing devices. Such devices typically include a processor, a processing device, or a controller, such as a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, a reduced instruction set computer (RISC) processor, an ASIC, a PLC, a field programmable gate array (FPGA), a digital signal processing (DSP) device, and / or any other circuit or processing device capable of performing the functions described herein. The methods described herein may be encoded as executable instructions embodied in a computer-readable medium, including but not limited to storage devices and / or memory devices. These instructions, when executed by a processing device, cause the processing device to perform at least a portion of the methods described herein. The above embodiments are exemplary only and are therefore not intended to limit the definition and / or meaning of the terms "processor" and "processing device" in any way.

[0026] As used herein, the term "non-transitory computer-readable medium" is intended to mean any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as computer-readable instructions, data structures, program modules and submodules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible non-transitory computer-readable medium, including but not limited to storage devices and / or memory devices. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. In addition, as used herein, the term "non-transitory computer-readable medium" includes all tangible computer-readable media, including but not limited to non-transitory computer storage devices, including but not limited to volatile and non-volatile media, and removable and non-removable media such as firmware, physical and virtual storage devices, CD-ROMs, DVDs, and any other digital sources such as networks or the Internet, as well as digital means to be developed, with the only exception of transient propagation signals.

[0027] The system and process are not limited to the specific embodiments described herein. In addition, the components of each system and each process can be practiced independently and separated from other components and processes described herein. Each component and process can also be used in combination with other component packages and processes.

[0028] Embodiments described herein provide an inspection system and method for detailed inspection of a component. The inspection system includes an image capture system and a processor, the image capture system being oriented to capture at least one component image of a component from at least one viewing angle, the at least one component image including a set of features selected to enable features of the component and features of defects on a surface of the component to be identified. The processor is configured to classify the type of defect detected on the surface of the component in the at least one component image, and to assign a region to the surface of the component captured in the at least one component image.

[0029] As used herein, the term "defect" refers to an area or region of the surface of a component that has been damaged due to normal operation and use of the component. Defects may include, but are not limited to, only dislocations, cracks, wear, voids, inelastic deformation, stress concentrations, ruptures, cracks, inclusions of foreign particles, corrosion, surface stress fractures, fatigue cracking, spalling, and surface finish defects such as pitting due to erosion or corrosion. Defects can be perceived on the surface of the component via visual inspection.

[0030] Figure 1An exemplary configuration of a computing device 110 of an exemplary inspection system 100 is shown. In an exemplary embodiment, the computing device 110 can be operated by a user 102 and communicatively connected to a remote device, such as an imaging device 150 and / or a positioning device 160. The computing device includes a processor 120 for executing instructions. In some embodiments, executable instructions are stored in a memory 122. The processor 120 may include one or more processing units (e.g., in a multi-core configuration). The memory 122 may be any device that enables information such as executable instructions and / or transaction data to be stored and retrieved. The memory 122 may include one or more computer-readable media. The memory 122 may be located locally on the computing device 110, or may be stored on a remote network or cloud that can be accessed by the computing device 110.

[0031] Computing device 110 may also include a communication interface 130 coupled to imaging device 150 and positioning device 160. Communication interface 130 may include, for example, a wired or wireless network adapter and / or a wireless data transceiver for use with a mobile telecommunications network.

[0032] In some embodiments, the computing device 110 also includes at least one media output component 124 that presents information to the user 102. The media output component 124 can be any component that can convey information to the user 102. In some embodiments, the media output component 124 includes an output adapter (not shown), such as a video adapter and / or an audio adapter. The output adapter can be operatively connected to the processor 120 and can be operatively connected to an output device, such as a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED) display, or an "electronic ink" display) or an audio output device (e.g., a speaker or headphones). In some embodiments, the media output component 124 presents a graphical user interface (e.g., a web browser and / or a client application) to the user 102. The graphical user interface may include, for example, an interface for viewing the analysis results of one or more subject systems. In some embodiments, the computing device 110 includes an input device 126 for receiving input from the user 102. Input device 126 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch-sensitive control pad (e.g., a trackpad or touch screen), a gyroscope, an accelerometer, a position detector, a biometric input device, and / or an audio input device. A single component, such as a touch screen, may be used as an output device for media output component 124 and as input device 126.

[0033] Stored in the memory 122 are computer readable instructions for providing a user interface to the user 102 via the media output component 124 and optionally receiving and processing input from the input device 126. The user interface may include a web browser and / or a client application, etc. The processor 120 executes computer executable instructions for implementing various aspects of the present disclosure. In some embodiments, by executing computer executable instructions or by being programmed in other ways, the processor 120 is converted into a special-purpose microprocessor.

[0034] like Figure 2 As shown, a component 154 to be inspected is positioned substantially flat on a platform 152 so that a surface 155 of the component 154 can be captured by an imaging device 150. The platform 152, the component 154, and the imaging device 150 are oriented in a space defined by the XYZ axes. The platform 152 is oriented horizontally relative to a plane defined by the XZ axes, and the imaging device 150 is positioned above the platform 152 and can move through the XYZ space.

[0035] In some embodiments, the imaging device 150 is a fixed multi-camera system. In some embodiments, the imaging device 150 can be articulated by the positioning device 160. In some embodiments, the imaging device 150 can be articulated so that the field of view 151 of the imaging device captures at least a portion of the surface 155 of the component 154 from different angles. In some embodiments, multiple imaging devices 150 can be positioned at different angles relative to the platform 152 so that the multiple imaging devices 150 can capture at least one component image 170 of a side surface of the component 154, or more generally, capture the component surface 155 from multiple different angles relative to the component 154. In some embodiments, the imaging device 150 is attached to a robotic arm.

[0036] In some embodiments, the platform 152 can remain in a fixed position throughout the inspection process. For example, the platform 152 can remain stationary, while the imaging device 150 is selectively moved relative to the platform 152 during the inspection process. Alternatively, the platform 152 can be selectively rotated during the inspection process via a rotation assembly (not shown) or a turntable (not shown).

[0037] The operation and actuation of the imaging device 150 and the positioning device 160 are controlled by the processor 120 and more generally by the computing device 110. For example, the positioning device 160 may include a plurality of articulated arms or tracks and actuators (not shown). The imaging device 150 is attached to at least one of the arms or tracks. In response to control signals transmitted by the processor 120, the actuator can selectively move along the track, or optionally, the articulated arm can be articulated in one or more directions in the XYZ space.

[0038] The imaging device 150 captures two-dimensional digital images (each referred to as a component image 170) of the surface 155 of the component 154 from multiple angles. The imaging device 150 captures at least one component image 170 and converts these images into data, which is transmitted to the processor 120 and stored in the memory 122 as image data. In some embodiments, the imaging device 150 is a sensing array. In some embodiments, the imaging device 150 is a detector array. In some embodiments, the imaging device 150 is an infrared capture device. In some embodiments, the imaging device 150 is a photodiode or position sensor that detects and measures the wavelength of light along the spectrum captured by the imaging device 150. In some embodiments, the imaging device 150 converts the wavelength measurement into a vector measurement between the imaging device 150 and the surface 155 of the component 154. In some embodiments, the imaging device 150 is a charge coupled device (CCD) camera or a charge injection device (CID) camera. CCD and CID cameras utilize image sensors to record visible light as electronic signals and convert the light into two-dimensional pixelated images, wherein each image includes the surface 155 of the component 154. In some embodiments, imaging device 150 is an RGB-D camera, wherein imaging device 150 captures a two-dimensional pixelated image and a depth of surface 155 of component 154 relative to at least one imaging device 150. In some embodiments, imaging device 150 may include one or more filters and / or lenses that limit the wavelengths allowed to travel through the filters and / or lenses. As an example, imaging device 150 may include a barrier filter that allows only specific wavelengths in the visible light spectrum to pass through the filter.

[0039] In some embodiments, multiple component images 170 from different perspectives relative to platform 152 are compiled into a single component image 170. It should be understood that complex components may require additional perspectives. Each component image in multiple component images 170 includes metadata, and in some embodiments, the metadata includes relative position information of imaging device 150 from each angle. Processor 120 can merge multiple component images 170 into a single component image 170 with associated position metadata.

[0040] At least one component image 170 collected by the imaging device 150 for each viewing angle is accessible by the processor 120. For each component image 170, a set of feature points 171 are identified and the corresponding coordinates on the XYZ axis are recorded as image metadata. The feature points 171 are selected based on their ability to be easily matched and based on their invariance to viewpoint, illumination, translation, rotation, and scaling. When component images 170 from different angles are generated during the inspection process, each viewing angle and the associated component image 170 and a set of feature points 171 are recorded in the memory 122 and stored as image metadata. The same feature points 171 are then located and identified on at least one component image 170 as image metadata.

[0041] A set of feature points 171 and associated positional metadata are used to identify and classify features of the type of part captured in the part image 170, and to identify and classify defects. Therefore, a set of feature points 171 used to identify and classify features of the type of part is referred to herein as a set of part type feature points 171 (where applicable), and a set of feature points 171 used to identify and classify features of defects is referred to herein as a set of defect type feature points 171. The corresponding coordinates of the set of defect type feature points 171 are used by the processor 120 to determine the size, shape, and size of each identified defect.

[0042] In some embodiments, a serial number or more generally an identification provided by user 102 via input device 126 may be used to identify the type of part captured in at least one part image 170, and this input may be stored as image metadata in database 1290 and / or memory 122. Because the part type has been provided via user input, identification and classification of the part type is not necessary.

[0043] As will be explained in further detail below, a neural network and, in some embodiments, a machine learning algorithm can be executed by the processor 120 in identifying a set of part type feature points 171 and in determining a part type from a set of part type feature points 171. The part type feature points 171 of the at least one part image 170 are then aligned with the part type feature points of the reference image or reference three-dimensional virtual model. An n-point perspective (PnP) process performed by the processor 120 then maps the defects by projecting or back-projecting the defects identified in the at least one part image 170 onto the reference image or reference three-dimensional virtual model. The n-point perspective (PnP) process, mapping, projection, and back-projection are exemplary methods and algorithms and are not intended to be limiting. As will be explained in further detail below, an area stored as metadata for a reference image or reference three-dimensional virtual model may also be projected or back-projected from a reference image or reference three-dimensional virtual model using an n-point perspective (PnP) process.

[0044] Figure 3A A component image 170 of an exemplary airfoil component 250 captured by the imaging device 150 is shown. It should be understood that the airfoil component 250 is merely an illustrative example of a machine component that needs to be inspected for defects and is not intended to be limiting. The airfoil component 250 includes identifiable component features and defect features that are used by the processor 120 to classify the type of component captured in the component image 170. Such identifiable features may include, for example, a leading edge 252, a trailing edge 254, a base 258, and a cantilever tip 256 of the airfoil component 250. Features may also include surface variations on the airfoil component 250. Due to normal use of the airfoil component 250 in an assembly such as a turbine, defects may occur on the airfoil component 250. Such defects may include, for example, spalling 270, stress fractures 272, and / or surface finish defects 274, such as pitting due to corrosion. The component features and defect features can be identified by the processor 120 by executing the classification module 180, such as Figures 4 to 5 shown and explained in further detail below. Figure 3A A single component image 170 is shown, however it should be understood that multiple component images 170 from different perspectives may be captured by imaging device 170. The captured identifiable features and associated location information in each component image in multiple component images 170 are stored in metadata. Each component image in multiple component images 170 and the metadata may be stitched together and the metadata combined.

[0045] Figure 3B A reference image 172 of an exemplary airfoil component 250 is shown, which is stored in the memory 122 and accessible by the processor 120 for use in aligning identifiable component features and defect features (shown in at least one component image 170 and captured by the imaging device 150) with reference component features of the reference image 172. In the illustrated embodiment, the reference image 172 is a single two-dimensional image stored in the memory 122. In some embodiments, a plurality of reference images 172 may be stored in the memory 122, each reference image in the plurality of reference images 172 corresponding to a different perspective from which the imaging device 150 captured the component image 170. In some embodiments, a reference three-dimensional virtual model is stored in the memory 122. The reference image 172 of the exemplary airfoil component 250 is merely illustrative of a reference image of a component stored in the memory 122. It should be understood that reference images of any type of component may be stored in the memory 122.

[0046] The reference image 172 (or the reference three-dimensional virtual model) includes predefined virtual areas on the surface 260 of the exemplary airfoil component 250. The surface 260 of the airfoil component 250 is divided into virtual areas 280E, 280C, 280B arranged in a matrix. Each of the designated areas 280E, 280C, 280B extends across a surface area defined by at least three points 282. Each of the at least three points 282 is connected together via a dividing line 284. The areas 280E, 280C, 280B may have any shape and / or size (indicating the resolution of the area matrix). In the illustrated embodiment, those areas adjacent to the leading edge 252 and the trailing edge 254 are defined as edge areas 280E, those areas adjacent to the base 258 are defined as base areas 280B, and those areas oriented centrally between the edge areas 280E are defined as center areas 280C. In some embodiments, regions 280E, 280C, 280B located at critical regions of the airfoil component 250 may have greater resolution and may include a smaller surface area relative to those regions 280E, 280C, 280B defined at non-critical regions of the airfoil component 250. As an example, in an exemplary embodiment, the base region 280B encompasses a larger surface area than the edge region 280E.

[0047] According to one or more parameters, virtual area 280E, 280C, 280B are shaped and selected, and regional priority is assigned to it. In some embodiments, regional priority represents the relative value of the importance of this area. As an example, due to the relative fragility of leading edge 252 and trailing edge 254, edge area 280E can be designated as having a higher regional priority relative to central area 280C or base area 280B. Similarly, due to the relative stability of base 258, lower regional priority can be assigned to base area 280B. Usually, regional priority is determined by empirical data or modeling data (such as finite element analysis). Determination may include at least one of the following: relative fragility and stability of this area, difficulty of repairing this area, time and cost of repairing defects in this area, type of one or more defects in each of these areas, method of repairing one or more defects of this type in each of these areas and material required for repairing one or more defects in each of these areas. The regions 280E, 280C, 280B and priorities are generated and assigned by the processor 120 using the partitioning module 200 and are explained in further detail below.

[0048] like Figure 4As shown, stored in the memory 122 are at least one component image 170 captured by the imaging device 150, a reference image 172 (and / or a three-dimensional virtual model), a classification module 180, and a partitioning module 200. A database 190 is also stored in the memory 122. The at least one component image 170, the reference image 172 (and / or a three-dimensional virtual model), the classification module 180, and the partitioning module 200 may be stored in the database 190 or may be stored in a separate database. The database 190 includes one or more digital data matrices that can be accessed by the processor 120, the classification module 180, and the partitioning module 200. The functions of the classification module 180 and the partitioning module 200 are accessed and executed by the processor 120, however, it should be understood that the functions of the classification module 180 may be an integral function of the processor 120.

[0049] In some embodiments, database 190 is pre-populated by user 102. In some embodiments, database 190 is populated and / or updated by processor 120. In some embodiments, database 190 is populated and / or updated by processor 120 using machine learning or artificial intelligence (AI) algorithms. Stored in database 190 are, but are not limited to, application source code, application object code, configuration data, additional input events, application state, assertion statements, verification results, and / or any other type of data.

[0050] The data matrices include, but are not limited to, a component type data matrix 191, a defect type data matrix 192, and a region weight data matrix 193. Each of the component type data matrix 191, the defect type data matrix 192, and the region weight data matrix 193 is accessible by the processor 120 independently or in combination.

[0051] Features and associated types of components are stored in the component type data matrix 191. Non-limiting examples of component types may include airfoils, liner, nozzle, shaft, wheel, piston, etc. Features generally refer to characteristic elements of the component 154, such as contours, edges, holes, perforations, and / or surface finishes observed by the imaging device 150. As an example, in Figure 3A , features associated with exemplary airfoil component 250 are leading edge 252, trailing edge 254, base 258, and cantilever tip 256, as well as perforations formed across surface 260 of airfoil component 250. In some embodiments, component type data matrix 191 is pre-populated by user 102. In some embodiments, component type data matrix 191 is loaded into memory 122 from an external source. Component type data matrix 191 is populated by processor 120 using machine learning.

[0052] Defect features and associated defect types are stored in the defect type data matrix 192. Non-limiting examples of defect types may include dislocations, cracks, wear, voids, inelastic deformations, stress concentrations, ruptures, cracks, inclusions of foreign particles, corrosion, surface stress fractures, fatigue cracking, spalling, and / or surface finish defects such as pitting due to erosion or corrosion. Defect features generally refer to characteristic elements of a defect observed by the imaging device 150, such as contours, edges, holes, and reflective changes. As an example, in Figure 3A , the defect features associated with the airfoil component 250 are spalling 270 , stress fractures 272 , and surface finish defects 274 on the surface 260 of the airfoil component 250 .

[0053] The regional weight data matrix 193 includes parameters for defining regions across any type of component. The processor 120 accesses the regional weight data matrix 193 to assign regional priorities to each region using a regional priority data matrix stored in memory. The regional priorities for each type of component are preprogrammed by the engineering or design department. In some embodiments, the regional priorities can be defined based on empirical data from previous manual inspections, or can be defined at least in part by finite element analysis simulations. As an example, in Figure 3, the regional weight data matrix 193 is programmed to assign edge regions 280E to the leading edge 252 and trailing edge 254 of the airfoil component 250. Similarly, the regional weight data matrix 193 is programmed to assign base regions 280B to the base 258 of the airfoil component 250. The regional weight data matrix 193 also includes parameters for assigning priorities to regions. These parameters include, but are not limited to: (i) the relative fragility or robustness of the area, (ii) the difficulty of repairing the area, (iii) the time and cost to repair the area, (iv) the type of defect in the area, (v) the method of repairing the defect of that type in the area, and (vi) the material required to repair the defect of that type in the area. Parameters may also be determined based on the complex geometry of the component and the operating conditions in which the component operates. Different sections of the component may experience different degradation and produce defects of different types and sizes. The sum of the determined parameters produces a component weight value. The component weight value can be used to determine whether the inspected component is repairable, scrapable, or whether the component has passed the inspection.

[0054] Figure 5 An exemplary system operation diagram of the classification module 180 and the partitioning module 200 executed by the processor 120 is shown. The at least one part image 170 is generally processed by the classification module 180 and the partitioning module 200, however, it should be understood that the functions of the classification module 180 may be an integral function of the processor 120.

[0055] At least one component image 170 is captured by imaging device 150 and stored in memory 122 as previously described. Imaging device 150 is oriented to capture at least one component image 170 of a component from at least one viewing angle. At least one component image 170 includes a set of component type feature points 171 for identifying the type of component captured in at least one component image 170, and at least one component image 170 includes a set of defect type feature points 171 for identifying defects 274 on surface 155 of at least one component image 170. The set of component type feature points 171 for identifying the type of component and the set of defect type feature points 171 for identifying defects 274 enable features of the component and features of defects on the surface of the component to be identified. In some embodiments, a serial number or more generally an identification provided by user 102 via input device 126 may be used to identify the type of component captured in at least one component image 170, and the input may be stored in database 1290 and / or memory 122 as image metadata.

[0056] The at least one part image 170 is classified by accessing a matrix stored in the database 190 by the classification module 180. The classification module 180 identifies features found in the at least one part image 170 and determines the type of the part. To assign a part type to a part captured in the part image 170, the classification module 180 detects identifiable features found in the at least one part image 170 and compares those identifiable features to a part type. Figure 4 The component type data matrix 191 is compared with the component type data matrix 191. After a threshold number of identifiable features are found in at least one component image 170, the classification module 180 assigns a component type to the at least one component image 170 and stores the component type in the database 190 as image metadata.

[0057] In some embodiments, features may be identified through a multi-class recognition and classification process, and features (such as feature points 171) are identified based on a classification process refined during one or more training phases. In some embodiments, the classification module 180 utilizes a training process involving autonomous learning and image enhancement to improve the accuracy of the classification process for at least one component image 170. The processor 120 classifies at least one component image 170 and learns from the classification process using a deep convolutional neural network (CNN) such as a U-net type image segmentation method. More generally, in some embodiments, the neural network is a segmentation neural network and is trained using a machine learning algorithm.

[0058] Thus, as processor 120 classifies additional component images 170, processor 120 is facilitated to increase the accuracy with which it classifies at least one component image 170. In some embodiments, processor 120 accessing classification module 180 identifies and reclassifies incorrectly classified component images 170. Thus, classification module 180 may correct any classification errors that resulted in an initial incorrect classification of at least one component image 170, and subsequently correctly identify at least one component image 170.

[0059] In some embodiments, classification module 180 uses a threshold value of part images 170 in each category to determine whether classification module 180 has classified a sufficient number of part images 170 to be considered accurate for identifying features or classifying part images 170 corresponding to the respective category. Features used to correctly classify the type of part are stored in database 190.

[0060] As previously referenced Figure 3B As described, at least one reference image 172 (or reference three-dimensional virtual model) includes a predefined virtual area on the surface of the component. At least one reference image 172 and at least one component image 170 are aligned so that the area of ​​at least one reference image 172 can be mapped to at least one component image 170 by the partition module 200, or vice versa. In addition, defects captured in at least one component image 170 can be mapped to at least one reference image 172. The resulting composite image 199 thus includes both areas and defects. In some embodiments, the composite three-dimensional virtual includes defects captured in at least one component image 170 and areas (from at least one reference image 172 or reference three-dimensional virtual model).

[0061] To apply the virtual area of ​​the at least one reference image 172 to the at least one component image 170, the partition module 200 aligns the component features (stored as a set of component type feature points 171) of the at least one component image 170 with the reference points of the reference image 172, so that the at least one component image 170 is aligned with the corresponding at least one reference image 172 taken from the same viewing angle by the imaging device 150. Similarly, to apply the virtual area of ​​the reference three-dimensional virtual model to the at least one component image 170, the partition module 200 aligns the component features (stored as a set of component type feature points 171) of the at least one component image 170 with the reference points of the reference three-dimensional virtual model, so that the at least one component image 170 is aligned with the reference three-dimensional virtual model. Position metadata of the at least one component image from each of the at least one viewing angle facilitates the alignment with the three-dimensional virtual model.

[0062] After the at least one component image 170 is properly aligned with the at least one reference image 172 (or the reference three-dimensional virtual model), the zoning module 200 is configured to project each defect of the at least one component image 170 onto the resulting composite image 199 (or the resulting three-dimensional virtual model). In some embodiments, each defect within the region where each defect is located is covered. In some embodiments, the region is back-projected from the at least one reference image 172 (or the reference three-dimensional virtual model) onto the at least one reference image 172 to produce the resulting composite image 199.

[0063] FIG. 6A to FIG. 6C 3 is a flow chart of an exemplary method 300 for region-guided repair extent optimization of a component. The method 300 includes providing 302 an image captured by an image capture system (such as an imaging device 150), the image capture system being oriented to capture at least one component image 170 of a component (154, 250) from at least one viewing angle, the at least one component image 170 including a set of features 171 selected to enable features of the component and features of a defect 274 on a surface of the component to be identified. The method also includes classifying 304 a type of defect 274 detected on a surface 155 of a component 154 in the at least one component image 170, and assigning 306 a region (such as regions 280B, 280C, 280E) to the surface 155 of the component 154 captured in the at least one component image 170.

[0064] In some embodiments, method 300 also includes assigning 308 a regional priority to each of the regions (280B, 280C, 280E), and determining 310 the regional priority of each of the regions (280B, 280C, 280E) from empirical data, the empirical data including at least one of the following: the relative fragility and stability of the region, the difficulty of repairing the region, the time and cost of repairing defects 274 in the region, the type of one or more defects 274 in each of the regions (280B, 280C, 280E), the method of repairing one or more defects 274 of the type in each of the regions (280B, 280C, 280E), and the materials needed to repair one or more defects 274 in each of the regions (280B, 280C, 280E).

[0065] In some embodiments, method 300 also includes identifying 312 part features found in at least one part image 170 and aligning 314 a set of features of at least one part image 170 with a corresponding set of features on a surface of one of three-dimensional virtual model 202 and at least one reference image.

[0066] In some embodiments, method 300 also includes executing 316 a neural network to identify a set of defect features for each defect 274 on surface 155 of at least one part image 170. In some embodiments, the neural network is trained using a machine learning algorithm.

[0067] In some embodiments, the method 300 further includes identifying 318 each defect 274 based on at least one of the following: the type of defect, the physical size of the defect, the repairability of the defect, the maintainability of the defect, and the criticality of the region (280B, 280C, 280E) where the defect 274 is located. In some embodiments, the method 300 further includes assigning 320 a weight to each region where the defect 274 is identified, the weight assigned according to the region priority of each region (280B, 280C, 280E). In some embodiments, the method 300 further includes projecting 322 each defect 274 of the at least one component image 170 onto at least one of the three-dimensional virtual model and the at least one component image 170, and overlaying 324 each defect 274 within the region (280B, 280C, 280E) where the defect 274 is located.

[0068] In some embodiments, method 300 also includes displaying 326 an area (280B, 280C, 280E) and a defect 274 of at least one of the three-dimensional virtual model and at least one component image 170 onto an output device 124, and in some embodiments, method 300 also includes outputting 328 at least one of the following: a cost estimate data sheet for repairing a component, a time to repair a component, maintainability and repairability of a component, a method for repairing a component, a location of a repaired component, wiring information, a three-dimensional virtual model having a color-coded area (280B, 280C, 280E) with a defect 274, and a color-coded image of the area (280B, 280C, 280E) with a defect 274.

[0069] The described system and method provide an inspection system that can use empirical test data to facilitate optimized inspection of components. The described system and method facilitates improved repair time and cost by leveraging existing knowledge about repair time and cost for a given area. This is particularly important because defects of the same type and size in different areas may take different times and / or different costs to repair.

[0070] The inspection results can be used by the inspection facility to estimate the number of scrapped parts and the inventory required to replace or repair damaged parts. Depending on the type, size and location of the defect, different repair strategies can be adopted to improve the repair process. The inspection system covers each defect within the area where each defect is located, and displays these areas and defects as user output. The inspection system is configured to output at least one of the following: a cost estimate data table to repair the part, a time estimate to repair the part, the maintainability and repairability of the part, a method to repair the part, the location of the repair part, wiring information, a three-dimensional virtual model with color-coded areas with defects, and a color-coded image of the area with defects. The described system and method also standardize the inspection, which improves accuracy and reduces costs.

[0071] It should be understood that the above description is intended to be exemplary and not restrictive. For example, the above embodiments (and / or aspects thereof) may be used in combination with each other. In addition, without departing from the scope of the subject matter presented herein, many modifications may be made to adapt specific situations or materials to the teachings of the subject matter presented herein. Although the size and type of the materials described herein are intended to limit the parameters of the disclosed subject matter, they are not restrictive and are exemplary embodiments. In reviewing the above description, many other embodiments will be apparent to those of ordinary skill in the art. Therefore, the scope of the subject matter described herein should be determined with reference to the full scope of the equivalent forms of the appended claims and such claims authorized. In the appended claims, the terms "including" and "wherein" are used as the plain English equivalent forms of the corresponding terms "including" and "wherein". In addition, in the following claims, the terms "first", "second" and "third" etc. are used only as marks and are not intended to impose numerical requirements on their objects. In addition, the limitations of the following claims are not written in the format of average value plus function, and are not intended to be interpreted based on 35 U.S.C. § 112 (f), unless and until such claims are limited to the description of the void function of other structures and the phrase "for ..." is explicitly used.

[0072] This written description uses examples to disclose several embodiments of the subject matter presented herein, including the best mode, and also enables a person of ordinary skill in the art to practice the embodiments of the disclosed subject matter, including making and using the apparatus or system and performing the method. The patentable scope of the subject matter described herein is defined by the claims, and may include other examples that occur to a person of ordinary skill in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements that are not substantially different from the literal language of the claims.

[0073] In addition, the disclosed disclosure can be embodied in the form of computer-implemented processes and devices for practicing those processes. The disclosure can also be embodied in the form of computer program code of instructions embodied in tangible media such as floppy disks, CD-ROMs, hard disk drives, or any other computer-readable storage media, wherein, when the computer program code is loaded into a computer and executed by the computer, the computer becomes a device for practicing the disclosure. The disclosure can also be embodied in the form of computer program code, for example, whether the computer program code is stored in a storage medium, loaded into a computer and / or executed by a computer, or as a data signal transmitted (whether or not modulated carrier) through some transmission medium (such as through wires or cables, through optical fibers, or via electromagnetic radiation), wherein, when the computer program code is loaded into a computer and executed by the computer, the computer becomes a device for practicing the disclosure. When implemented on a general-purpose microprocessor, the computer program code segment configures the microprocessor to create a specific logic circuit.

[0074] The methods, systems, and compositions disclosed herein are not limited to the specific embodiments described herein, but rather the steps of the methods, the elements of the systems, and / or the elements of the compositions can be used independently and separately from other steps and / or elements described herein. For example, the methods, systems, and compositions are not limited to being practiced with only the rotating machines as described herein. Instead, the methods, systems, and compositions can be implemented and used in conjunction with many other applications.

[0075] Although specific features of various embodiments may be shown in some drawings and not in others, this is for convenience only. Furthermore, reference to "one embodiment" in the above description is not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the described features. In accordance with the principles of the present disclosure, any feature of a drawing may be referenced and / or claimed in conjunction with any feature of any other drawing.

[0076] This written description uses examples, including the best mode, to enable any person skilled in the art to practice the present disclosure, including making and using any device or system and performing any combined method. The patentable scope of the present disclosure is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements that are not substantially different from the literal language of the claims.

[0077] Other aspects of the disclosure are provided by the subject matter of the following clauses.

[0078] An inspection system includes an image capture system oriented to capture at least one component image of a component from at least one viewing angle, the at least one component image including a set of features selected to enable features of the component and features of defects on a surface of the component to be identified; and a processor configured to: classify a type of defect detected on the surface of the component in the at least one component image; and assign a region to the surface of the component captured in the at least one component image.

[0079] An inspection system as described in the preceding clause, wherein the processor uses a zone priority data matrix stored in the memory to assign a zone priority to each of the zones.

[0080] An inspection system according to any of the preceding clauses, wherein the memory is configured to store the zone priority data matrix, in which the zone priority is determined by empirical data, the empirical data including at least one of: the relative fragility and stability of the zone, the difficulty of repairing the zone, the time and cost of repairing defects in the zone, the type of one or more defects in each of the zones in the zone, the method of repairing one or more defects of that type in each of the zones in the zone, and the materials required to repair the one or more defects in each of the zones in the zone.

[0081] An inspection system as described in any preceding clause, wherein the processor is further configured to identify component features found in the at least one component image.

[0082] An inspection system according to any preceding clause, wherein the memory is configured to store at least one of: a three-dimensional virtual model of the component having an area of ​​the surface map across the component and at least one reference image of the component having an area of ​​the surface map across the component.

[0083] An inspection system according to any preceding clause, wherein the processor is configured to align the set of features of the at least one component image with a corresponding set of features on a surface of one of the three-dimensional virtual model and the at least one reference image.

[0084] An inspection system as described in any preceding clause, wherein the processor is configured to execute a neural network to identify a set of defect features for each defect on the surface of the at least one component image.

[0085] An inspection system according to any preceding clause, wherein the neural network is a segmentation neural network.

[0086] An inspection system according to any preceding clause, wherein the neural network is trained using a machine learning algorithm.

[0087] An inspection system according to any preceding clause, wherein the processor is further configured to identify each defect based on at least one of: the type of defect, the physical size of the defect, the repairability of the defect, the maintainability of the defect, and the criticality of the area where the defect is located.

[0088] An inspection system according to any preceding clause, wherein the processor is further configured to assign a weight to each region in which a defect is identified, the weight being assigned according to a region priority of each region.

[0089] An inspection system as described in any preceding clause, wherein the processor is configured to project each defect of the at least one component image onto at least one of the three-dimensional virtual model and the at least one component image.

[0090] An inspection system according to any preceding clause, wherein the processor is configured to overlay each defect within the region where each defect is located.

[0091] The inspection system of any preceding clause, wherein areas and defects of at least one of the three-dimensional virtual model and the at least one component image are stored in a memory and displayed on an output device.

[0092] An inspection system according to any preceding clause, wherein the processor is configured to output at least one of: a cost estimate data sheet to repair a component, a time estimate to repair a component, maintainability and repairability of a component, a method to repair a component, a location of a repair component, wiring information, a three-dimensional virtual model having color-coded areas with defects, and a color-coded image of an area with defects.

[0093] A method for region-guided repair range optimization of a component, the method comprising: providing an image captured by an image capture system, the image capture system being oriented to capture at least one component image of a component from at least one viewing angle, the at least one component image comprising a set of features selected to enable features of the component and features of defects on a surface of the component to be identified; and classifying a type of defect detected on the surface of the component in the at least one component image; and assigning a region to the surface of the component captured in the at least one component image.

[0094] A method as described in the preceding clause, further comprising assigning a regional priority to each of the regions.

[0095] A method according to any of the preceding clauses, the method also includes determining a regional priority for each area in the area based on empirical data, the empirical data including at least one of the following: the relative fragility and stability of the area, the difficulty of repairing the area, the time and cost of repairing defects in the area, the type of one or more defects in each area in the area, the method of repairing one or more defects of that type in each area in the area, and the materials required to repair the one or more defects in each area in the area.

[0096] A method as described in any preceding clause, further comprising identifying component features found in the at least one component image.

[0097] A method according to any preceding clause, further comprising aligning the set of features of the at least one component image with a corresponding set of features on a surface of one of the three-dimensional virtual model and at least one reference image.

[0098] A method according to any preceding clause, wherein the neural network is a segmentation neural network.

[0099] A method as in any preceding clause, further comprising executing, by a processor, a neural network to identify, for each defect on the surface of the at least one part image, a set of defect features.

[0100] A method according to any preceding clause, further comprising training the neural network using a machine learning algorithm.

[0101] A method according to any preceding clause, further comprising identifying each defect based on at least one of: a type of defect, a physical size of the defect, a repairability of the defect, a maintainability of the defect, and a criticality of an area in which the defect is located.

[0102] A method as described in any preceding clause, further comprising assigning a weight to each region in which a defect is identified, the weight being assigned according to a region priority of each region.

[0103] A method according to any preceding clause, further comprising projecting each defect of the at least one component image onto at least one of the three-dimensional virtual model and the at least one component image.

[0104] A method according to any preceding clause, further comprising covering each defect within a region in which each defect is located.

[0105] A method according to any preceding clause, further comprising displaying regions and defects of at least one of the three-dimensional virtual model and the at least one component image on an output device.

[0106] A method according to any of the preceding clauses, the method further comprising outputting at least one of: a cost estimate data sheet to repair a component, a time estimate to repair a component, the maintainability and repairability of a component, a method to repair a component, a location to repair a component, wiring information, a three-dimensional virtual model having color-coded areas with defects, and a color-coded image of an area with defects.

Claims

1. An inspection system (100), comprising: an image capture system (150) oriented to capture at least one component image of a component (250) from at least one viewing angle, the at least one component image (170) comprising a set of features (171) selected to enable features of the component (250) and features of a defect (274) on a surface (260) of the component (250) to be identified; A processor (120), the processor being configured to: classifying a type of defect (274) detected on a surface (260) of the component (250) in the at least one component image (170); as well as A region (280) is assigned (308) to a surface (260) of the component (250) captured in the at least one component image (170).

2. The inspection system (100) of claim 1, wherein the processor (120) uses a zone priority data matrix stored in a memory (122) to assign a zone priority to each of the zones (280).

3. The inspection system according to claim 2, wherein the memory (122) is configured to store the regional priority data matrix, in which the regional priority is determined by empirical data, and the empirical data includes at least one of the following: the relative fragility and stability of the region (280), the difficulty of repairing the region (280), the time and cost of repairing the defects in the region (280), the type of one or more defects in each of the regions (280), the method of repairing one or more defects (274) of the type in each of the regions (280), and the materials required to repair the one or more defects in each of the regions (280).

4. The inspection system (100) of claim 1, wherein the processor (120) is further configured to identify component features found in the at least one component image (170).

5. The inspection system of claim 4, wherein the memory is configured to store at least one of: a three-dimensional virtual model (172) of the component having an area (280) mapped across a surface (260) of the component (250) and at least one reference image (172) of the component having an area (280) mapped across a surface (260) of the component (250).

6. The inspection system of claim 5, wherein the processor is configured to: aligning (314) the set of features of the at least one part image (170) with a corresponding set of features on a surface (260) of one of the three-dimensional virtual model (172) and the at least one reference image (172); executing (316) a neural network to identify a set of defect features (171) for each defect on the surface (260) of the at least one part image (170); and Each defect is identified based on at least one of: a type of defect, a physical size of the defect, a repairability of the defect, a maintainability of the defect, and a criticality of the area where the defect is located.

7. The inspection system according to claim 1, wherein the processor (100) is further configured to: assigning (320) a weight to each region (280) in which a defect is identified, the weight being assigned according to a region priority of each region (280); projecting (322) each defect of the at least one component image (170) onto at least one of the three-dimensional virtual model (172) and the at least one component image (172); and Each defect within the region where each defect is located is covered (324).

8. A method for optimizing the area-guided repair range of a component (250), the method comprising: providing (302) an image (170) captured by an image capture system (150), the image capture system being oriented to capture at least one component image (170) of a component (250) from at least one viewing angle, the at least one component image (170) comprising a set of features (171) selected to enable features of the component (250) and features of a defect (274) on a surface (260) of the component (250) to be identified; classifying (304) a type of defect (274) detected on a surface (260) of the component (250) in the at least one component image (170); and A region (280) is assigned (308) to a surface (260) of the component (250) captured in the at least one component image (170).

9. The method according to claim 8, further comprising: assigning (308) a region priority to each of the regions (280); determining (310) the area priority for each of the areas (280) from empirical data, the empirical data comprising at least one of the following: relative fragility and robustness of the areas (280), difficulty in repairing the areas (280), time and cost to repair the defects in the areas (280), type of one or more defects (274) in each of the areas (280), method of repairing one or more defects (274) of the type in each of the areas (280), and materials required to repair the one or more defects (274) in each of the areas (280); as well as Part features (171) found in the at least one part image (170) are identified (312).

10. The method according to claim 9, further comprising: aligning (314) the set of features (171) of the at least one component image (170) with a corresponding set of features on a surface (155) of one of a three-dimensional virtual model (172) and at least one reference image (172); executing (316) a neural network by a processor to identify a set of defect features for each defect (274) on the surface (155) of the at least one part image (172); as well as Each defect (274) is identified (318) based on at least one of: a type of defect, a physical size of the defect (264), a repairability of the defect (274), a maintainability of the defect (274), and a criticality of the area (280) in which the defect (274) is located.

Images