System and methods for zone-guided repair scope optimization

Abstract

To provide a system and method for zone-guided repair scope optimization.SOLUTION: An inspection system (100) is an image capturing system oriented to capture at least one component image (170) of a component from at least one viewing angle, the at least one component image (170) including a set of features selected to enable features of the components and features of defects on surfaces of the component to be identified, and includes an image capturing system and a processor (120). The processor (120) is configured to classify a type of defect detected on surfaces of the component in the at least one component image (170) and assign zones to surfaces of the component captured in the at least one component image (170).SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The subject matter described herein relates to systems and methods for part inspection, and more particularly, to zone-guided systems and methods for part inspection. [Background technology]

[0002] Manufacturers and end users commonly perform periodic inspections of parts to ensure that they are free of mechanical and material defects and to determine that they are in proper operating condition. Known defects may include, for example, commonly surface stress fractures, fatigue cracks, inelastic deformation, and surface finish defects such as pitting and / or chipping due to erosion or corrosion.

[0003] Known inspections may include visual inspection to detect surface defects and measurements to determine tolerance variations. As a result of such inspection, the operator may determine that the part is not defective. Alternatively, the operator may determine that the part requires repair or refurbishment, that the part should be scrapped, or that more detailed inspection and testing with specialized equipment and processes is required. In making such a determination, the operator may consider factors such as the type and size of the defect, the location of the defect on the part, the acceptable tolerance limits of the part, and / or other known factors that may affect the recommended course of action for the part (such as the time required to perform the repair or refurbishment, reliability analysis, and calculations of the remaining useful life of the part). Furthermore, depending on the part and / or the extent of the damage or defect, the operator may weigh the cost of further inspection and processing against replacing the part entirely.

[0004] Although operators are trained in inspection techniques and cost assessment, the process is subjective and may vary between operators and / or between different repair facilities. Additionally, the decision-making process is inherently inefficient and subject to inaccuracies caused by user error. Even with properly adopted guidelines and training procedures, operators typically do not have the authority necessary to approve part scrapping and replacement; this decision is typically made by management, which can increase inconsistency and lengthen the process.

[0005] At least some known facilities use automated inspection systems to improve process consistency and potentially reduce the time associated with the process. At least some known automated inspection systems perform part imaging and defect recognition via machine learning algorithms that analyze digital two-dimensional images of the part's surface. During analysis of the surface, surface defects are detected using conventional machine learning algorithms. Using high resolution imaging, relatively small defects (such as hairline cracks) can be detected and classified by the machine learning algorithms, but high resolution imaging of hairline cracks can be processor intensive and increase processing time.

[0006] Therefore, there is a need to provide a system and method for inspecting parts in a manner that can improve the process, accuracy, and consistency of defect classification and repair. Summary of the Invention

[0007] In one aspect, an inspection system is disclosed that includes an image capture system oriented to capture at least one part image of a part from at least one view angle, the at least one part image including a set of features selected to identify features of the part and features of defects on a surface of the part, and a processor configured to classify a type of defect detected on the surface of the part in the at least one part image and assign a number of zones to the surface of the part captured in the at least one part image.

[0008] In another aspect, a method for zone-guided repair coverage optimization of a part includes providing images captured by an image capture system oriented to capture at least one part image of the part from at least one view angle, the at least one part image including a set of features selected to identify features of the part and features of defects on a surface of the part; classifying a type of defect detected on the surface of the part in the at least one part image; and assigning a zone to the surface of the part captured in the at least one part image. [Brief description of the drawings]

[0009] The subject matter of the present disclosure will be explained in detail in the following description with reference to exemplary embodiments illustrated in the drawings. [Figure 1] FIG. 1 is a schematic diagram of an exemplary computer processing system. [Diagram 2] 1 is a schematic diagram of an exemplary inspection system including an imaging device that captures part images. [Figure 3A] 3 illustrates an exemplary part image captured by the imaging device of FIG. 2. [Figure 3B] FIG. 2 is a diagram showing a reference image stored in a memory. [Figure 4] 2 illustrates exemplary data stored in memory of the computer processing system of FIG. 1. [Diagram 5] 2 illustrates an exemplary operational diagram of a classification module and a zoning module operated by a processor of the computer processing system of FIG. 1. [Figure 6A] FIG. 3 illustrates an example flow diagram of an example method 300 for inspecting a part. [Figure 6B] FIG. 3 illustrates an example flow diagram of an example method 300 for inspecting a part. [Figure 6C] FIG. 3 illustrates an example flow diagram of an example method 300 for inspecting a part.

[0010] The symbols used in the drawings and their meanings are summarized in the list of symbols. As a rule, the same symbols are used in the drawings to refer to the same parts. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the following specification and claims, reference will be made to a number of terms that shall be defined to have the following meanings.

[0012] References herein to "an example or exemplary embodiment" or "one embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0013] As used herein, the singular forms "a," "an," and "the" include plurals unless the context clearly dictates otherwise. The terms "comprising," "including," "having," and "including" are intended to be inclusive and mean that there may be additional elements other than the listed elements. The term "optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where the event occurs and instances where the event does not occur.

[0014] Unless otherwise indicated, approximation language used herein (such as "generally," "substantially," and "approximately") indicates that the term modified by the language is not absolute or complete, but merely approximate as recognized by those skilled in the art. Thus, values ​​modified by terms such as "approximately," "about," and "substantially" are not limited to the exact value specified. In at least some instances, approximation language may correspond to the precision of an instrument for measuring the value. In the present specification and claims, range limits may be specified. The ranges are combinable and / or interchangeable, and include all subranges subsumed therein, unless the context or language dictates otherwise.

[0015] Moreover, unless otherwise indicated, terms such as "first," "second," and the like are used herein merely as labels, and are not intended to impose any sequential, positional, or hierarchical requirements on the items they refer to. Moreover, reference to, for example, a "second" item does not require or preclude the presence of, for example, a "first" or lower-numbered item or a "third" or higher-numbered item.

[0016] In one embodiment, a computer program is provided, the program being embodied in a computer readable medium. In one exemplary embodiment, the system runs on a single computer system without requiring connection to a server computer. In another embodiment, the system runs in a Windows environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Washington). In yet another embodiment, the system runs in a mainframe environment and a UNIX server environment (UNIX is a registered trademark of X / Open Company Limited, Reading, Berkshire, United Kingdom). In another embodiment, the system is implemented in an iOS environment (iOS is a registered trademark of Cisco Systems, Inc., San Jose, Calif.). In yet another embodiment, the system runs in a Mac OS environment (Mac OS is a registered trademark of Apple Inc., Cupertino, Calif.). In yet another embodiment, the system runs on the Android OS (Android is a registered trademark of Google, Inc., Mountain View, Calif.). In another embodiment, the system runs on the Linux OS (Linux is a registered trademark of Linus Torvalds, Boston, Mass.). The application is flexible and designed to run in a variety of different environments without compromising its primary functionality. In some embodiments, the system includes multiple components distributed across multiple computing devices. One or more of the components may be in the form of computer-executable instructions embodied in a computer-readable medium.

[0017] As used herein, the terms "software" and "firmware" are used interchangeably and include any computer program that is stored in memory (such as RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory) and executed by a processor. The above memory types are merely exemplary and thus are not limiting of the types of memory that may be used to store computer programs.

[0018] As used herein, the terms "processor" and "computer" and related terms (e.g., "processing device," "computing device," and "controller") are not limited to integrated circuits referred to in the art as computers, but broadly refer to microcontrollers, microcomputers, analog computers, programmable logic controllers (PLCs), application specific integrated circuits (ASICs), and / or other programmable circuits, and such terms are used interchangeably herein. In embodiments described herein, memory includes, but is not limited to, computer readable media, such as random access memory (RAM), computer readable non-volatile media, such as flash memory. Alternatively, floppy disks, compact disk read only memory (CD-ROM), magneto-optical disks (MOD), and / or digital versatile disks (DVDs) may be used. Also, in embodiments described herein, additional input channels may be, but are not limited to, computer peripherals associated with an operator interface, such as a touch screen, mouse, and keyboard, or other computer peripherals, such as, but not limited to, a scanner. Further, in an exemplary embodiment, the additional output channels may include, but are not limited to, an operator interface monitor or a heads-up display. Some embodiments use one or more electronic or computing devices. Such devices typically include a processor, processing device, or controller, such as a general purpose central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, a reduced instruction set computer (RISC) processor, an ASIC, a PLC, a field programmable gate array (FPGA), a digital signal processing (DSP) device, and / or any other circuit or processing device capable of performing the functions described herein.The methods described herein may be encoded as executable instructions embodied in a computer-readable medium, such as, but not limited to, a storage device and / or a memory device. Such instructions, when executed by a processing device, cause the processing device to perform at least a portion of the methods described herein. The above examples are illustrative only, and thus are not intended to limit in any way the definition and / or meaning of the terms processor and processing device.

[0019] As used herein, the term "non-transitory computer-readable medium" is intended to represent any tangible computer-based device implemented by any method or technology for short-term and long-term storage of information (such as computer-readable instructions, data structures, program modules and sub-modules, or other data of any device). Thus, the methods described herein can be encoded as executable instructions embodied in a tangible, non-transitory computer-readable medium (such as, but not limited to, storage and / or memory devices). Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Furthermore, as used herein, the term "non-transitory computer-readable medium" includes all tangible computer-readable media. All tangible computer-readable media includes, but is not limited to, non-transitory computer storage (including, but not limited to, volatile and non-volatile media and removable and non-removable media (such as firmware, physical storage, virtual storage, CD-ROM, DVD, etc.)), any other digital source (such as a network or the Internet), and digital means yet to be developed, the only exception being a transitory propagating signal.

[0020] The systems and processes are not limited to the specific embodiments described herein. In addition, each system component and each step can be performed separately and independently from other components and steps described herein. Each component and each step can also be used in combination with other assembly packages and steps.

[0021] The embodiments described herein provide an inspection system and method for detailed inspection of a part. The inspection system includes an image capture system oriented to capture at least one part image of the part from at least one view angle, the at least one part image including a set of features selected to identify features of the part and features of defects on a surface of the part. The processor is configured to classify types of defects detected on the surface of the part in the at least one part image and assign a number of zones to the surface of the part captured in the at least one part image.

[0022] As used herein, the term "defect" refers to a portion or area of ​​a surface of a component that is damaged as a result of normal operation and use of the component. Defects include, but are not limited to, dislocations, crevices, wear, voids, inelastic deformations, stress concentrations, fractures, cracks, inclusions of foreign bodies, corrosion, surface stress fractures, fatigue cracks, spalling, and surface finish defects (such as pitting due to erosion or corrosion). These defects are discernible on the surface of the component by visual inspection.

[0023] FIG. 1 illustrates an exemplary configuration of a computing device 110 of an exemplary inspection system 100. In an exemplary embodiment, the computing device 110 can be operated by a user 102 and is communicatively coupled to a remote device (such as an imaging device 150 and / or a positioning device 160). The computing device includes a processor 120 for executing instructions. In some embodiments, the executable instructions are stored in a memory 122. The processor 120 can include one or more processing units (e.g., in a multi-core configuration). The memory 122 can be any device capable of storing and retrieving information (such as executable instructions and / or transactional data). The memory 122 can include one or more computer-readable media. The memory 122 can be locally connected to the computing device 110 or can be stored on a remote network or cloud accessible by the computing device 110.

[0024] Computing device 110 may also include a communication interface 130 coupled to imaging device 150 and positioning device 160. Communication interface 130 may include, for example, a wired or wireless network adapter and / or a wireless data transceiver for use with a mobile communication network.

[0025] In some embodiments, the computing device 110 also includes at least one media output component 124 that presents information to the user 102. The media output component 124 may be any component capable of communicating information to the user 102. In some embodiments, the media output component 124 includes an output adapter (not shown), such as a video adapter and / or an audio adapter. The output adapter may be operably coupled to the processor 120 and may be operably coupled to an output device, such as a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED) display, or an "electronic ink" display) or an audio output device (e.g., a speaker or headphones). In some embodiments, the media output component 124 presents a graphical user interface (e.g., a web browser and / or a client application) to the user 102. The graphical user interface may include, for example, an interface for viewing analysis results of one or more systems of interest. In some embodiments, the computing device 110 includes an input device 126 for receiving input from the user 102. The input devices 126 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch-sensitive device (e.g., a touchpad or touch screen), a gyroscope, an accelerometer, a position detector, a biometric input device, and / or a voice input device. A single component (e.g., a touch screen) may function as both an output device for the media output component 124 and as an input device 126.

[0026] Stored in memory 122 are computer readable instructions for, for example, providing a user interface to user 102 through media output components 124, and optionally receiving and processing input from input devices 126. The user interface may include a web browser and / or a client application, among other possibilities. Processor 120 executes computer executable instructions for implementing aspects of the present disclosure. In some embodiments, processor 120 is transformed into a special purpose microprocessor by executing computer executable instructions or being otherwise programmed.

[0027] 2, a part 154 to be inspected is placed on a platform 152 so that a surface 155 of the part 154 can be imaged by the imaging device 150. The platform 152, the part 154, and the imaging device 150 are arranged in a space defined by the XYZ axes. The platform 152 is arranged horizontally with respect to a plane defined by the XZ axes, and the imaging device 150 is arranged on the platform 152 and can move in the XYZ space.

[0028] In some embodiments, imaging device 150 is a fixed, multi-camera system. In some embodiments, imaging device 150 can be articulatable by positioning device 160. In some embodiments, imaging device 150 is articulatable such that imaging device field of view 151 images at least a portion of surface 155 of part 154 from different angles. In some embodiments, multiple imaging devices 150 can be positioned at different angles relative to platform 152 such that multiple imaging devices 150 can capture at least one part image 170 of a side of part 154, or more generally, image part surface 155 from multiple different angles relative to part 154. In some embodiments, imaging device 150 is attached to a robotic arm.

[0029] In some embodiments, the platform 152 may remain in a fixed position throughout the inspection process. For example, the platform 152 may remain stationary while the imaging device 150 selectively moves relative to the platform 152 during the inspection process. Alternatively, the platform 152 may be selectively rotated during the inspection process by a pivot assembly (not shown) or a turntable (not shown).

[0030] The movement and actuation of the imaging device 150 and the positioning device 160 are controlled by the processor 120, or more generally by the computing device 110. For example, the positioning device 160 may include a number of articulated arms or rails and an actuator (not shown). The imaging device 150 is fixed to at least one of the arms or rails. In response to control signals transmitted by the processor 120, the actuator may selectively move along the rail, or alternatively, the articulated arm may articulate in one or more directions in XYZ space.

[0031] The imaging device 150 captures multiple two-dimensional digital images of the surface 155 of the part 154 from multiple angles, each two-dimensional digital image being referred to as a part image 170. The imaging device 150 captures at least one part image 170 and converts the image into data that is sent to the processor 120 and stored in the memory 122 as image data. In some embodiments, the imaging device 150 is a sensing array. In some embodiments, the imaging device 150 is a detector array. In some embodiments, the imaging device 150 is an infrared capture device. In some embodiments, the imaging device 150 is a photodiode or position sensor that detects and measures wavelengths of light in the spectrum captured by the imaging device 150. In some embodiments, the imaging device 150 converts the wavelength measurements into vector measurements between the imaging device 150 and the surface 155 of the part 154. In some embodiments, the imaging device 150 is a charge-coupled device (CCD) camera or a charge-injection device (CID) camera. CCD and CID cameras utilize image sensors to store visible light as electronic signals and convert the light into pixelated, two-dimensional images. Each image includes the surface 155 of the part 154. In some embodiments, the imagers 150 are RGB-D cameras that capture pixelated, two-dimensional images and the depth of the surface 155 of the part 154 relative to at least one imager 150. In some embodiments, the imagers 150 can include one or more filters and / or lenses that limit the wavelengths that are allowed to pass through the filters and / or lenses. As an example, the imagers 150 can include a barrier filter that allows only certain wavelengths of the visible light spectrum to pass through the filter.

[0032] In some embodiments, multiple part images 170 from different view angles relative to platform 152 are compiled into a single part image 170. It should be understood that for complex parts, additional view angles may be required. Each part image in multiple part images 170 includes metadata, which in some embodiments includes relative position information of imager 150 from each angle. Processor 120 can merge multiple part images 170 into a single part image 170 with associated position metadata.

[0033] At least one part image 170 collected by the imaging device 150 for each view angle can be accessed by the processor 120. For each part image 170, a set of feature points 171 are identified and corresponding coordinates in the XYZ axes are recorded as image metadata. The feature points 171 are selected based on whether they can be easily matched and whether they are invariant to viewpoint, lighting, translation, rotation, and scale. As part images 170 from different angles are generated during the inspection process, each view angle and associated set of part images 170 and feature points 171 are recorded and stored in memory 122 as image metadata. The same feature points 171 are then located and identified on at least one part image 170 as image metadata.

[0034] The set of feature points 171 and associated location metadata are used in identifying and classifying part type features and identifying and classifying defects captured in part image 170. Thus, the set of feature points 171 used in identifying and classifying part type features is referred to herein as the set of part type feature points 171 (if applicable), and the set of feature points 171 used to identify and classify defect features is referred to herein as the set of defect type feature points 171. The corresponding coordinates of the set of defect type feature points 171 are used by processor 120 in determining the dimensions, shape, and size of each identified defect.

[0035] In some embodiments, a serial number, or more generally, an identifier, provided by user 102 through input device 126 may be used to identify the type of part captured in at least one part image 170, and that input may be stored as image metadata in database 1290 and / or memory 122. Since the part type has been provided through user input, it is not necessary to identify and classify the part type.

[0036] As described in more detail below, in identifying the set of part type feature points 171 and determining the part type from the set of part type feature points 171, the processor 120 may execute neural networks and, in some embodiments, machine learning algorithms. The part type feature points 171 of the at least one part image 170 are then aligned with the part type feature points of the reference image or reference three-dimensional virtual model. A Perspective-n-Point (PnP) process executed by the processor 120 then maps defects by projecting or back-projecting the defects identified in the at least one part image 170 onto the reference image or reference three-dimensional virtual model. The Perspective-n-Point (PnP) process, mapping, projection and back-projection are exemplary methods and algorithms and are not intended to be limiting. Also, as described in more detail below, zones stored as metadata of the reference image or reference three-dimensional virtual model may also be projected or back-projected from the reference image or reference three-dimensional virtual model by using the Perspective-n-Point (PnP) process.

[0037] 3A illustrates a part image 170 of an exemplary airfoil part 250 captured by the imaging device 150. It is understood that the airfoil part 250 is merely exemplary of a mechanical part requiring inspection for defects and is not intended to be limiting. The airfoil part 250 includes identifiable part features and defect features that are utilized by the processor 120 in classifying the type of part captured in the part image 170. Such identifiable features may include, for example, the leading edge 252, the trailing edge 254, the base 258, and the cantilever tip 256 of the airfoil part 250. The features may also include surface variations of the airfoil part 250. Defects may occur in the airfoil part 250 during normal use in an assembly (such as a turbine). Such defects may include, for example, chipping 270, stress fractures 272, and / or surface finish defects 274 (such as corrosion pitting). The part features and defect features may be identified by the processor 120 by executing a classification module 180, as shown in Figures 4-5 and described in further detail below. While Figure 3A shows a single part image 170, it is understood that multiple part images 170 may be captured by the imaging device 170 from different view angles. For each image in the multiple part images 170, the captured identifiable features and associated location information are stored in metadata. The metadata and each of the multiple part images 170 may be stitched together to combine the metadata.

[0038] 3B illustrates a reference image 172 of an exemplary airfoil part 250 stored in memory 122 and accessible by processor 120 for aligning identifiable and defect features of the part (shown in at least one part image 170 and imaged by imaging device 150) with reference features of the part in reference image 172. In the illustrated embodiment, reference image 172 is a single two-dimensional image stored in memory 122. In some embodiments, multiple reference images 172 may be stored in memory 122, each of the multiple reference images 172 corresponding to a different view angle at which imaging device 150 images part image 170. In some embodiments, a reference three-dimensional virtual model is stored in memory 122. Reference image 172 of exemplary airfoil part 250 is merely illustrative of a reference image for a part stored in memory 122. It is understood that reference images of any type of part may be stored in memory 122.

[0039] The reference image 172 (or the reference three-dimensional virtual model) includes predefined virtual zones on the surface 260 of the exemplary airfoil part 250. The surface 260 of the airfoil part 250 is divided into virtual zones 280E, 280C, 280B arranged in a matrix. Each designated zone 280E, 280C, 280B spans a surface area defined by at least three points 282. Each of the at least three points 282 is connected to each other via a dividing line 284. The zones 280E, 280C, 280B can have any shape and / or size (representing the resolution of the zone matrix). In the illustrated embodiment, the zones adjacent to the leading edge 252 and the trailing edge 254 are defined as edge zones 280E, the zones adjacent to the base 258 are defined as base zones 280B, and the zone centrally located between the edge zones 280E is defined as a central zone 280C. In some embodiments, the zones 280E, 280C, 280B located in critical zones of the airfoil part 250 may have greater resolution and smaller surface area compared to the zones 280E, 280C, 280B defined in non-critical zones of the airfoil part 250. By way of example, in the exemplary embodiment, the base zone 280B includes a greater surface area than the edge zone 280E.

[0040] The virtual zones 280E, 280C, 280B are formed, selected, and assigned zone priorities according to one or more parameters. In some embodiments, the zone priorities represent a relative numerical value of the importance of the zones. As an example, the edge zone 280E may be designated to have a higher zone priority relative to the central zone 280C or the base zone 280B due to the relative fragility of the leading edge 252 and the trailing edge 254. Similarly, the base zone 280B may be assigned a lower zone priority due to the relative robustness of the base 258. Generally, the zone priorities are determined by empirical or modeling data (e.g., finite element analysis). Factors in the determination include at least one of the following: the relative brittleness and robustness of the zones, the difficulty of repairing the zones, the time and cost to repair defects in the zones, the type or types of defects in each zone, the method of repairing the type or types of defects in each zone, and the material required to repair the defect or defects in each zone. Zones 280E, 280C, 280B and priorities are created and assigned by processor 120 using zoning module 200. Zones 280E, 280C, 280B and priorities are described in further detail below.

[0041] 4, the memory 122 stores at least one part image 170 captured by the imaging device 150, a reference image 172 (and / or a three-dimensional virtual model), a classification module 180, and a zoning module 200. The memory 122 also stores a database 190. The at least one part image 170, the reference image 172 (and / or a three-dimensional virtual model), the classification module 180, and the zoning module 200 may be stored in the database 190 or in separate databases. The database 190 includes one or more digital data matrices accessible by the processor 120, the classification module 180, and the zoning module 200. The functions of the classification module 180 and the zoning module 200 are accessed and executed by the processor 120, although it will be understood that the functions of the classification module 180 may be functions integrated into the processor 120.

[0042] In some embodiments, database 190 is pre-populated by user 102. In some embodiments, database 190 is populated and / or updated by processor 120. In some embodiments, database 190 is populated and / or updated by processor 120 using machine learning or artificial intelligence (AI) algorithms by processor 120. Database 190 may store, but is not limited to, application source code, application object code, configuration data, additional input events, application state, assertion statements, validation results, and / or any other type of data.

[0043] The data matrices include, but are not limited to, part type data matrix 191, defect type data matrix 192, and zone weight data matrix 193. Each data matrix of part type data matrix 191, defect type data matrix 192, and zone weight data matrix 193 can be accessed by processor 120 independently or in combination.

[0044] Part features and associated types are stored in a part type data matrix 191. Non-limiting examples of part types include airfoil, combustor liner, nozzle, shaft, wheel, piston, etc. Features generally represent distinctive elements of the part 154, such as contours, edges, openings, perforations, and / or surface finishes observed by the imaging device 150. By way of example, in FIG. 3A, features associated with an exemplary airfoil part 250 are leading edge 252, trailing edge 254, base 258, and cantilever tip 256, as well as perforations formed in surface 260 of the airfoil part 250. In some embodiments, the part type data matrix 191 is pre-populated by the user 102. In some embodiments, the part type data matrix 191 is loaded into memory 122 from an external source. Using machine learning, the part type data matrix 191 is populated by the processor 120.

[0045] The defect features and associated defect types are stored in a defect type data matrix 192. Non-limiting examples of defect types include dislocations, cracks, wear, voids, inelastic deformations, stress concentrations, fractures, cracks, inclusions, corrosion, surface stress fractures, fatigue cracks, chips, and / or surface finish defects (such as pitting due to erosion or corrosion). Defect features generally represent characteristic elements of the defect (such as contours, edges, openings, and reflective changes observed by the imaging device 150). By way of example, in FIG. 3A, defect features associated with the airfoil component 250 are a peel 270, a stress fracture 272, and a surface finish defect 274 in the surface 260 of the airfoil component 250.

[0046] The zone weight data matrix 193 includes parameters for defining zones for each type of part. The processor 120 accesses the zone weight data matrix 193 and assigns zone priorities to each zone using a zone priority data matrix stored in memory. The zone priorities for each type of part are preprogrammed by an engineering or design department. In some embodiments, the zone priorities may be defined based on empirical data from past manual inspections or may be defined at least in part by finite element analysis simulations. As an example, in FIG. 3, the zone weight data matrix 193 is programmed to assign edge zone 280E to leading edge 252 and trailing edge 254 of airfoil part 250. Similarly, the zone weight data matrix 193 is programmed to assign root zone 280B to base 258 of airfoil part 250. The zone weight data matrix 193 also includes parameters for assigning priorities to zones. The parameters include, but are not limited to, the following: (i) relative brittleness or robustness of the zone, (ii) difficulty of repairing the zone, (iii) time and cost of repairing the zone, (iv) type of defect in the zone, (v) method of repairing the defect in the zone, and (vi) material required to repair the defect in the zone. The parameters can also be determined based on the complex geometry of the part and the operating conditions under which the part operates. Different parts in a part may undergo different degradation resulting in different types and sizes of defects. The sum of the determined parameters is the weight value of the part. The weight value of the part can be used in determining whether the part being inspected is repairable, scrapped, or whether the part has passed inspection.

[0047] 5 illustrates an exemplary system operational diagram of the classification module 180 and the zoning module 200 executed by the processor 120. At least one part image 170 is generally processed by the classification module 180 and the zoning module 200, although it will be understood that the functionality of the classification module 180 may be an integrated function of the processor 120.

[0048] At least one part image 170 is captured by the imaging device 150 as described above and stored in the memory 122. The imaging device 150 is oriented to capture at least one part image 170 of the part from at least one view angle. The at least one part image 170 includes a set of part type feature points 171 for identifying a type of part captured in the at least one part image 170, and the at least one part image 170 includes a set of defect type feature points 171 for identifying defects 274 in the surface 155 of the at least one part image 170. The set of part type feature points 171 for identifying the part type and the set of defect type feature points 171 for identifying the defects 274 allow for identification of the part features and the defect features of the surface of the part. In some embodiments, a serial number (or, generally, an identifier) ​​provided by the user 102 through the input device 126 may be used to identify the type of part captured in the at least one part image 170, and the input may be stored in the database 1290 and / or memory 122 as image metadata.

[0049] The at least one part image 170 is classified by a classification module 180 that has access to matrices stored in a database 190. The classification module 180 identifies features found in the at least one part image 170 to determine a part type. To assign a part type to a part captured in the part image 170, the classification module 180 detects identifiable features found in the at least one part image 170 and compares the identifiable features to the part type data matrix 191 of FIG. 4. After a threshold number of identifiable features are found in the at least one part image 170, the classification module 180 assigns a part type to the at least one part image 170 and stores the part type as image metadata in the database 190.

[0050] In some embodiments, features may be identified by a multi-class recognition and classification process, and features (e.g., feature points 171) may be identified based on a classification process refined during one or more training phases. In some embodiments, the classification module 180 utilizes a training process that includes self-learning and image augmentation to improve the accuracy of the classification process for at least one part image 170. The processor 120 classifies at least one part image 170 and learns from the classification process using a deep convolutional neural network (CNN) (e.g., a U-Net style image segmentation method). More generally, in some embodiments, the neural network is a segmentation neural network and is trained using machine learning algorithms.

[0051] Thus, the accuracy with which processor 120 classifies at least one part image 170 may be improved as processor 120 classifies additional part images 170. In some embodiments, incorrectly classified part images 170 are identified and reclassified by processor 120 accessing classification module 180. Thus, classification module 180 may correct classification errors that cause at least one part image 170 to be initially misclassified, and thereafter correctly identify at least one part image 170.

[0052] In some embodiments, classification module 180 uses a threshold of part images 170 in each class to determine whether classification module 180 has classified a sufficient number of part images 170 to be considered accurate in identifying features or classifying the part images 170 that correspond to the respective class. The features used to correctly classify the part type are stored in database 190.

[0053] As previously described with reference to FIG. 3B, the at least one reference image 172 (or the reference three-dimensional virtual model) includes predefined virtual zones on the surface of the part. The at least one reference image 172 and the at least one part image 170 are aligned such that the zones of the at least one reference image 172 are mapped to the at least one part image 170 (or vice versa) by the zoning module 200. Furthermore, the defects captured in the at least one part image 170 are mapped to the at least one reference image 172. Thus, the resulting composite image 199 includes both the zones and the defects. In some embodiments, the three-dimensional composite virtual image includes the defects captured in the at least one part image 170 and the zones obtained from the at least one reference image 172 or the reference three-dimensional virtual model.

[0054] To apply the virtual zones of the at least one reference image 172 to the at least one part image 170, the zoning module 200 aligns the part features (stored as a set of part type feature points 171) of the at least one part image 170 with the reference points of the reference image 172 such that the at least one part image 170 is aligned with the at least one reference image 172 captured by the imaging device 150 from the same view angle. Similarly, to apply the virtual zones of the reference three-dimensional virtual model to the at least one part image 170, the zoning module 200 aligns the part features (stored as a set of part type feature points 171) of the at least one part image 170 with the reference points of the reference three-dimensional virtual model such that the at least one part image 170 is aligned with the reference three-dimensional virtual model. The positional metadata of the at least one part image from each view angle of the at least one view angle facilitates the alignment with the three-dimensional virtual model.

[0055] After the at least one part image 170 is properly aligned with the at least one reference image 172 (or reference three-dimensional virtual model), the zoning module 200 projects each defect in the at least one part image 170 onto the resulting composite image 199 (or resulting three-dimensional virtual model). In some embodiments, each defect is superimposed within a zone in which the defect is located. In some embodiments, the zone is back-projected from the at least one reference image 172 (or reference three-dimensional virtual model) onto the at least one reference image 172, resulting in the composite image 199.

[0056] 6A-6C are flow diagrams of an exemplary method 300 for zone-guided repair coverage optimization of a part. The method 300 includes providing (302) images captured by an image capture system (e.g., image capture device 150) oriented to capture at least one part image 170 of the part (154, 250) from at least one view angle, the at least one part image 170 including a set of features 171 selected such that features of the part and features of defects 274 on a surface of the part are identifiable. The method further includes classifying (304) types of defects 274 detected on a surface 155 of the part 154 in the at least one part image 170 and assigning (306) zones (e.g., zones 280B, 280C, 280E) to the surface 155 of the part 154 captured in the at least one part image 170.

[0057] In some embodiments, the method 300 further includes assigning (308) a zone priority to each of the plurality of zones (280B, 280C, 280E); the relative fragility and robustness of the zones; Difficulty of repairing the zone, the time and cost to repair the zone defects 274; one or more types of defects 274 in each of a plurality of zones (280B, 280C, 280E); a method for repairing one or more defects 274 of said type in each zone of a plurality of zones (280B, 280C, 280E); and The material required to repair one or more defects 274 in each of the multiple zones (280B, 280C, 280E). determining (310) a zone priority 310 for each of a plurality of zones (280B, 280C, 280E) according to empirical data including at least one of: Includes.

[0058] In some embodiments, the method 300 further includes identifying (312) part features found in the at least one part image 170 and aligning (314) a set of features of the at least one part image 170 with a corresponding set of surface features of one of the three-dimensional virtual model 202 and the at least one reference image.

[0059] In some embodiments, the method 300 further includes running (316) the neural network to identify a set of defect features for each defect 274 on the surface 155 of the at least one part image 170. In some embodiments, the neural network is trained using a machine learning algorithm.

[0060] In some embodiments, the method 300 further includes identifying (318) each defect 274 based on at least one of a type of defect, a physical dimension of the defect, whether the defect can be repaired, whether the defect can be inspected, and whether the zone (280B, 280C, 280E) in which the defect 274 is located is a hazardous condition. In some embodiments, the method 300 further includes assigning (320) a weight to each zone in which the defect 274 is identified, the weight being assigned according to a zone priority of each zone (280B, 280C, 280E). In some embodiments, the method 300 further includes projecting (322) each defect 274 of the at least one part image 170 onto at least one of the three-dimensional virtual model and the at least one part image 170, and overlaying (324) each defect 274 within the zone (280B, 280C, 280E) in which the defect 274 is located.

[0061] In some embodiments, the method 300 further includes displaying (326) the at least one zone (280B, 280C, 280E) and the defect 274 of at least one of the three-dimensional virtual model and the at least one part image 170 on the output device 124, and in some embodiments, the method 300 further includes outputting (328) at least one of a data table regarding an estimated cost to repair the part, an estimate of the time it will take to repair the part, whether the part can be inspected and whether it can be repaired, how to repair the part, where the part is to be repaired, routing information, the three-dimensional virtual model with color-coded zones (280B, 280C, 280E) having the defect 274, and an image in which the zones (280B, 280C, 280E) having the defect 274 are color-coded.

[0062] The described systems and methods provide an inspection system that can optimize the inspection of parts using empirical test data. The described systems and methods can facilitate improving repair times and costs by leveraging prior knowledge of the time and cost to repair a given zone. This is particularly important because different zones may have the same type and size of defects that result in different repair times and / or costs.

[0063] The inspection results can be used by the inspection facility to estimate the number of scrapped parts and inventory that will need to be replaced or repaired for damaged parts. Depending on the type, size and location of the defect, different repair methods can be adopted so that the repair process is improved. The inspection system overlays each defect within a defective zone and displays the zone and the defect as a user output. The inspection system is configured to output at least one of a data table regarding an estimated cost to repair the part, an estimate of the time to repair the part, whether the part can be inspected and repaired, how to repair the part, where the part is repaired, routing information, a three-dimensional virtual model with color-coded zones with defects, and color-coded images of the zones with defects. The described system and method also standardize the inspection, which increases accuracy and reduces costs.

[0064] It should be understood that the above description is intended to be illustrative, and not limiting. For example, the above-described embodiments (and / or aspects thereof) can be used in combination with each other. In addition, many modifications can be made to adapt a particular situation or material to the teachings of the subject matter described herein without departing from the scope of the subject matter. While the dimensions and types of materials described herein are intended to define the parameters of the disclosed subject matter, the dimensions and types of materials described herein are by no means limiting, but are exemplary embodiments. Many other embodiments will be apparent to those skilled in the art upon review of the above description. Thus, the scope of the subject matter described herein should be determined with reference to the claims, along with the full scope of equivalents which are encompassed by such claims. In the claims, the terms "including" and "in which" are used as the plain English equivalents of the respective terms "comprising" and "wherein". Furthermore, in the claims, terms such as "first", "second" and "third" are used merely as labels and are not intended to impose numerical requirements on their subject matter. Moreover, no claim limitation is written in means-plus-function form and is not intended to be construed under 35 U.S.C. § 112(f), unless the claim limitation uses a description of a function without further structure following the phrase “means for.”

[0065] This specification uses examples to disclose certain embodiments of the subject matter described herein, including the best mode, and also to enable any person skilled in the art to practice the disclosed subject matter, including making and using the devices or systems and performing the methods. The patentable scope of the subject matter described herein is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if the other examples have structural elements that do not differ from the literal language of the claims, or if the other examples include equivalent structural elements that are not substantially different from the literal language of the claims.

[0066] Furthermore, the disclosed subject matter may be embodied in the form of computer-implemented processes and apparatuses for carrying out those processes. The present disclosure may also be embodied in the form of computer program code, including instructions embodied in a tangible medium (such as a floppy disk, a CD-ROM, a hard drive, or any other computer-readable storage medium), which, when loaded into a computer and executed by the computer, makes the computer an apparatus for carrying out the present disclosure. The present disclosure may also be embodied in the form of computer program code, which may, for example, be stored in a storage medium, loaded into a computer, and / or executed by a computer, or transmitted as a data signal over some transmission medium, whether or not a modulated carrier wave (e.g., by electrical wire or cable, through optical fiber, or by electromagnetic radiation). When the computer program code is loaded into a computer and executed by the computer, the computer becomes an apparatus for carrying out the present disclosure. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.

[0067] The methods, systems, and compositions disclosed herein are not limited to the specific embodiments described herein, and the method steps, system elements, and / or composition elements may be utilized separately and independently from other steps and / or other elements described herein. For example, the methods, systems, and compositions are not limited to practice with only the rotating machines described herein. The methods, systems, and compositions may be implemented and utilized in many other applications.

[0068] Although specific features of various embodiments may be shown in some drawings and not in others, this is for convenience only. Moreover, references to "one embodiment" in the above description are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the referenced features. In accordance with the basic concept of the present disclosure, any feature of a drawing may be referenced and / or claimed in combination with any feature of another drawing.

[0069] This specification uses examples, including the best mode, to enable any person skilled in the art to practice the present disclosure, including making and using any devices or systems and performing the incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements that are not substantially different from the literal language of the claims.

[0070] Other aspects of the invention are provided by the following embodiments. [Embodiment 1] 1. An inspection system comprising: an image capture system oriented to capture at least one component image of a component from at least one view angle, the at least one component image including a set of features selected to identify features of the component and features of defects on a surface of the component; and a processor configured to classify a type of defect detected on the surface of the component in the at least one part image and to assign a number of zones to the surface of the component captured in the at least one part image. [Embodiment 2] 2. The inspection system of embodiment 1, wherein the processor assigns a zone priority to each of the plurality of zones using a zone priority data matrix stored in memory. [Embodiment 3] An inspection system as described in embodiment 1 or 2, wherein the memory is configured to store a zone priority data matrix, the zone priority data matrix being a data matrix in which zone priority is determined by empirical data including at least one of relative brittleness and robustness of the zones, difficulty of repairing the zones, time and cost to repair defects in the zones, types of one or more defects in each of the plurality of zones, a method of repairing one or more defects of said types in each of the plurality of zones, and materials required to repair one or more defects in each of the plurality of zones. [Embodiment 4] The inspection system of any one of embodiments 1 to 3, wherein the processor is further configured to identify component features found in the at least one component image. [Embodiment 5] An inspection system as described in any one of embodiments 1 to 4, wherein the memory is configured to store at least one of a three-dimensional virtual model of a part having zones mapped to a surface of the part and at least one reference image of a part having zones mapped to a surface of the part. [Embodiment 6] An inspection system as described in any one of embodiments 1 to 5, wherein the processor is configured to perform alignment of a set of features of the at least one part image with a corresponding set of surface features of one of the three-dimensional virtual model and the at least one reference image. [Embodiment 7] An inspection system as described in any one of embodiments 1 to 6, wherein the processor is configured to execute a neural network to identify a set of defect features for each defect on the surface of the at least one part image. [Embodiment 8] The inspection system according to any one of the first to seventh embodiments, wherein the neural network is a segmentation neural network. [Embodiment 9] The inspection system according to any one of the preceding embodiments, wherein the neural network is trained using a machine learning algorithm. [Embodiment 10] The inspection system of any one of embodiments 1 to 9, wherein the processor is further configured to identify each defect based on at least one of the following: type of defect, physical dimensions of the defect, whether the defect can be repaired, whether the defect can be inspected, and whether a zone containing the defect is in a dangerous state. [Embodiment 11] An inspection system as described in any one of embodiments 1 to 10, wherein the processor is further configured to assign a weight to each zone in which a defect is identified, the weight being assigned according to a zone priority of each zone. [Embodiment 12] An inspection system described in any one of embodiments 1 to 11, wherein the processor is configured to perform projecting each defect in the at least one part image onto at least one of a three-dimensional virtual model and the at least one part image. [Embodiment 13] An inspection system as described in any one of embodiments 1 to 12, wherein the processor is configured to perform overlapping of each defect within a defective zone. [Embodiment 14] An inspection system as described in any one of embodiments 1 to 13, wherein at least one zone and defect of at least one of the three-dimensional virtual model and the at least one part image are stored in a memory and displayed on an output device. [Embodiment 15] The inspection system of any one of embodiments 1 to 14, wherein the processor is configured to output at least one of a data table regarding an estimated cost to repair the part, an estimate of a time to repair the part, whether the part can be inspected and repaired, how to repair the part, where the part is to be repaired, routing information, a three-dimensional virtual model with color-coded zones having defects, and a color-coded image of the zones having defects. [Embodiment 16] 1. A method for zone-guided repair coverage optimization of a component, comprising: 1. A method comprising: providing images captured by an image capture system oriented to capture at least one component image of a component from at least one view angle, the at least one component image including a set of features selected to identify features of the component and features of defects on a surface of the component; classifying a type of defect detected on the surface of the component in the at least one component image; and assigning a zone to the surface of the component captured in the at least one component image. [Embodiment 17] 17. The method of any one of embodiments 1-16, further comprising assigning a zone priority to each of the plurality of zones. [Embodiment 18] 18. The method of claim 16 or 17, further comprising determining zone priority according to empirical data including at least one of the relative brittleness and robustness of the zones, the difficulty of repairing the zones, the time and cost to repair defects in the zones, the type or types of defects in each of the plurality of zones, the method of repairing the type or types of defects in each of the plurality of zones, and the material required to repair the one or more defects in each of the plurality of zones. [Embodiment 19] The method of any one of embodiments 16 to 18, further comprising identifying part features found in the at least one part image. [Embodiment 20] The method according to any one of embodiments 16 to 19, further comprising aligning a set of features of the at least one part image with a corresponding set of surface features of one of the three-dimensional virtual model and the at least one reference image. [Embodiment 21] The method according to any one of embodiments 16 to 20, wherein the neural network is a segmentation neural network. [Embodiment 22] 22. The method of any one of embodiments 16-21, further comprising running a neural network by a processor to identify a set of defect features for each defect on a surface of the at least one part image. [Embodiment 23] The method of any one of embodiments 16 to 22, further comprising training the neural network using a machine learning algorithm. [Embodiment 24] Identifying each defect based on at least one of the following: type of defect, physical size of the defect, whether the defect can be repaired, whether the defect can be inspected, and whether the defective zone is a hazardous condition. The method according to any one of embodiments 16 to 23, further comprising: [Embodiment 25] 25. The method according to any one of embodiments 16 to 24, further comprising assigning a weight to each zone in which defects are identified, the weight being assigned according to a zone priority of each zone. [Embodiment 26] The method according to any one of embodiments 16 to 25, further comprising projecting each defect in the at least one part image onto at least one of a three-dimensional virtual model and the at least one part image. [Embodiment 27] 27. The method according to any one of embodiments 16-26, further comprising overlapping each defect within the defective zone. [Embodiment 28] The method of any one of embodiments 16-27, further comprising displaying at least one zone and defect of at least one of the three-dimensional virtual model and the at least one part image on an output device. [Embodiment 29] The method according to any of the embodiments 16 to 28, configured to output at least one of a data table regarding an estimated cost to repair the part, an estimate of a time to repair the part, whether the part can be inspected and repaired, how to repair the part, where the part is to be repaired, routing information, a three-dimensional virtual model with color-coded zones having defects, and a color-coded image of the zones having defects. [Explanation of symbols]

[0071] 100 Inspection Systems 102 users 110 Equipment 120 processors 122 Memory 124 Media Output Component 126 Input Devices 130 Communication Interface 150 Imaging device 151 Views 152 Platform 155 Surface 160 Positioning device 170 Parts Images 171 feature points 172 Reference Images 180 Classification Module 190 Databases 191 Part Type Data Matrix 192 Defect type data matrix 193 Zone Weights Data Matrix 199 Composite Images 200 Zoning Module 202 3D Virtual Model 250 Airfoil Parts 252 Leading Edge 254 Trailing edge 256 Tip 258 Proximal end 260 surface 270 Chipping 272 Stress fracture 274 Defects 280 Zone 284 Line 300 ways

Claims

1. An inspection system (100), comprising: an image capture system (150) oriented to capture at least one part image (170) of a part (250) from at least one view angle, the at least one part image (170) including a set of features (171) selected to identify features of the part (250) and features of defects (274) on a surface (260) of the part (250); A processor (120), classifying a type of defect (274) detected on a surface (260) of the part (250) in the at least one part image (170); and assigning (308) a plurality of zones (280) to a surface (260) of a part (250) captured in the at least one part image (170); a processor configured to execute An inspection system (100) comprising:

2. 2. The inspection system of claim 1, wherein the processor assigns a zone priority to each zone of the plurality of zones using a zone priority data matrix stored in a memory.

3. 3. The inspection system of claim 2, wherein the memory (122) is configured to store a zone priority data matrix, the zone priority data matrix being a data matrix in which zone priority is determined by empirical data including at least one of relative brittleness and robustness of the zones (280), difficulty of repairing the zones (280), time and cost to repair defects in the zones (280), types of one or more defects in each of the plurality of zones (280), a method of repairing one or more defects (274) of the types in each of the plurality of zones (280), and materials required to repair one or more defects in each of the plurality of zones (280).

4. The inspection system (100) of claim 1, wherein the processor (120) is further configured to identify part features found in the at least one part image (170).

5. 5. The inspection system of claim 4, wherein the memory is configured to store at least one of a three-dimensional virtual model (172) of a part (250) having zones (280) mapped to a surface (260) of the part (250) and at least one reference image (172) of the part (250) having zones (280) mapped to a surface (260) of the part (260).

6. The processor, aligning (314) a set of features of the at least one part image (170) with a corresponding set of features of a surface (260) of the three-dimensional virtual model (172) and the at least one reference image (172); running (316) a neural network to identify a set of defect features (171) for each defect on the surface (260) of the at least one part image (170); and Identifying each defect based on at least one of the following: a type of defect, a physical size of the defect, whether the defect can be repaired, whether the defect can be inspected, and whether the defective zone is a hazardous condition. The inspection system of claim 5 configured to perform the following:

7. The processor (100) further comprises: assigning (320) a weight to each zone (280) in which defects are identified, the weight being assigned according to a zone priority of each zone (280); projecting (322) each defect in the at least one part image (170) onto at least one of a three-dimensional virtual model (172) and the at least one part image (172); and Overlapping each defect within the defect zone (324) The inspection system of claim 1 configured to perform:

8. 1. A method for zone-guided repair coverage optimization of a part (250), comprising: providing (302) an image (170) captured by an image capture system (150) oriented to capture at least one part image (170) of the part (250) from at least one view angle, the at least one part image (170) including a set of features (171) selected such that features of the part (250) and features of defects (274) on a surface (260) of the part (250) are identifiable; classifying (304) a type of defect (274) detected on a surface (260) of the part (250) in the at least one part image (170); and assigning (308) zones (280) to surfaces (260) of a part (250) captured in the at least one part image (170); A method comprising:

9. assigning (308) a zone priority to each zone of the plurality of zones (280); determining (310) zone priorities according to empirical data including at least one of the relative brittleness and robustness of the zones (280), the difficulty of repairing the zones (280), the time and cost to repair defects in the zones (280), a type of one or more defects (274) in each of the plurality of zones (280), a method of repairing one or more defects (274) of said type in each of the plurality of zones (280), and materials required to repair the one or more defects (274) in each of the plurality of zones (280); and Identifying (312) part features (171) found in the at least one part image (170). The method of claim 8 further comprising:

10. aligning (314) a set of features (171) of the at least one part image (170) with a corresponding set of features of a three-dimensional virtual model (172) and one of the surfaces (155) of the at least one reference image (172); executing (316) the neural network by a processor to identify, for each defect (274) on the surface (155) of the at least one part image (172), a set of defect features; Identifying (318) each defect (274) based on at least one of the following: a type of defect, a physical dimension of the defect (264), whether the defect (274) can be repaired, whether the defect (274) can be inspected, and whether a zone (280) in which the defect (274) is located is unsafe. The method of claim 9 further comprising:

Images