A system and method for determining the predicted lifetime of components using a machine learning model.

Abstract

This invention provides a computer implementation method and system for determining the predicted lifespan of a component. [Solution] The method includes acquiring images of a component by a computing system comprising one or more computing devices. The method further includes acquiring a stress map of the component by the computing system. The method further includes processing the images and stress map of the component using a machine learning model by the computing system to generate a predicted lifetime of the component as the output of the machine learning model.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for determining the predicted life of components by utilizing artificial intelligence and / or machine learning.

Background Art

[0002] Throughout various applications, generally, a consistent and accurate prediction of the life of components is desired. Such predictions can enhance efficiency by reducing damage due to component failure and enabling improved planning for taking components out of service.

[0003] One application where such consistent and accurate predictions are desired is in applications where components are subjected to a number of extreme conditions (e.g., high temperature, high pressure, large stress loads, etc.). Over time, the individual components of a device can undergo creep, deformation, fatigue cracks, etc., which can shorten the useful life of the components. Such concerns can apply to some turbomachines, such as gas turbine systems. During operation of a turbomachine, various components within the turbomachine, particularly within the turbine section of the turbomachine, such as turbine blades (collectively known as turbine components), can be subjected to creep due to high temperature and stress. In the case of turbine blades, creep can cause part or all of the blade to elongate, causing the blade tip to contact a fixed structure, such as a turbine casing, which can cause undesirable vibrations and / or performance degradation during operation. Additionally, excessive creep can cause creep rupture and resulting component failure, which can lead to unplanned shutdowns and damage to other components within the system.

[0004] Therefore, components such as turbine components can be monitored for creep. One method for monitoring components for creep is to configure strain sensors on the components and analyze the strain sensors at various intervals to monitor deformations associated with creep strain. This positioning is time-consuming and costly, and therefore can lead to inefficiencies in the deformation monitoring process.

[0005] Therefore, improved systems and methods for predicting the lifespan of components are desired. For example, systems and methods that can consistently and accurately predict creep and thus enable predictive life planning based on such predictions would be advantageous. [Overview of the project]

[0006] The aspects and advantages of the systems and methods described herein are partially described in the following description, may become apparent from the description, or can be learned through practice of the technology.

[0007] According to one embodiment, a computer implementation method for determining the predicted lifetime of a component is provided. The method includes acquiring an image of the component by a computing system comprising one or more computing devices. The method further includes acquiring a stress map of the component by the computing system. The method further includes processing the image and stress map of the component using a machine learning model by the computing system to generate a predicted lifetime of the component as the output of the machine learning model.

[0008] According to another embodiment, a computing system is provided. The computing system includes one or more processors. The computing system further includes one or more non-temporary computer-readable media that collectively store instructions that, when executed by one or more processors, cause the computing system to perform an action, the action includes obtaining an image of a component. The action further includes obtaining a stress map of the component. The action further includes processing the image and stress map of the component using a machine-learned model to generate a predicted lifetime of the component as the output of the machine-learned model.

[0009] These and other features, aspects, and advantages of the system and method will be better understood by referring to the following description and the appended claims. The appended drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments of the art and, together with the description in the specification, are useful in illustrating the principles of the art.

[0010] A complete and implementable disclosure of the System and Method, including the best mode of construction and use of the System and Method directed to those skilled in the art, is described herein with reference to the attached figures. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a system for acquiring images of components according to an embodiment of the present disclosure. [Figure 2] This is a process flowchart for generating the predicted lifetime of a component according to an embodiment of the present disclosure. [Figure 3] This figure shows an example of applying machine learning and / or computer vision to enhance an image according to embodiments of the present disclosure. [Figure 4] This figure shows an example of applying machine learning and / or computer vision to correct an image according to an embodiment of the present disclosure. [Figure 5]This figure shows a video of the components according to an embodiment of the present disclosure. [Figure 6] This figure shows multiple images of the components according to an embodiment of the present disclosure. [Figure 7] This figure shows the processing of constituent images by cropping an image according to an embodiment of the present disclosure. [Figure 8] This figure shows the processing of a component image by at least one of pixel analysis, computer vision, and / or a machine learning model, according to embodiments of the present disclosure. [Figure 9] This is a schematic diagram of the detected crystal grain structure on a component according to an embodiment of the present disclosure. [Figure 10(A)] This figure shows a comparison between the detected grain structure and the stress map of the constituent elements according to an embodiment of the present disclosure. [Figure 10(B)] This figure shows a comparison between the detected grain structure and the stress map of the constituent elements according to an embodiment of the present disclosure. [Figure 10(C)] This figure shows a comparison between the detected grain structure and the stress map of the constituent elements according to an embodiment of the present disclosure. [Figure 10(D)] This figure shows a comparison between the detected grain structure and the stress map of the constituent elements according to an embodiment of the present disclosure. [Figure 11] This figure shows a computing system according to an embodiment of the present disclosure. [Figure 12] This flowchart shows a method for determining the predicted lifetime of a component according to embodiments of the present disclosure. [Modes for carrying out the invention]

[0012] Herein, we refer in detail to embodiments of the System and Method, one or more of which are shown in the drawings. Each example is provided for illustrative purposes of the Art and is not intended to limit the Art. Indeed, it will be apparent to those skilled in the art that modifications and changes can be made in the Art without departing from the scope or spirit of the claimed Art. For example, features illustrated or described as part of one embodiment can be used in conjunction with another embodiment to create further embodiments. Accordingly, this disclosure is intended to include modifications and changes that fall within the scope of the appended claims and their equivalents.

[0013] The term “exemplary” is used herein to mean “serving as an example, case, or illustration.” Not all implementations described herein as “exemplary” should be construed as necessarily preferable or advantageous to other implementations. In addition, unless otherwise specified, all embodiments described herein should be considered exemplary.

[0014] In the detailed description, numerals and letters are used to refer to features of the drawings. Similar or identical reference numerals in the drawings and description are used to refer to similar or identical parts of the invention. Where used herein, the terms “first,” “second,” and “third” may be used interchangeably to distinguish one component from another and are not intended to indicate the location or importance of any individual component.

[0015] As used herein, the terms "upstream" (or "forward") and "downstream" (or "rearward") refer to the relative directions with respect to the flow of fluid in a fluid path. For example, "upstream" refers to the direction from which the fluid has flowed, and "downstream" refers to the direction in which the fluid is flowing. However, the terms "upstream" and "downstream" as used herein may also refer to the flow of electricity. The term "radially" refers to a relative direction that is substantially perpendicular to the axial centerline of a particular component, the term "axially" refers to a relative direction that is substantially parallel to and / or coaxial with the axial centerline of a particular component, and the term "circumferentially" refers to a relative direction that extends around the axial centerline of a particular component.

[0016] Approximating terms such as "about", "approximately", "substantially", and "essentially" are not limited to the exact values specified. In at least some instances, the language indicating approximation may correspond to the accuracy of the equipment used to measure the value, or the accuracy of the method or machine used to construct or manufacture the component and / or system. In at least some instances, the language indicating approximation may correspond to the accuracy of the equipment used to measure the value, or the accuracy of the method or machine used to construct or manufacture the component and / or system. For example, the language indicating approximation may refer to being within a margin of 1, 2, 4, 5, 10, 15, or 20% at any of the individual values, ranges of values, and / or endpoints defining the ranges of values. When used in the context of an angle or direction, such terms include the range of plus or minus 10 degrees of the described angle or direction. For example, "substantially perpendicular" includes directions within 10 degrees in any direction, such as clockwise or counterclockwise, from perpendicular.

[0017] Terms such as “combined,” “fixed,” and “attached” refer to both direct combination, fixation, or attachment, as well as indirect combination, fixation, or attachment via one or more intermediate components or features, unless otherwise specified herein. Where used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” and “having,” or any other variations thereof, are intended to encompass non-exclusive inclusion. For example, a process, method, article, or apparatus that includes an enumeration of features is not necessarily limited to those features alone, but may include other features not expressly enumerated, or other features inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated otherwise, “or” refers to a comprehensive OR and not an exclusive OR. For example, condition A or B is satisfied by one of the following: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); or both A and B are true (or exist).

[0018] Here, and throughout this specification and the claims, limitations on scope are combined and replaced, and unless the context and wording specifically indicate otherwise, such scope is identified and includes all sub-scopes contained therein. For example, all scopes disclosed herein include endpoints, which are independently combinable with respect to one another.

[0019] This disclosure relates in general to a method for predicting the lifetime of a component (e.g., the creep lifetime of a component). In particular, the component may be a component of a gas turbine engine, such as a compressor component (e.g., compressor blades or stator vanes) or a turbine component (e.g., turbine blades or fixed nozzles). The method may include two approaches. In the first approach (i.e., a manual approach), an image of the component (e.g., a photograph, scan, video) may be received or acquired by a computing system. The computing system may also receive a stress map of the component. The stress map may be acquired by finite element analysis (FEA) of the component. Based on the stress map and the image, the system may determine or identify a region of interest and crop the region of interest. The system may detect contours, filter contours representing true grains on the component, and overlay those grains with the stress map. Based on the comparison of the grains with the stress map, the computerized system may extract one or more features and / or make a prediction about the remaining creep lifetime of the component. Alternatively or additionally, in a second method, images and stress maps may be provided to a machine-learned model, which may then generate a prediction of the remaining creep lifetime based on the input. Referring here to Figure 1, a component 10 is provided. Component 10 (more specifically, the substrate of the entire component 10) may include various types of components used in various different applications, such as components used in high-temperature applications (e.g., components including nickel or cobalt-based superalloys, austenitic steel, etc.). In some embodiments, component 10 may include industrial gas turbine or steam turbine components, such as combustion components or high-temperature gas path components. In some embodiments, component 10 may include turbine blades, compressor blades, impellers, shrouds, rotors, or transition pieces. In other embodiments, component 10 may include any other components of a turbine, such as any other components for a gas turbine, steam turbine, etc.In some embodiments, the component may include non-turbine components including, but not limited to, automotive components (e.g., cars, trucks, etc.), aerospace components (e.g., airplanes, helicopters, space shuttles, aluminum parts, etc.), locomotive or railway components (e.g., trains, railway tracks, etc.), structural, infrastructure, or civil engineering components (e.g., bridges, buildings, construction machinery, etc.), and / or power generation devices or chemical processing components (e.g., pipes used in high-temperature applications).

[0020] In an exemplary embodiment, component 10 is an equiaxed or directionally solidified component. For example, component 10 can be a cast component, and after casting, the melt in the mold can advantageously become equiaxed or be directionally solidified.

[0021] In some embodiments, component 10 may be a new component (e.g., one that has never been installed in a machine or has never experienced an operating cycle). However, in an exemplary embodiment, component 10 may have experienced at least one operating cycle. For example, in an embodiment where component 10 is a turbine component, component 10 may have experienced at least one operating cycle within a gas turbine. That is, during operation of the gas turbine, the turbine component may be exposed to high-temperature combustion gases, operating vibrations, and mechanical forces that degrade, distort, and / or creep the turbine component over time. As used herein, an "operating cycle" can refer to one or more components that have spent a certain amount of time in operation within a machine (such as a gas turbine). Thus, a component that has not experienced an operating cycle has never been used in the operation of a machine (such as a gas turbine). An operating cycle for a gas turbine may be from about 0 to about 50,000 operating hours (or, for example, from about 500 to about 50,000 hours, or from about 1000 to about 50,000 hours, or from about 10,000 to about 50,000 hours).

[0022] Figure 1 further illustrates the coordinate system. The coordinate system includes the X-axis 50, Y-axis 52, and Z-axis 54, all of which are orthogonal to each other and defined relative to component 10.

[0023] Figure 1 further illustrates the computing system 100, which may include, for example, a data acquisition system 102 and a user computing system 104. The data acquisition system 102 generally collects data relating to the components 10, and the computing system 104 generally analyzes the data and performs various calculations and other functions as described herein. In particular, the computing system 100 according to this disclosure provides accurate and efficient prediction of the lifetime of the components 10, as described herein.

[0024] It should be noted that various subsystems within the computing system 100, such as the data acquisition system 102, the user computing system 104, and other appropriate subsystems, may be linked to one another as described herein, or they may be separate discrete systems.

[0025] According to one embodiment, the data acquisition system 102 may include an imaging device 106 for acquiring one or more images of the components 10. Such images may be discrete images (e.g., photographs) or in the form of a video containing multiple video frame images extracted from a video. For example, the imaging device 106 may include a lens assembly 110 and an image capture device 112. The lens assembly 110 can generally magnify the image seen by the lens assembly 110 for processing by the image capture device 112. In some embodiments, the lens assembly 110 may be, for example, a suitable camera lens, a telescope lens, etc., and may include one or more lenses spaced apart to provide the required magnification. The image capture device 112 may generally communicate with the lens assembly 110 to receive light from the lens assembly 110 and process it to generate an image. In an exemplary embodiment, for example, the image capture device 112 may be a camera sensor that receives and processes light from a camera lens to generate an image, such as a digital image, as is commonly understood. In some embodiments, the image capture device 112 (and device 106 as a whole) may further communicate with a computing system 104, for example, via a suitable wired or wireless connection, to store and analyze images from the image capture device 112 and device 106 as a whole. In some embodiments, the user computing system 104 may operate the imaging device 106 to perform various disclosed steps. In other embodiments, the imaging device 106 may be a standalone device operated separately by the user and may be linked to the user computing system 104 or may be a separate discrete system.

[0026] Additionally or alternatively, the data acquisition system 102 may further include a three-dimensional data acquisition device 108 for inspecting the outer surface 11 of the component 10. The device 108 according to this disclosure generally utilizes surface measurement techniques to acquire direct measurements of the component 10 along three axes. In particular, non-contact surface measurement techniques may be utilized in exemplary embodiments. Generally, any suitable three-dimensional data acquisition device 108 that utilizes surface measurement techniques to acquire direct measurements in three dimensions may be utilized (e.g., blue light scanning). In exemplary embodiments, the device 108 is a non-contact device that utilizes non-contact surface measurement techniques.

[0027] According to one embodiment, in some exemplary embodiments, device 108 is a laser scanner that generates a laser scanning image. The laser scanner generally includes a laser 120 that emits light in the form of a laser beam toward an object, such as the component 10 as a whole in these embodiments. The light is then detected by a sensor 124 of device 108. For example, in some embodiments, the light is reflected from a surface it contacts, and the light is received by the sensor 124 of device 108. The round-trip time for the light to reach the sensor 124 is used to determine measurements along various axes. These devices are commonly known as time-of-flight devices. In other embodiments, the sensor 124 detects light on a surface it contacts and determines measurements based on the relative position of the light within the field of view of the sensor 124. These devices are commonly known as triangulation devices. Data points for the X, Y, and Z axes are calculated based on the detected light, as described above.

[0028] In some embodiments, the light emitted by the laser 120 is emitted over a bandwidth broad enough to reflect off a portion of the object being measured. In these embodiments, a robotic arm (as described herein) or other suitable mechanism for moving the laser 120 may be used to move the laser 120 and the emission bandwidth as needed until the light is reflected off the entire object being measured.

[0029] In other embodiments, other suitable surface measurement devices may be used. For example, in some embodiments, device 108 may be an X-ray scanner that provides images in the form of X-rays. In some embodiments, device 108 may be a high-resolution crystal orientation system that provides images in the form of Laue diffraction patterns, such as Laue orientation images. In some embodiments, device 108 may be a scanning electron microscope that provides images in the form of electron channeling contrast images. In some embodiments, device 108 may be a three-dimensional scanner that provides images in the form of three-dimensional scanning shapes.

[0030] In some embodiments, the data acquisition system 102 may include a robotic arm 130. The robotic arm 130 may support and facilitate the movement of other components of the data acquisition system 102 relative to a component in order to acquire images of that component. For example, an imaging device 106 and a data acquisition device 108 (or its components, such as a light source) may be mounted on the robotic arm 130. Movement of the robotic arm 130 may, in exemplary embodiments, position the data acquisition system 102 or its components (such as a light source) relative to a component 10. In some embodiments, other components, such as the imaging device 106, may remain fixed, while components such as a light source are movable. In exemplary embodiments, the robotic arm 130 is a 6-degree-of-freedom arm 130 that provides movement along and around axes 50, 52, and 54.

[0031] In some embodiments, the user computing system 104 may operate the data acquisition system 102 to perform various disclosed steps. In other embodiments, the data acquisition system 102 may be a standalone device operated separately by the user and may be linked to the user computing system 104, or it may be a separate discrete system. For example, in some embodiments, the user may manually acquire images and upload those images to the user computing system 104.

[0032] Referring to Figure 2, a process flowchart for generating predicted lifetimes 522 of component 10 according to embodiments of the present disclosure is shown. Process 500 may be carried out by the computing system 100 described above (and further described below). In particular, process 500 shows two distinct paths for generating predicted component lifetimes 522, e.g., a multi-step path 501 and an end-to-end path 503. Both paths 501, 503 may be achieved by the computing system 100, but the end-to-end path 503 may utilize a machine-learned model 520 that streamlines the multi-step 501 (e.g., in an end-to-end method). Data utilized and / or generated by the multi-step 501 (e.g., input 502 and output predicted lifetimes 522) may be provided to teach or train the machine-learned model 520. In some embodiments, the machine-learned model 520 may be trained independently without any reference to the data generated via the multi-step path 501, e.g. Both paths 501 and 503 may be automated, but a machine learning model may replace some steps within the multi-step path 501. The machine learning model 520 may be one (or a combination thereof) of a multi-input neural network (e.g., a multimodal neural network), a CNN with tabular data fusion (an arbitrary extension of a convolutional neural network (CNN)), deep embedded clustering (DEC), autoencoders, residual networks (ResNet), and / or capsule networks (CapsNets). The machine learning model 520 may be capable of directly predicting creep lifetime based on a combination of tabular data (e.g., operating conditions), images, and / or stress maps.

[0033] Process 500 may include one or more inputs 502, including an image 504, a stress map 506, and / or motion data 508. As shown in the figure, image 504 may be at least one of photographs 510A, 510B, video 512, laser scan image, X-ray scan image, Laue orientation image, electron channeling contrast image, or three-dimensional scan shape 514. Photographs 510A, 510B may be at least one of photographs 510A of the outer surface of a component, or photograph 510B of the inner surface of a component. That is, photograph 510B may be a photograph of a metallurgical sample made of the same or similar material as the component in some embodiments. In other embodiments, photograph 510B may be a cross-section of another component. For example, if the component being analyzed by process 500 (e.g., to predict lifetime) is a gas turbine airfoil, one of the inputs 502 may be an image of a cross-section (or metallurgical sample) of another gas turbine airfoil.

[0034] Process 500 may include, in step 516, dividing the video 512 into one or more (or more) frames. That is, the video 512 may include multiple frames that can be divided and / or selected (e.g., by the computing system 100) to generate multiple images. Similarly, in step 518, an image of the three-dimensional scan shape 514 may be generated, for example, by the computing system 100. It should be noted that such an image may be provided by the data acquisition system 102 and the computing system 100 as a whole, or independently of the data acquisition system 102 and the computing system 100 as a whole. In many embodiments, process 500 may further include, in step 524, establishing a single "average" image. The average image may be generated from multiple frames generated from the video 512 in step 516 and / or from multiple images generated from the three-dimensional scan 514. Establishing the average image in step 524 may include collecting images, initializing the total image, summing the pixel values, calculating the average, converting the image format, and saving the average image. Establishing the average image may include extracting the average, median, minimum, maximum, weighted average, and any other aggregations of pixel intensity.

[0035] In many embodiments, process 500 may include, in step 526, representing the three-dimensional scan 514 of the component 10 as a collection (or a plurality of slices). Each slice in the plurality of slices may be an image showing a different cross-sectional portion of the scanned component. In this way, when the plurality of slices are stacked together, they collectively form the scanned component. The plurality of slices may be generated by the computing system 100, or they may be generated separately (e.g., by a separate computing system) and provided to the computing system 100 for analysis. In other embodiments, the three-dimensional scan may be a point cloud dataset generated by a lidar sensor. In other embodiments, the three-dimensional scan may be a wave-based detection dataset. In further embodiments, the three-dimensional scan may be an original CAD drawing.

[0036] The stress map 506 may be generated, for example, by a finite element analysis ("FEA") of the component 10, which may be performed, for example, by the computing system 100, or performed separately (e.g., via a separate computing system) and provided to the computing system 100. Examples of FEA software suitable for such analysis include, for example, ANSYS, Simulia, and Nastran. The stress map 506 may also show high-stress and low-stress regions within the component 10, which may be based at least in part on operating data associated with the component 10 (such as operating time, the average temperature to which the component was exposed, or other operating data).

[0037] The operation data 508 may also relate to the component 10 and may include time data and temperature data. The time data may indicate the amount of time the component 10 spent operating. For example, in an embodiment where the component is a gas turbine component, the time data may indicate the amount of time the gas turbine component spent inside the operating gas turbine. The temperature data may indicate multiple temperatures (or average temperatures) to which the component was exposed during operation. For example, in an embodiment where the component is a gas turbine component, the temperature data may be the temperature to which the gas turbine component was exposed during operation, and this temperature may be multiple temperatures over a period of time or an average temperature. In many embodiments, the temperature data may describe a temperature range, a temperature delta over operating time, and / or a set of temperature deltas at a given interval to identify rapid temperature changes.

[0038] In many embodiments, process 500 (e.g., a multi-step path 501) may include, in step 527, identifying a region of interest of component 10 based on a stress map 506 of component 10. For example, the stress map 506 may indicate locations on component 10 that experience increased stress relative to other locations. These areas with increased stress may be more susceptible to material creep. In various implementations, process 500 may further include, in step 528, cropping the region of interest identified based on the stress map 506. For example, the region of interest shown in image 504, identified by the stress map 506, may be cropped (e.g., by a computing system 100). Cropping image 504 may include removing portions of image 504 that are not needed for further processing (e.g., regions of component 10 shown in image 504 that do not correspond to the areas with increased stress identified by the stress map 506). The cropped image may, in many implementations, be provided to a machine learning model 520. The cropped image may depend on the region indicated by the stress map 506, which can be generated by the finite element analysis (FEA) system.

[0039] In various embodiments, process 500 may further include, in step 530, applying computer vision (CV) and machine learning (ML) techniques to enhance, modify, and / or transform the image 504. For example, Figures 3 and 4 illustrate examples of how ML and CV may be used (e.g., by computing system 100) to enhance or modify an image. In Figure 3, a blurred image 600 may be processed by computing system 100 using CV and / or ML to produce an enhanced image 602. In particular, computing system 100 may include a machine learning model configured to receive the blurred image 600 and produce the enhanced image 602. That is, the computing system may process the blurred image 600 using a machine learning model and / or computer vision to produce the enhanced image 602. The enhanced image 602 may be sharper than the blurred image 600 and may have clearer lines (e.g., by more pixels). In Figure 4, the grayscale image 700 can be processed by the computing system 100 using CV and / or ML to generate a black and white image 702 representing the grain boundaries. Referring again to Figure 2, the enhanced image 602 and / or black and white image 702 can be provided to a machine learning model 520 in many implementations.

[0040] Examples of ML models that may be used by the computing system 100 to enhance or modify images include, but are not limited to, convolutional neural networks (CNNs), deep image priors (DIPs), autoencoders and denoising autoencoders (DAEs), generative adversarial networks (GANs), image super-resolution models, recurrent neural networks (RNNs), and long short-term memory (LSTMs). Examples of CV techniques that may be used by the computing system 100 to enhance or modify images include, but are not limited to, histogram equalization and adaptive histogram equalization, image binarization, smoothing methods such as Gaussian smoothing, unsharp masking, sharpening filters, median filtering, and blob analysis and removal. An example of computer vision software suitable for such analysis is, for example, the OpenCV Python library. The OpenCV Python library includes a wide range of computer vision techniques, including image processing, video analysis, object detection, feature detection, and matching.

[0041] Histogram equalization may be used to redistribute the intensity values ​​of pixels, resulting in pixels covering the full range of possible values, thereby improving the visibility of features in the image. Image binarization may be used to convert a grayscale or color image into a binary image that can consist only of black and white. Gaussian smoothing may involve convolving the image with a Gaussian function, which generates a weighted average of the surrounding pixel values, resulting in a smoother image. Blob analysis and removal may be used to remove "noise" or "blobs" from an image, thereby making the image clearer.

[0042] Referring again to Figure 2, in many embodiments, process 500 may further include detecting the contour of image 504 in step 532 and filtering that contour to represent the true crystal grains of component 10 in step 534. Furthermore, in step 536, process 500 may include overlaying (or superimposing) the crystal grains with the stress map 506. In step 538, process 500 may further include extracting crystal grain properties (e.g., feature extraction). Steps 532 to 538 are described in more detail below. Based at least in part on the relationship between the crystal grains identified in step 536 and the stress map, process 500 may include determining the predicted lifetime 522 of component 10. In many embodiments, process 500 may include determining the predicted lifetime 522 of the component based at least in part on temperature data and exposure time data (e.g., time exposed to peak temperature).

[0043] In step 538, feature extraction may involve extracting features associated with individual crystal grains, or it may represent a set of crystal grains of component 10 (e.g., number, mean, median, maximum, minimum, etc.). For example, features may include the following properties of the crystal grains that can be extracted from an image of component 10: namely, the relative orientation of stress with respect to the grain boundary at a given point, the number of crystal grains, region, perimeter, centroid, bounding box, aspect ratio, range, solidity, equivalent diameter, orientation, major axis length, minor axis length, eccentricity, convex hull, convex defects, moment, and contour length.

[0044] In an exemplary embodiment, process 500 (e.g., end-to-end path 503) may include a machine learning model 520 which may be stored in the computing system 100 and used by the computing system 100 to generate a predicted lifetime 522 of the component 10. That is, the input 502 may be processed by the computing system 100 using the machine learning model 520 to generate a predicted lifetime 522 of the component 10 as the output of the machine learning model 520.

[0045] As can be understood, the predicted lifetimes 522 generated using the multistep path 501 can be used to train a machine learning model 520. Thus, the predicted lifetimes 522 generated using the multistep path 501 can be multistep predicted lifetimes 522 for the purpose of training a machine learning model 520. Alternatively, the machine learning model 520 may be trained on manually labeled samples (e.g., images of manually labeled components, and / or manually labeled images of components).

[0046] The predicted lifetime 522 can be at least one of a binary output 540, an integer output 542, and / or a probability density function 544. The binary output 540 can be a single output (e.g., yes or no) indicating whether component 10 can survive after another operating cycle. The integer output 542 can be the number of operating cycles in which component 10 can survive (e.g., one more cycle, two more cycles, three more cycles, etc.). The probability density function 544 can indicate the amount of time in which component 10 can survive during operation. More specifically, the probability density function 544 can describe the probability that a continuous random variable takes a particular value. In particular, a cumulative distribution function (CDF) describing the probability that a random variable (X) takes a value less than or equal to a particular value (x) can be output as the predicted lifetime 522. For example, the CDF as the predicted lifetime 522 can indicate the probability (e.g., as a percentage) of reaching time X.

[0047] Machine learning models that may be applicable to generate a predicted lifetime of 522 include, but are not limited to, parametric survival models such as logistic regression and general log-linear models (GLL), decision trees, random forests, gradient boosting machines (GBM), XGBoost (extreme gradient boosting), LightGBM (light gradient boosting machine), CatBoost, support vector machines (SVM), K-nearest neighbors (KNN), naive Bayes, generalized linear models, and / or artificial neural networks (ANN).

[0048] In many embodiments, process 500 may include an optional step 546 of evaluation by a subject expert (SME). During step 546, the SME may use stress maps to evaluate the predicted lifetime, components, and superimposed grains to ensure that accurate predictions have been made. The SME may also make decisions based on the data without automated prediction tools.

[0049] Figure 5 shows an exemplary embodiment of multiple images 504 in the form of a video containing multiple video frames in this embodiment. Figure 6 shows multiple images 504, which may be multiple video frames from Figure 5, multiple photographs, or multiple images obtained from another suitable embodiment of the data acquisition system 102 described above. Note that such images may be provided by the data acquisition system 102 and the computing system 100 as a whole, or independently of the data acquisition system 102 and the computing system 100 as a whole.

[0050] A computing system 100, such as the data acquisition system 102 and / or its user computing system 104, may be capable of processing one or more images 504 of the component 10. Such processing may detect one or more crystal grain structures on the component 10.

[0051] The grain structure according to this disclosure may be one or more of the following: grains, grain boundaries, grain or grain boundary size (e.g., length), grain shape factor, minimum, maximum, median, aggregate grain size, grain or grain boundary orientation, and / or grain boundary triple points.

[0052] Computer vision may be used to perform cropping and / or pixel analysis of image 504. For example, in some embodiments, such processing may include cropping one or more images 504. Cropping allows focusing on a particular region of interest on component 10, such as a portion of the component that is particularly susceptible to high temperatures and high stresses. Computer vision and cropping may also be used to remove unwanted backgrounds that would otherwise consume processing power. Figure 7 shows image 504 with its cropped portion 202.

[0053] Additionally or alternatively, in some embodiments, such processing may include performing computer vision techniques and / or pixel analysis of image 504. This analysis generally distinguishes a reference object (e.g., a crystal grain structure) from a background (e.g., a component surface and the background) based on differences in color depth (i.e., differences in color or grayscale). The analysis may be performed for each individual pixel or group of pixels that define image 504. For pixel analysis to be performed, the number of pixels per bit of the image, i.e., 128, 256, etc., may be divided into, for example, two or more groups (e.g., a group with a brighter color depth and a group with a darker color depth). Each group is classified as either a reference object portion or a background portion. For example, color depth analysis may classify pixels or multi-pixel groups with a darker or brighter color depth as representing a reference object (i.e., a surface feature relative to a component, or a component relative to a background), and pixels or multi-pixel groups with the other of the darker or brighter color depths as representing the background (i.e., a component relative to a surface feature, or a background relative to a component). In particular, different divisions into lighter and darker groups may be used to distinguish surface features from their constituent elements and to distinguish the constituent elements from the background.

[0054] Figure 8 shows one embodiment of pixel analysis in which the image contrast is gradually improved through pixel analysis so that the crystal grain structure can be detected.

[0055] Figure 9 shows multiple detected grain structures 210 on the component 10. In particular, the detected grain structures may be detected relative to axes 50, 52, and 54 in exemplary embodiments. Understanding the localization (e.g., orientation, position, and / or shape) of the detected grain structures is important for creep and lifetime prediction according to this disclosure.

[0056] The computing system 100, such as the data acquisition system 102 and / or its user computing system 104, may further enable the comparison of the detected grain structure 210 with a stress map 220 of the component 10. This advantageously facilitates the correlation between structural details and directional stresses in specific regions. The stress map 220 may include maps of stress directions, such as contour maps of various stress tensor components in one or more directions, such as along axes 50, 52, and / or 54, and along directions at various angles to axes 50, 52, and / or 54. The stress map 220 may be generated, for example, by a finite element analysis ("FEA") of the component 10, which may be performed, for example, by the computing system 100 or performed separately (e.g., via a separate computing system) and provided to the computing system. Examples of FEA software suitable for such analysis include, for example, ANSYS, Simulia, and Nastran.

[0057] Comparing the detected grain structure 210 with the stress map 220 may allow for the determination of the grain structure and the regions or directions of stress increase. Furthermore, such a comparison may allow for a comparison between the localization of such grain structures (e.g., with respect to axes 50, 52, 54) and the direction of stress increase or stress direction.

[0058] For example, referring to Figures 10(A), 10(B), 10(C), and 10(D), in some embodiments, the comparison step may include overlaying the stress map 220 (e.g., stress map 506, referring to Figure 2) onto the processed image 504 in order to compare the localization of the detected grain structure 210 with the stress direction 222 (e.g., the direction of stress concentration) of the stress map 220. This overlay may be performed with respect to axes 50, 52, and 54 such that the axes in the processed image 504 coincide with those in the stress map 220.

[0059] Such comparisons may include, for example, the detection of grain structures 210 located within a stress direction 222, and the determination of the localization of those detected grain structures 210 relative to a specific stress direction 222. For example, Figures 10(A) and 10(B) show the superposition of stress maps 220 onto a processed image 504 for comparing the localization of detected grain structures 210 with the stress direction 222 along the X-axis 50. As shown in Figure 10(A), arrow 230 indicates an exemplary grain structure 210 located within the stress direction 222 and generally perpendicular to the stress direction 222. As shown in Figure 10(B), arrow 232 indicates an exemplary grain structure 210 located within the stress direction 222 and generally parallel to the direction of the stress direction 222. Figures 10(C) and 10(D) show the superposition of stress maps 220 onto a processed image 504 for comparing the localization of detected grain structures 210 with the stress direction 222 along the Y-axis 52. As shown in Figure 10(C), arrow 234 indicates an exemplary grain structure 210 that lies within the stress direction 222 and is generally perpendicular to the stress direction 222. As shown in Figure 10(D), arrow 236 indicates an exemplary grain structure 210 that lies within the stress direction 222 and is generally parallel to the stress direction 222.

[0060] The computing system 100, such as the data acquisition system 102 and / or its user computing system 104, may further be capable of determining the predicted lifetime of the component 10 based on the creep behavior of the material, etc. The predicted lifetime of the component 10 may be based on the localization of one or more detected grain structures 210 and a stress map 220 of the component 10. For example, such a determination may be based on the detection of grain structures 210 within a stress direction 222 and the determination of the localization of those detected grain structures 210 in a specific, e.g., relative to a stress direction 222.

[0061] In exemplary embodiments, the determination steps include determining creep probabilities based on the localization of one or more detected grain structures 210 and the stress directions 222 of the stress map 220, and determining the predicted lifetime of the component 10 based on the creep probabilities. For example, the detection of grain structures 210 located within the stress directions 222, and the determination of the localization of those detected grain structures 210 relative to a particular direction of the stress directions 222, may be used to determine the creep probabilities of the component 10. One or more creep probabilities may be generated based on such detection and determination. The creep probabilities may then be used to adjust the predicted lifetime of the component 10, such as against a baseline predicted lifetime.

[0062] For example, as explained, Figure 10(A) shows an exemplary grain structure 210 that is within the stress direction 222 and generally perpendicular to the stress direction 222. Figure 10(C) also shows an exemplary grain structure 210 that is within the stress direction 222 and generally perpendicular to the stress direction 222. Such generally perpendicular grain structures 210 increase the probability of creep and therefore decrease the predicted lifetime of component 10. Figure 10(B) shows an exemplary grain structure 210 that is within the stress direction 222 and generally parallel to the stress direction 222. Figure 10(D) also shows an exemplary grain structure 210 that is within the stress direction 222 and generally parallel to the stress direction 222. Such generally parallel grain structures 210 may decrease, not increase, or increase the probability of creep by a relatively small amount compared to, for example, perpendicular grains, and therefore increase or decrease the predicted lifetime of component 10.

[0063] It should be understood that this disclosure is not limited to comparisons with stress directions along axes 50, 52, and / or 54 only, or to predictive lifetime determinations based solely on generally parallel or generally perpendicular grain structures. Rather, such comparisons and determinations are exemplary embodiments, and this disclosure encompasses predictive lifetime determinations based on stress directions in any suitable direction and on grain structures having any suitable localization with respect to the stress directions. The inventors have discovered the ability to detect macrostructures on actual three-dimensional components using non-destructive characterization techniques and the ability to link the extracted grain structures with stress maps to perform automated computer-based inferences regarding the creep lifetime of a particular component. This disclosure advantageously facilitates more accurate and consistent lifetime prediction and creep probability determination based on this.

[0064] In some embodiments, additional variables may be used to determine the predicted lifespan of component 10. For example, such additional variables may be used to determine the creep probability. Examples of such variables may include, for example, the material type of the component, the service time of component 10, and / or the service temperature of component 10.

[0065] The grain structure, or one or more values ​​or magnitudes of such additional variables, may be used to adjust the predicted lifetime. For example, relatively large grain boundaries (e.g., relative to a given threshold for component 10) may increase the predicted lifetime, and relatively small grain structures (e.g., relative to the threshold) may decrease the predicted lifetime. Similarly, relatively short usage time or low usage temperature (e.g., relative to a given threshold for component 10) may increase the predicted lifetime, and relatively long usage time or high usage temperature (e.g., relative to the threshold) may decrease the predicted lifetime. Such increases and / or decreases may be made relative to a baseline predicted lifetime for the component, which may be predetermined or determined using the computing system 100.

[0066] The values ​​or magnitudes of such additional variables, as well as thresholds, may be determined by the computing system 100, or they may be determined independently and provided to the computing system 100 for use in the determination step.

[0067] The determined predicted lifespan may, advantageously, be output from the computing system 100, thereby allowing the user of the computing system to receive the determined predicted lifespan. The predicted lifespan determination may, advantageously, provide a relatively accurate lifespan estimate and thus allow the user to extend the actual service life of component 10 to an acceptable degree in accordance with the determined predicted lifespan while reducing concerns about the risk of failure. The predicted lifespan determination may, even more advantageously, allow the user to shorten the actual service life of component 10 and discontinue use of component 10 before the risk of failure etc. occurs in accordance with the determined predicted lifespan, and thus reduce the problem of unplanned downtime.

[0068] Figure 11 shows a block diagram of an exemplary computing system 100 that performs one or more of the various functions and steps described herein according to embodiments of the present disclosure. Computing system 100 may include multiple computing systems that communicate over network 1680. Computing system 100 may include one or more user computing systems 104 and one or more data acquisition systems 102.

[0069] The user computing system 104 may include one or more computing devices. The computing devices may include mobile computing devices (e.g., smartphones or tablets), laptop computing devices, desktop computing devices, wearable computing devices (e.g., smartwatches, smart jackets, smart glasses, smart backpacks, etc.), smart appliances (e.g., smart thermostats, smart refrigerators, smart washing machines, smart dryers, etc.), embedded computing devices, surveillance computing devices (e.g., drones), or any other type of computing device.

[0070] The user computing system 104 may include one or more processors 1612 that can be used to perform one or more operations. One or more processors 1612 may include any suitable processing device (e.g., a processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be one or more processors that are operablely connected. One or more processors 1612 may perform operations in series and / or in parallel. One or more processors 1612 may be dedicated to a particular computing device and / or may be used by multiple devices to perform processing tasks.

[0071] The user computing system 104 may include a memory 1614 capable of storing data 1616 and / or instructions 1618. The memory 1614 may include one or more non-temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The data 1616 may include user data, application data, operating system data, and the like. The data 1616 may include text data, image data, audio data, statistical data, latent encoded data, and the like. The instructions 1618 may include instructions that, when executed by one or more processors 1612, cause the user computing system 104 to perform an action.

[0072] In some implementations, the user computing system 104 can store and utilize one or more machine learning models 1620 (such as the machine learning models 520 described above, or other models). One or more machine learning models 1620 may include computer vision models, which may include object detection models, for example. One or more machine learning models 1620 may further include detection models, natural language processing models, segmentation models, classification models, augmentation models, generative models, discrimination models, and / or one or more other model types. In some implementations, one or more machine learning models 1620 may include various machine learning models, such as neural networks (e.g., deep neural networks), or other types of machine learning models, including nonlinear models and / or linear models. Neural networks may include convolutional neural networks, feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), or other forms of neural networks. One or more machine learning models 1620 may include one or more transformer models.

[0073] The user computing system 104 may include one or more user input components 1622, one or more user interfaces 1624, and / or one or more sensors 1626. One or more user input components 1622 may be configured to receive user input and / or environmental input. For example, one or more user input components 1622 may include touch-sensitive components (e.g., touch-sensitive displays or touchpads) that respond to touch of user input objects (e.g., fingers or styluses). Touch-sensitive components can help implement a virtual keyboard. Other exemplary user input components include computer mice, remotes, controllers, microphones, conventional keyboards, or other means by which a user can provide user input. In some implementations, one or more user input components 1622 may include one or more gesture processing engines for determining touch gestures, audio gestures, and / or body gestures. One or more user interfaces 1624 may be configured to acquire and / or display data. One or more user interfaces 1624 may be associated with an operating system, one or more applications, one or more web platforms, and / or one or more devices. One or more sensors 1626 may include one or more image sensors, one or more infrared sensors, one or more light-detecting and ranging (lidar) sensors, one or more audio sensors, one or more touch sensors, one or more sonar navigation and ranging (sonar) sensors, and / or one or more thermal sensors.

[0074] The user computing system 104 may be communicably connected to the data acquisition system 102 via a network 1680 which may include the Internet (e.g., Ethernet and / or Wi-Fi), Bluetooth, and / or direct wiring.

[0075] The data collection system 102 may include one or more computing devices. The computing devices may include mobile computing devices (e.g., smartphones or tablets), laptop computing devices, desktop computing devices, wearable computing devices (e.g., smartwatches, smart jackets, smart glasses, smart backpacks, etc.), smart appliances (e.g., smart thermostats, smart refrigerators, smart washing machines, smart dryers, etc.), embedded computing devices, surveillance computing devices (e.g., drones), or any other type of computing device.

[0076] The data acquisition system 102 may include one or more processors 1712 that can be used to perform one or more operations. One or more processors 1712 may include any suitable processing device (e.g., a processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be one or more processors that are operablely connected. One or more processors 1712 may perform operations in series and / or in parallel. One or more processors 1712 may be dedicated to a particular computing device and / or may be used by multiple devices to perform processing tasks.

[0077] The data acquisition system 102 may include a memory 1714 capable of storing data 1716 and / or instructions 1718. The memory 1714 may include one or more non-temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The data 1716 may include user data, application data, operating system data, and the like. The data 1716 may include text data, image data, audio data, statistical data, latent encoded data, and the like. The instructions 1718 may include instructions that cause the user computing system 104 to perform an action when executed by one or more processors 1712.

[0078] In some implementations, the data acquisition system 102 can store and utilize one or more machine learning models 1720. One or more machine learning models 1720 may include, for example, computer vision models that include object detection models. One or more machine learning models 1720 may further include detection models, natural language processing models, segmentation models, classification models, augmentation models, generative models, discrimination models, and / or one or more other model types. In some implementations, one or more machine learning models 1720 may include various machine learning models such as neural networks (e.g., deep neural networks), or other types of machine learning models including nonlinear models and / or linear models. Neural networks may include convolutional neural networks, feedforward neural networks, recurrent neural networks (e.g., long-short-term memory recurrent neural networks), or other forms of neural networks. One or more machine learning models 1620 may include one or more transformer models.

[0079] The data acquisition system 102 may include one or more user input components 1722, one or more user interfaces 1724, and / or one or more sensors 1726. One or more user input components 1722 may be configured to receive user input and / or environmental input. For example, one or more user input components 1722 may include touch-sensitive components (e.g., touch-sensitive displays or touchpads) that respond to touch of a user input object (e.g., a finger or stylus). Touch-sensitive components can help implement a virtual keyboard. Other exemplary user input components include a computer mouse, remote, controller, microphone, conventional keyboard, or other means by which a user can provide user input. In some implementations, one or more user input components 1722 may include one or more gesture processing engines for determining touch gestures, audio gestures, and / or body gestures. One or more user interfaces 1724 may be configured to acquire and / or display data. One or more user interfaces 1724 may be associated with an operating system, one or more applications, one or more web platforms, and / or one or more devices. One or more sensors 1726 may include one or more image sensors, one or more infrared sensors, one or more light-detecting and ranging (lidar) sensors, one or more audio sensors, one or more touch sensors, one or more sonar navigation and ranging (sonar) sensors, and / or one or more thermal sensors.

[0080] The data acquisition system 102 may be communicably connected to the user computing system 104 via a network 1680 which may include the Internet (e.g., Ethernet and / or Wi-Fi), Bluetooth, and / or direct wiring.

[0081] The user computing system 104 and / or the data acquisition system 102 can train models 1620 and / or 1720 by interacting with a training computing system 150 which is communicatively coupled via the network 1680. The training computing system 150 may be separate from the data acquisition system 102 or may be part of the data acquisition system 102.

[0082] The training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, graphics processing unit (GPU), tensor processing unit (TPU), microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be one or more processors operably connected. The memory 154 may include one or more non-temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 executed by the processors 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes one or more server computing devices, or is otherwise implemented by them.

[0083] The training computing system 150 may include a model trainer 160 that trains machine-learned models 1620 and / or 1720 stored in the user computing system 104 and / or the data acquisition system 102 using various training or learning techniques, such as backpropagation of errors. For example, a loss function can be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Parameters can be iteratively updated over several training iterations using gradient descent techniques.

[0084] In some implementations, performing error backpropagation may include performing truncated time-series backpropagation. The model trainer 160 can perform several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the trained model.

[0085] In particular, the model trainer 160 can train machine-learned models 1620 and / or 1720 based on a set of training data (or training dataset) 162. The training data 162 can include training images (e.g., training image data) of multiple training components. For example, if the components are gas turbine components (e.g., airfoils), the training components may also be gas turbine components (e.g., airfoils), and the machine-learned models 1620 and / or 1720 can be trained with respect to training images of multiple training components. The training images may include pre-operation and post-operation images of the training components. For example, a pre-operation image may be an image of the training component before it has experienced an operation cycle. A post-operation image may be an image of the training component after it has experienced one or more operation cycles.

[0086] In exemplary embodiments, machine learning models 1620, 1720 are trained on training data 162, which includes training images of multiple training components, training stress maps of multiple training components, and multistep predicted lifetimes generated by the multistep path 501 described above. For example, the training images and training stress maps (as well as training motion data) are collected or acquired from multiple training components (which may be components of the same type as component 10). The training images, training stress maps, and / or training motion data may be executed through the multistep path 501 described above with reference to Figure 2 to generate multistep predicted lifetimes of the training components. In this way, machine learning models 1620, 1720 (such as machine learning model 520 described above) may be provided with the same training input data provided to the multistep path 501 to generate predicted lifetimes of components, and then the accuracy of machine learning models 1620, 1720 can be determined by comparing the predicted lifetimes of components with the multistep predicted lifetimes.

[0087] In other words, referring again to Figure 2, during the training of the machine learning model 520, training images, training stress maps, and / or training motion data may be provided to both the multistep path 501 and the machine learning model 520 via an end-to-end path. The multistep path 501 may output a multistep predicted lifetime, and the end-to-end path (e.g., via the machine learning model 520) may output a predicted lifetime that can be compared to the multistep predicted lifetime of the multistep path 501. This process may be repeated many times to train the machine learning model 520 and / or to improve the accuracy of the machine learning model 520.

[0088] The model trainer 160 includes computer logic used to provide a desired function. The model trainer 160 may be implemented in hardware, firmware, and / or software that control a general-purpose processor. For example, in some implementations, the model trainer 160 includes a program file stored in a storage device, loaded into memory, and executed by one or more processors. In other implementations, the model trainer 160 includes one or more sets of computer executable instructions stored in a tangible computer-readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0089] Network 1680 can include any type of communication network (e.g., a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or any combination thereof) and can include any number of wired or wireless links. Communication over Network 1680 can take place over any type of wired and / or wireless connection using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or formatting (e.g., HTML, XML), and / or protection methods (e.g., VPN, Secure HTTP, SSL).

[0090] Computing system 100 may be used to implement the systems and methods disclosed herein. Other computing systems, including other system configurations, may be used to implement the systems and methods disclosed herein.

[0091] Referring here to Figure 12, a flowchart of a method 1100 for determining the predicted lifetime of component 10 according to an embodiment of the subject matter is shown. One or more steps of such a method may be performed by a computing system 100, for example, as described herein. Generally, method 1100 is described herein with reference to the computing system 100, component 10, and process 500 described above with reference to Figures 1 to 10. However, it will be understood by those skilled in the art that the disclosed method 1100 may generally be used in any other suitable system configuration. In addition, although Figure 12 shows steps performed in a particular order for illustrative and explanatory purposes, the methods described herein are not limited to any particular order or arrangement unless specifically specified in the claims. Those skilled in the art will understand that using the disclosures provided herein, various steps of the methods disclosed herein can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of this disclosure. Dashed boxes indicate optional steps of method 1100.

[0092] In some embodiments, the method may include utilizing a component in one or more operating cycles of a machine. For example, in an exemplary implementation, the component may be a gas turbine component. One or more operating cycles may include operating the gas turbine for a set period of time (e.g., about 10,000 to about 35,000 operating hours) with the gas turbine component installed. After the operating cycle, the method may include removing the gas turbine component from the gas turbine.

[0093] For example, method 1100 may include, in (1102), preparing the components for imaging of the components or the like. Such preparation may include, for example, surface preparation of the components (e.g., acid etching, electrochemical treatment, or other chemical treatment, or mechanical polishing, etc.) and / or other suitable processes to facilitate the visibility of the grain structure of the surface of the components.

[0094] In many implementations, method 1100 may include, in (1104), acquiring images of the components by a computing system comprising one or more computing devices. The images may be acquired by the data acquisition system 102 described above with reference to Figure 1. The images may be at least one of photographs, videos, laser scan images, X-ray scan images, Laue orientation images, electron channeling contrast images, or three-dimensional scan shapes.

[0095] In certain implementations, method 1100 may include, in (1106), obtaining a stress map of the component by a computing system. The stress map may be generated, for example, by a finite element analysis ("FEA") of the component, which may be performed, for example, by the computing system or performed separately (e.g., via a separate computing system) and provided to the computing system. Examples of FEA software suitable for such analysis include, for example, ANSYS, Simulia, and Nastran. The stress map may show high-stress and low-stress regions within the component, and these regions may be based at least in part on operating data associated with the component (e.g., operating time, average temperature to which the component was exposed, or other operating data).

[0096] In various embodiments, method 1100 may include, in (1108), obtaining operational data related to a component. The operational data may be related to a component and may include time data and temperature data. The time data may indicate the amount of time the component spent operating. For example, in embodiments where the component is a gas turbine component, the time data may indicate the amount of time the gas turbine component spent in an operating gas turbine. The temperature data may indicate multiple temperatures (or average temperatures) to which the component was exposed during operation. For example, in embodiments where the component is a gas turbine component, the temperature data may be the temperatures to which the gas turbine component was exposed during operation, and these temperatures may be multiple temperatures over a period of time or an average temperature. Steps 1104, 1106, and 1108 may be performed in parallel (e.g., simultaneously) and / or in an alternative order other than the illustrated order.

[0097] In exemplary embodiments, Method 1100 may include, in (1110), processing images and stress maps of a component using a machine learning model by a computing system to generate a predicted lifetime of the component as the output of the machine learning model. In some implementations, operating parameters such as operating time data and temperature data may also be utilized by the machine learning model to generate the predicted lifetime. In some implementations, Method may include, by a computing system, processing images, stress maps, and operating data using a machine learning model to generate a predicted lifetime of the component. The predicted lifetime may be at least one of a binary output, an integer output, and / or a probability density function. A binary output may be a single output indicating whether the component can survive after another operating cycle (e.g., yes or no). An integer output may be the number of operating cycles in which the component can subsequently survive (e.g., one more cycle, two more cycles, three more cycles, etc.). A probability density function may indicate the amount of time in which the component can survive during operation. A probability density function may describe the probability of component survival over a given operating time. More specifically, a probability density function can describe the likelihood that a continuous random variable will take a particular value. In particular, a cumulative distribution function (CDF) that describes the probability that a random variable (X) will take a value less than or equal to a specific value (x) can be output as the predicted lifetime. For example, the CDF as the predicted lifetime may show the probability (e.g., as a percentage) of reaching time X.

[0098] In many embodiments, the machine-learned model (e.g., machine-learned model 520) may be a first machine-learned model, and the system may further include a second machine-learned model. The second machine-learned model may be configured to modify, enhance, or transform the image (e.g., before being used in the first machine-learned model to process and generate predicted lifetimes). The computing system may process the image using the second machine-learned model to generate an enhanced image. In such an implementation, the method may further include the computing system processing an enhanced image and stress map of the components using the first machine-learned model to generate predicted lifetimes of the components as the output of the first machine-learned model.

[0099] In many embodiments, the method may include training a machine learning model via a model trainer that can train the machine learning model on a set of training data (or training dataset). The training data may include training images of multiple training components. For example, if the components are gas turbine components (such as airfoils), the training components may also be gas turbine components (e.g., airfoils), and the machine learning model may be trained on training images of multiple training components. The training images may include pre-operation and post-operation images of the training components. For example, a pre-operation image may be an image of the training component before it has experienced an operation cycle. A post-operation image may be an image of the training component after it has experienced one or more operation cycles.

[0100] In an exemplary embodiment, a machine learning model is trained on training data including training images of multiple training components, training stress maps of multiple training components, and multi-step predicted lifetimes. For example, the training images and training stress maps (as well as training motion data) are collected or obtained from multiple training components (which may be components of the same type as component 10). The training images, training stress maps, and / or training motion data may be executed through a multi-step path 501 described above with reference to Figure 2 to generate multi-step predicted lifetimes of the training components. Thus, a machine learning model may be provided with the same training input data provided to the multi-step path 501 to generate predicted lifetimes of components, and then the accuracy of the machine learning model can be determined by comparing the predicted lifetimes of the components with the multi-step predicted lifetimes.

[0101] In many implementations, training a machine learning model may further involve adjusting or modifying the weights or parameters of the machine learning model based on a comparison between the predicted lifetimes of its components and the multi-step predicted lifetimes. This process may be repeated until the error (or difference) between the output generated by the machine learning model and the multi-step output is minimized or completely eliminated.

[0102] For example, to generate a multi-step predicted lifetime for each of a group of training components, the method may further include processing a training image of the training component to detect the grain structure on the training component, comparing the detected grain structure with the corresponding training stress map of the training component, and determining the multi-step predicted lifetime of the training component based on the localization of the detected grain structure and the training stress map. In such embodiments, processing the training image may include performing pixel analysis of the training image by utilizing computer vision. In addition, in such implementations, the comparison step may include overlaying the training stress map onto the processed training image to compare the localization of the detected grain structure with the stress direction of the training stress map. Furthermore, in such implementations, the determination step may include determining the creep probability based on the localization of the detected grain structure and the stress direction of the training stress map. Based on the creep probability, the method may include determining the multi-step predicted lifetime of the training component.

[0103] In exemplary embodiments, the steps to be determined are further based on one of the material type of the component, the usage time of the training component, and / or the usage temperature of the training component, as described herein. The training component may be of the same type as component 10. For example, the training component and component 10 may be gas turbine components, such as turbine components or compressor components. The turbine component may be a turbine blade, turbine shroud, turbine power nozzle, or other turbine components. The compressor component may be a compressor rotor blade, compressor stator vane, or other compressor components.

[0104] This specification uses examples to disclose the present invention in best mode and to enable any person skilled in the art to practice the invention, including the fabrication and use of any device or system and the execution of any method incorporating it. The patentable scope of the present invention is defined by the claims and may include other examples that a person skilled in the art may conceive. Such other examples are within the claims if they include structural elements that are not different from the language of the claims, or equivalent structural elements that do not substantially differ from the language of the claims.

[0105] Further aspects of the present invention are provided by the subject matter of the following clauses.

[0106] A computer implementation method for determining the predicted lifetime of a component, the method comprising: acquiring an image of the component by a computing system comprising one or more computing devices; acquiring a stress map of the component by the computing system, wherein the stress map indicates high-stress and low-stress regions within the component; and processing the image of the component and the stress map using a machine learning model by the computing system to generate the predicted lifetime of the component as the output of the machine learning model.

[0107] A method of any of the preceding clauses in which a component has experienced at least one operating cycle.

[0108] A method of which any of the preceding clauses has a new component.

[0109] The predicted lifetime is at least one of the following: a binary output indicating whether the component can survive after another operating cycle, an integer output indicating the number of operating cycles the component can survive after, or a probability density function, as described in any of the preceding clauses.

[0110] The method according to any of the preceding clauses, wherein the image is one of a photograph, a video frame, a laser scanned image, an X-ray scanned image, a Laue orientation image, an electron channeling contrast image, or a three-dimensional scanned shape.

[0111] The method of any of the preceding clauses, further comprising: obtaining operational data related to a component by a computing system, wherein the operational data includes time data and temperature data; and processing images, stress maps, and operational data using a machine learning model to generate a predicted lifetime of the component as the output of the machine learning model by the computing system.

[0112] A machine learning model is trained on a training dataset containing training images of multiple training components, as described in any of the preceding clauses.

[0113] The method according to any of the preceding clauses, wherein the machine-learned model is a first machine-learned model, and the method further comprises processing an image using a second machine-learned model to generate an enhanced image, and processing the enhanced image and stress map of the components using the first machine-learned model by a computing system to generate predicted lifetimes of the components as output of the machine-learned model.

[0114] The method described in any of the preceding clauses, wherein a machine learning model is trained on a training dataset that includes training images of multiple training components, training stress maps of multiple training components, and multi-step predicted lifetimes of multiple training components.

[0115] The method according to any of the preceding clauses, wherein the multistep predicted lifetime for each of a plurality of training components is generated by processing a training image of the training component to detect a grain structure on the training component, comparing the detected grain structure with the corresponding training stress map of the training component, and determining the multistep predicted lifetime of the training component based on the localization of the detected grain structure and the training stress map.

[0116] The method described in any of the preceding clauses, wherein processing the training images includes performing pixel analysis of the training images by utilizing computer vision.

[0117] The method of any of the preceding clauses, wherein the comparison includes overlaying a training stress map onto a processed training image in order to compare the localization of the detected grain structure with the stress direction of the training stress map.

[0118] The method according to any of the preceding clauses, wherein determining the creep probability is based on the localization of the detected grain structure and the stress direction of the training stress map, and the multi-step predicted lifetime of the training component is based on the creep probability.

[0119] The method according to any of the preceding clauses, wherein a machine learning model is trained on a training dataset comprising training images of multiple training components, training stress maps of multiple training components, and multistep predicted lifetimes of multiple training components, wherein the multistep predicted lifetime for each of the multiple training components is generated by the following operations: processing the training images of the training component to detect a grain structure on the training component; comparing the detected grain structure with the corresponding training stress map of the training component; and determining the multistep predicted lifetime of the training component based on the localization of the detected grain structure and the training stress map.

[0120] A computing system for determining the predicted lifetime of a component, the system comprising one or more processors and one or more non-temporary computer-readable media that collectively store instructions that, when executed by one or more processors, cause the computing system to perform an operation, wherein the operation includes acquiring an image of the component and acquiring a stress map of the component, wherein the stress map indicates high-stress and low-stress regions within the component, and processing the image and stress map of the component using a machine learning model to generate the predicted lifetime of the component as the output of the machine learning model.

[0121] The predicted lifetime is at least one of the following for any system described in any of the preceding clauses: a binary output indicating whether a component can survive after another operating cycle, an integer output indicating the number of operating cycles in which the component can survive, or a probability density function.

[0122] A system as described in any of the preceding clauses, wherein the image is one of a photograph, video frame, laser scanned image, X-ray scanned image, Laue orientation image, electron channeling contrast image, or three-dimensional scanned shape.

[0123] The system described in any of the preceding clauses, wherein the operation involves the computing system acquiring operational data relating to a component, the operational data including time data and temperature data, and further involves the computing system processing images, stress maps, and operational data using a machine learning model to generate a predicted lifetime of the component as the output of the machine learning model.

[0124] A system described in any of the preceding clauses, in which a machine learning model has been trained on a training dataset containing training images of multiple training components.

[0125] A system as described in any of the preceding clauses, wherein the machine learning model is a first machine learning model, and the operation further includes processing an image using a second machine learning model to generate an enhanced image, and processing the enhanced image and stress map of the components using the first machine learning model to generate a predicted lifetime of the components as the output of the machine learning model.

[0126] A machine learning model is trained on a training dataset which includes training images of multiple training components, training stress maps of multiple training components, and multistep predicted lifetimes of multiple training components, wherein the multistep predicted lifetime for each of the multiple training components is generated by the following operations: processing the training images of the training component to detect a grain structure on the training component; comparing the detected grain structure with the corresponding training stress map of the training component; and determining the multistep predicted lifetime of the training component based on the localization of the detected grain structure and the training stress map, as described in any of the preceding clauses. [Explanation of Symbols]

[0127] 10 Components 11 Exterior 50 X axis 52 Y-axis 54 Z axis 100 Computing Systems 102 Data Acquisition System 104 User Computing Systems 106 Imaging devices 108 Three-dimensional data acquisition devices 110 Lens Assembly 112 Image Capture Devices 120 lasers 124 sensors 130 Robot Arms, 6-Degree-of-Freedom Arms 150 Training Computing Systems 152 One or more processors 154 memory 156 data 158 Command 160 Model Trainer 162 training data, training dataset 202 Cut-off portion 210 Grain structure 220 Stress Map 222 Stress direction 230 Arrow 232 Arrows 234 Arrow 236 Arrows 500 processes 501 Multi-step route, multi-step 502 Input 503 End-to-end, end-to-end route 504 images 506 Stress Map 508 Operation Data 510A Photo 510B Photo 512 Video 514 Three-dimensional scanning shape, three-dimensional scanning 520 First machine learning model 522 Predicted lifetime, predicted component lifetime 540 Binary Output 542 integer output 544 Probability Density Function 600 blurry images 602 Enhanced Images 700 grayscale images 702 Black and white image 1612 1 or more processors 1614 Memory, one or more non-temporary computer-readable media 1616 data 1618 command 1620 One or more machine learning models 1622 One or more user input components 1624 One or more user interfaces 1626 One or more sensors 1712 1 or more processors 1714 memory 1716 Data 1718 command 1720 One or more machine learning models 1722 One or more user input components 1724 One or more user interfaces 1726 One or more sensors 1100 methods

Claims

1. A computer implementation method for determining the predicted lifespan of a component, wherein the method is: A computing system (100) comprising one or more computing devices acquires (1104) an image (504) of the component (10), The computing system (100) acquires (1106) a stress map (506) of the component (10), wherein the stress map (506) indicates high-stress and low-stress regions within the component (10), The computing system (100) processes the image (504) and stress map (506) of the component (10) using the machine learning model (520) in order to generate a predicted lifetime (522) of the component (10) as the output of the machine learning model (520) (1110) Computer implementation methods, including those mentioned above.

2. The method according to claim 1, wherein the component (10) has experienced at least one operating cycle.

3. The method according to claim 1, wherein the aforementioned component (10) is a new component.

4. The method according to any one of claims 1 to 3, wherein the predicted lifetime (522) is at least one of a binary output (540) indicating whether the component (10) can survive after another operating cycle, an integer output (542) indicating the number of operating cycles in which the component (10) can survive, or a probability density function (544).

5. The method according to any one of claims 1 to 4, wherein the image (504) is one of a photograph (510A, 510B), a video frame (512), a laser scan image, an X-ray scan image, a Laue orientation image, an electron channeling contrast image, or a three-dimensional scan shape (514).

6. The computing system (100) acquires (1108) operational data (508) related to the component (10), wherein the operational data (508) includes time data and temperature data. The computing system (100) processes the image (504), the stress map (506), and the motion data (508) using the machine learning model (520) in order to generate the predicted lifetime (522) of the component (10) as the output of the machine learning model (520). The method according to any one of claims 1 to 5, further comprising:

7. The machine learning model (520) is the first machine learning model (520), and the method is Processing the image (504) using a second machine learning model to generate an enhanced image (602), The computing system (100) processes the enhanced image (602) and stress map (506) of the component (10) using the first machine learning model (520) in order to generate the predicted lifetime (522) of the component (10) as the output of the machine learning model (520). The method according to any one of claims 1 to 6, further comprising:

8. The method according to any one of claims 1 to 7, wherein the machine learning model (520) is trained on a training dataset (162) which includes training images of a plurality of training components.

9. The method according to claim 8, wherein the training dataset (162) further includes training stress maps of the plurality of training components and multistep predicted lifetimes of the plurality of training components.

10. The multi-step predicted lifetime for each of the plurality of training components is To detect the crystal grain structure on the training component, the training image of the training component is processed, The detected crystal grain structure is compared with the corresponding training stress map of the training component, Based on the detected localization of the crystal grain structure and the training stress map, the multi-step predicted lifetime of the training component is determined. The method according to claim 9, which is produced by...

11. The method according to claim 10, wherein processing the training images includes performing pixel analysis of the training images by utilizing computer vision.

12. The method according to claim 10 or 11, wherein the comparison includes superimposing the training stress map onto the processed training image in order to compare the localization of the detected crystal grain structure with the stress direction of the training stress map.

13. The method according to any one of claims 10 to 12, wherein the determination includes determining the creep probability based on the localization of the detected grain structure and the stress direction of the training stress map, and determining the multi-step predicted lifetime of the training component based on the creep probability.

14. A computing system (100) for determining the predicted lifespan of a component, wherein the system (100) One or more processors (1612), The system comprises one or more non-temporary computer-readable media (1614) that collectively store instructions (1618) which, when executed by one or more processors (1612), cause the computing system (100) to perform an operation, and the operation is To obtain an image (504) of the aforementioned component (10), The acquisition of a stress map (506) of the aforementioned component (10), wherein the stress map (506) shows high-stress and low-stress regions within the aforementioned component (10), To generate a predicted lifetime (522) of the component (10) as the output of the machine learning model (520), the machine learning model (520) is used to process the image (504) and the stress map (506) of the component (10). A computing system (100) including the above.

15. The system according to claim 14, wherein the predicted lifetime (522) is at least one of a binary output (540) indicating whether the component (10) can survive after another operating cycle, an integer output (542) indicating the number of operating cycles in which the component (10) can survive, or a probability density function (544).

Images