Predicting plant maintenance using infrared and normal images

Abstract

A computer-implemented method for predicting the operating time until a failure state of a system (140), wherein the computer-implemented method comprises: Receiving a plurality of historical image pairs of a system (140), such as image pairs of a failure state of the system (140), by one or more processors (304), wherein each image pair contains an infrared image and a corresponding normal image; Receiving an initial model by the one or more processors (304), which is trained to reconstruct each image pair of the system (140) in a functioning state (120) of the system (140);Generating a similarity coefficient between each historical image pair and a reconstruction of the historical image pair by the first model for the plurality of historical image pairs by the one or more processors (304), wherein the historical image pair contains a timestamp of a capture of the historical image pair; Receiving a second model by the one or more processors (304), which is trained to predict, based on the similarity coefficients and a timestamp of a capture of the plurality of historical image pairs, an operating time until a failure condition of the system (140) occurs;In response to receiving a real-time image pair of the system (140), the one or more processors (304) generate real-time similarity coefficients between the real-time image pair and the real-time image pair reconstructed by the first model, wherein the real-time image pair contains a timestamp of the acquisition of the real-time image pair; and the one or more processors (304) calculate a predicted operating time until the failure state of the system (140) based on a correlation between the similarity coefficients of the real-time image pair and the timestamp of the acquisition of the real-time image pair and a historical image pair from the plurality of historical image pairs with corresponding similarity coefficients and a time delta determined from the timestamp of the acquisition of the real-time image pair and a timestamp of the failure state of the system (140) after the historical image pair with the corresponding similarity coefficients.

Description

FIELD OF THE INVENTIONThe present invention relates to plant maintenance and failure prediction and, more particularly, to the use of machine learning plant state detection based on visible and infrared images of plants to predict maintenance intervention.BACKGROUNDEnterprise Asset Management (EAM) is a particular type of enterprise resource planning (ERP) that includes the integrated management of important business processes, often in real-time, and mediated by software and technology. EAM can be characterized as having functionalities that support management activities of large plants such as fleets of trucks, construction machines, turbines, oil platforms, and power generation plants. EAM systems and software enable the design, fabrication, operation, maintenance, and replacement of large facilities and plant components.Approaches to plant failure prevention include management using data that captures a mean time between failures (MTBF) of plant components. Companies and organizations benefit from accurately determining the MTBF and taking action to perform maintenance activities to prevent failures while avoiding maintenance from being performed before it is necessary.Infrared (IR) light is a type of electromagnetic radiation energy that is invisible to the human eye, but can be felt or measured as heat. Acquisition of IR images with special units such as thermal cameras adds additional details of anomalies that would otherwise not be detected by images in the visible light range or by the human eye. IR images provide something frequently, referred to as a thermal signature, and provide information about normal or expected operating conditions.AU 2016 102 407 A4 describes an image processing method for detecting the state of infrastructures on the basis of infrastructure image segments. CN 1 05 371 957 A describes an infrared temperature registration and positioning system for transformer stations, comprising an infrared camera, a visible light camera, a video server and a data analysis unit. US 2016 / 0 292 826 A1 describes a thermal imaging system that consists of a thermal sensor, a non-thermal image sensor, an image alignment system, an image correction system, and a controller that controls the sensors and systems, recognizes reflective surfaces, and creates corrected images in order to ensure accurate thermal data. US 2017 / 0 308 834 A1 describes a method for assessing the state of an asset by identification and image processing to determine whether the asset needs to be repaired or replaced.US 2019 / 0 087 256 A1 describes systems, devices and methods for evaluating assets and their operating conditions, in particular by defining and using a predictive model indicating whether a particular fault type is likely to occur within a particular time period at an asset. US 2020 / 0 097 720 A1 describes a method for automatic detection of anomalies in aerial images of an object by a trained neural network which reconstructs anomaly-free states and uses a comparison to generate anomaly warnings.SUMMARYIt is an object of the invention to provide an improved computer-implemented method, an improved computer program product and an improved computer system for predicting an operating time up to a failure state of a plant.The invention is described by the features of the independent claims. Embodiments are set out in the dependent claims.Embodiments of the present invention disclose a computer implemented method, computer program product and system for predicting an operating time to a plant failure condition. The computer-implemented method provides a computer processor to receive a sample of plant image pairs having a proper state status and a plurality of plant historical image pairs, such as plant fault state image pairs, wherein respective image pairs include an infrared image and a corresponding normal image. The processor trains a first model to reconstruct respective images of an image pair based on training data from the sample of image pairs of the plant having a proper state state. The processor generates, by the first model, a coefficient of similarity between a historical image pair and a reconstruction of the historical image pair for the plurality of historical image pairs, the historical image pair including a time mark detection of the historical image pair. The computer processor trains a second model to predict an operating time to a failure condition of the plant based on the similarity coefficients of the plurality of historical image pairs of the plant, respectively, and the time stamp associated with the acquisition of the plurality of historical image pairs. In response to receiving a real-time image pair of the plant, the processor generates real-time similarity coefficients between the real-time image pair and the real-time image pair reconstructed by the first model, the real-time image pair including the real-time image pair acquisition time mark, and the processor calculates a predicted operating time until a failure state of the plant based on a correlation between the similarity coefficients of the real-time image pair and the real-time image pair acquisition time mark and a historical image pair of the plurality of historical image pairs having corresponding similarity coefficients and a time delta determined from the real-time image pair acquisition time mark and the plant failure state time mark after the historical image pair having the corresponding similarity coefficients.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a functional block diagram illustrating a distributed computer computing environment according to an embodiment of the present invention. FIG. 2 is a flow chart illustrating the operations of a failure prediction program that includes an image reconstruction module and an image similarity module as components operating in the distributed computer computing environment of FIG. 1, in accordance with embodiments of the present invention. FIG. 3 illustrates a block diagram of components of a data processing system that includes a data processing unit configured to operatively perform the failure prediction program including the image reconstruction module and the image similarity module of FIG. 2, in accordance with an embodiment of the present invention.DETAILED DESCRIPTIONEmbodiments of the present invention take into account that precisely determining when to perform preventative maintenance of high quality installations can avoid expensive or catastrophic consequences of installation failure and prevent higher operating costs by replacing components no more than necessary or performing maintenance activity no more than necessary. The occurrence of a plant failure may often have significant consequences, which promotes a greater frequency of maintenance activity. Embodiments of the present invention take into account that plant owners in practice often decide for a "minor nausea" philosophy and accept the inconvenience and additional costs associated with the greater scheduled frequency of plant maintenance.Embodiments of the present invention also contemplate that plant inspection often requires manual activity to determine indications of whether the plant requires maintenance, which poses operational and human risks or may present challenges for precise inspection results due to limited access to plants or plant components. Embodiments further contemplate that plant owners balance between decisions in favor of performing more frequent plant inspection that includes higher costs and potential risk to inspection personnel, and the risk of greater consequences of a plant failure.Embodiments of the present invention provide a computer-implemented method, computer program product, and computer system for predicting a failure condition associated with a plant based on generating similarity coefficients from visible light range normal images and infrared images that provide a quantified difference between a condition of a current plant and the proper condition of the plant. Embodiments also provide a machine learning model to predict when a particular plant is likely to fail or require a maintenance-related intervention action by correlating the coefficients of similarity between the normal image and the infrared image of a current inspection with corresponding similarity coefficients of an image pair from the plurality of historical normal and infrared image pairs associated with prior plant failure conditions and prior plant condition inspection.In one embodiment of the present invention, the term "image(s)" refers to a digitized representation of objects represented as a mosaic of pixels and in formats that allow computer-based analysis. Embodiments include normal images that are images that capture electromagnetic wavelengths associated with the visible light range, as well as infrared images that capture electromagnetic wavelengths beyond (i.e., above) the visible red light range.In some embodiments, acquisition of the normal and infrared images occurs at approximately the same time and includes computer recognizable features associated with a grouping of pixels within the respective image. In some embodiments, the terms "coefficient of similarity" or "similarity coefficient(s)" refer to determining a value, often referred to as a distance between corresponding regions of a pair of current light source images (i.e., a current light source image and a previous light source image as a pair), and more particularly, to a magnitude value given to a vector in a multi-dimensional space that represents the distances between one or more features located within respective regions of the pair of images. A larger distance indicates a lower similarity between features of the image pair, while a smaller distance value indicates a higher similarity between features of the image pair. For example, a computer image analysis identifies a set of features in each of two infrared images of a component of a plant. A region of pixels within the respective regions of the infrared images contain the identified features, and the analysis includes generating a coefficient of similarity between the respective regions of the infrared images as well as attributes of the pixel regions that can be assigned as different dimensions of the vector representation.In some embodiments of the present invention, a plant refers to a physical object that can be inspected by image capture and performs operations that over time result in the plant not being able to perform an expected service due to a failure. In some embodiments, a plant includes large objects such as vehicles, construction machines, turbines, generators, aircraft components, and the like; however, embodiments of the invention are not limited to the size, type, or task of a plant. In some embodiments, a primary plant may include subcomponents that may also be considered a secondary plant of the primary plant. Embodiments of the present invention consider actions to prevent or prevent a failure of the operation of the plant as an intervention on the plant to perform preventive activity such as maintenance. In some embodiments, a reference to a plant failure state includes a plant performance degradation state. In some embodiments, inspection of a plant is performed before and after intervention at the plant and includes acquiring normal and infrared images of the plant. For purposes of clarity and simplicity, references herein to a plant failure, a plant failure event, or a plant failure condition include plant conditions of near-failure or imminent failure that support a preventative objective.Embodiments provide a practical solution to the problem of determining a precise prediction of when intervention should be performed on a plant for a maintenance or other preventive action prior to plant failure. Embodiments receive a sample of plant image pairs where the image pairs reflect a plant health condition and receive a plurality of plant historical image pairs that include a plant fault condition and plant inspection. In some embodiments, the historical image pairs of a set of similar or identical assets or asset components (besides the respective time mark information) may be combined into a historical collection of image pairs of the asset type. For example, pairs of images of a brake assembly of a fleet of similar trucks may be combined to produce a larger set of historical pairs of images. The historical image pairs of the plant include a time stamp of the image acquisition and information associated with the determined intervention activity. In some embodiments, a response to a failure condition of the plant involves interventional activity to repair, repair, or replace the plant. In other embodiments, the intervention provides inspection of the plant and includes a time stamp of the inspection besides capturing an image pair of the plant.For example, an enterprise organization may associate interventions at the plant with a work order or other formally documented information, e.g., a setting or scheduling of the intervention activity. The order of operation may include documentation of the intervention activity as well as the time stamp of the activity, and when the intervention involves a plant repair or a plant replacement, the work order information includes the type of failure, the duration of the repair or replacement, the parts replaced, the adjustments made, and the costs associated with the intervention. Embodiments of the present invention include, in addition to interventional activity information, inspection, maintenance activity, and failure event repairs within the plurality of historical image pairs. Time mark information associated with historical images of inspection and equipment failure events serve as data indicating the operation time until a next equipment failure event. Time mark information associated with the image pairs acquired during intervention provides data for machine learning prediction of an operating time to plant failure.Embodiments perform segmentation of the normal image and the infrared image of the pair of images to provide a more detailed representation of components of the plant. In some embodiments, segmenting an image generates a set of regions of pixels that form a square matrix of regions (e.g., n*2 regions).Embodiments train a first model to receive image regions of the plant as an input and perform a reconstruction of the input as an image of a proper state of the plant. Training the first model includes exclusively providing image pairs of a plant in a perfect state from the sample of image pairs of the plant as an input for training data. In this way, the first model provides a reference state for a correct state of the system at different image inputs from real-time inspection. The image pairs from real-time inspection can be acquired at different states, which can differ from the original sample of image pairs of a system in a perfect state. Using the original sample of image pairs as a reference for a perfect state of real-time image pairs may result in inaccurate results of similarity coefficients in regions of the image pair that otherwise have no influence. For example, the angle, illumination, or accumulated debris that has no effect may result in inaccurate calculation of similarity coefficients.In various embodiments, training the first model includes identifying and recognizing features of the plant within respective region segments of the received sample of image pairs. In some embodiments, the proper state state of the plant includes a new plant, a replaced component of the plant, and an adjusted plant or component of the plant that places the plant in an operating state with an absence or reduction of wear, stress, fatigue, vibration, excess heat, or other attributes that would indicate an operating state that is different from a new or near-new state and approximates a failure state. Training the model to identify the features within regions of the sample of the plant's image pair that have a good state status provides a reference point for generating similarity coefficients between historical images and the reference images. To generate reference images, the first model reconstructs a copy of the normal and infrared real-time images received as inputs in a proper state of the plant (i.e., reconstructs a most proper state) that serves as a reference for changes to the plant and provides improved accuracy of the calculated similarity coefficients. The reconstruction of the real-time images avoids problems associated with non-influence components of the images that interfere with the analysis and determination of similarity coefficients.Embodiments of the present invention generate a similarity coefficient between a reference image pair representing a proper condition of the plant and a respective image pair from the plurality of historical image pairs of the plant. Embodiments generate a similarity coefficient for each segmented region of each image pair of the plurality of historical image pairs. In various embodiments, providing the respective historical image pair to the first model enables the reconstruction of the historical image pair. The reconstruction forms a reference image pair containing regions of a plant in a perfect state. Embodiments generate a similarity coefficient between the segmented regions of the reconstructed normal and infrared images of the historical image pair and the segmented regions of the respective normal and infrared images of the historical image pair. In some embodiments, a similarity coefficient is generated for the respective image pairs for the plurality of historical image pairs.Embodiments perform generation of similarity coefficients for each historical image pair of the plurality of historical image pairs of the plant. The plurality of historical image pairs represent inspection and instances of a plant failure event detected from prior intervention activities of repair, maintenance, and inspection of the plant, and includes a timestamp and additional information associated with the intervention activity. The resulting set of similarity coefficients of the plurality of historical image pairs provides a range of similarity values associated with the plant from a proper state to a failure state. In some embodiments, the similarity coefficients are calculated as a Euclidean distance between pixels in a region of a reference image of the plant and the corresponding pixels of the region in the real-time image of a pair of images of an inspection of the plant. In other embodiments, the similarity coefficients are calculated based on cosine similarity methods or other similarity methods.Embodiments of the present invention train a second model to predict an operating time until a plant failure condition. The predicted operating time to a failure state allows planning and performing maintenance intervention on the plant to prevent a failure state. The determination of the predicted operating time until failure involves correlating the similarity coefficients of a real-time image pair from a plant inspection with corresponding similarity coefficients from image pairs of the plurality of historical image pairs. In addition, the operating time between the time stamp associated with the corresponding historical image pairs and a subsequent time stamp of a historical system failure state is an influencing factor in the determination of the predicted operating time until a failure of the system. In some embodiments, adjustments to the predicted time for intervention in a time frame prior to the predicted failure condition of the plant prevent occurrence of the failure. In some embodiments, information associated with the plurality of historical image pairs provides the data to train the second model for predicting a failure condition of the plant. The historical image pair information includes a time stamp of the historical intervention, the normal and infrared image pair acquired during the historical intervention, and details of the intervention such as details of the failure state, maintenance / repair performed, the duration of the intervention, and the cost of the intervention.In response to receiving a real-time image pair of the plant, embodiments of the present invention generate real-time similarity coefficients for respective regions of an infrared image and a normal image. In some embodiments, performing a real-time inspection of the plant in which a pair of images is captured includes a time stamp associated with the pair of images. Embodiments of the present invention generate similarity coefficients from the real-time pair of images of the inspection by passing a copy of the infrared and normal images as an input to the first model, which outputs a reconstruction of the passed images in approximation to an image representing a plant in a proper state as a reference. The real-time pair of images of the inspection and the reconstructed pair of images references provide the basis for generating similarity coefficients for the real-time inspection. Embodiments generate similarity coefficients for the respective segmented regions of the respective infrared images and the respective segmented regions of the respective normal images.Embodiments of the present invention calculate a predicted time to failure event of the plant based on a correlation between (i) the similarity coefficients of the regions of the real-time image pair and a time stamp of acquisition of the real-time image pair and (ii) the regions of a historical image pair of the plurality of historical image pairs having corresponding similarity coefficients and a time delta determined from the time stamp of acquisition of the real-time image pair and the time stamp of acquisition of an image pair of failure state of the plant after the historical image pair having the corresponding similarity coefficients.In some embodiments, jobs or other documented information sources for plant intervention include the pair of images (i.e., an infrared image and a normal image from the visible light range) of the plant state as well as a timestamp of an image acquisition, and may include the duration of the intervention, the activities performed during the intervention, whether the intervention was in response to a failure state of the plant, and the cost of the intervention, or a combination thereof. In some embodiments, the work orders associated with each of the plurality of historical images of the plant include plant inspection interventions after a proper state status of the plant and before a failure state of the plant.In some embodiments, the first model includes an autoencoder neural network and training that uses normal and infrared images of a plant with a good state state. In some embodiments, a separately developed and trained version of the first model performs a reconstruction of infrared images as a corresponding infrared image, and similarly, another version of the first model, developed and trained using normal images, performs a reconstruction of normal images as a corresponding normal image that represents a similarity (i.e., very small similarity distance values for similarity coefficients) with a proper state state of the plant, respectively. In some embodiments, development and training of the second model includes separate versions that use normal image similarity coefficients and infrared image similarity coefficients from real-time inspection of the plant, respectively. Embodiments of the present invention contemplate that the second model performs a regression function to learn the patterns of the time delta up to a plant failure condition from inspection images and time stamps associated with the acquisition of the inspection images as well as from similarity coefficients of historical failure condition image representations, historical inspection image representations, and the time stamps associated with the intervention.The descriptions of the various embodiments of the present invention have been presented for purposes of illustration and are not to be considered as exhaustive or limited to the disclosed embodiments. Those skilled in the art will appreciate that numerous modifications and variations are possible without departing from the scope of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application, or technical improvement over technologies found on the market, and to enable others skilled in the art to understand the embodiments disclosed herein.The present invention will now be described in detail with reference to the figures. FIG. 1 illustrates a functional block diagram illustrating a distributed computer computing environment, generally designated 100, in accordance with an embodiment of the present invention. FIG. 1 merely provides an illustration of an embodiment and does not imply any restrictions with respect to the environments in which various embodiments may be implemented. Those skilled in the art may make many modifications to the environment illustrated without departing from the scope of the invention as claimed.The distributed computer computing environment 100 includes a computing device 110, plant image pairs 120 in a proper state, historical intervention datasets 130, and a current plant inspection 140, all of which are interconnected via the network 150. The distributed computer computing environment further includes a plant 160 from which a current plant inspection 160 is obtained.The data processing unit 110 includes a user interface 115 and a failure prediction program 200 further illustrated as including the functionality of an image reconstruction module 117 and an image similarity module 119. In some embodiments, computing device 110 may be a blade server, a web server, a laptop computer, a desktop computer, a stand-alone mobile computing device, a smart phone, a tablet computer, or another electronic device or computing system capable of receiving, transmitting, and processing data. In other embodiments, computing device 110 may be a computing device that interacts with applications and services hosted and operated in a cloud computing environment. In another embodiment, computing device 110 may be a netbook computer, a personal digital assistant (PDA), or other programmable electronic devices capable of receiving data from other devices (shown and not shown) in distributed computing computing computing environment 100 over network 150 and exchanging data with these devices as well as performing operations of failure prediction program 200. Alternatively, in some embodiments, the computing device 110 may be communicatively coupled to the failure prediction program 200 and operated at a remote location. The computing device 110 may include internal and external hardware components, which are shown in more detail in FIG. 3.User interface 115 provides an interface to access the features and functions of computing device 110. In some embodiments of the present invention, user interface 115 provides access to execute and select options for risk mitigation program 200, and may also assist in initiating support training of a machine learning model component of failure prediction program 200. User interface 115 may also provide access to and operation of controllers of other applications, features, and functions of computing device 110 (not shown). In some embodiments, user interface 115 provides display output and display input functions to computing device 110. In other embodiments, user interface 115 provides display output and allows selection of options and functions associated with failure prediction program 200 operating in computing device 110.User interface 115 supports access to alerts and notifications and provides access to types of data transmissions. In one embodiment, user interface 115 may be a graphical user interface (GUI) or a web user interface (WUI), and may receive user input and may include text, documents, web browser windows, user options, application interfaces, and instructions for operation, as well as information (such as graphics, text, and sound) that a program presents to a user, as well as the control sequences that the user uses to control the program. In another embodiment, user interface 115 may also include mobile application software that provides respective interfaces to features and functions of computing device 110. The user interface 115 enables respective users of the computing device 110 to receive input, display, listen, and respond to access applications, display content of online conversation dialogs, and perform available functions.The failure prediction program 200 includes the image restoration module 117 and the image similarity module 119. The failure prediction program 200 predicts the operation time from inspection of a plant including acquisition of an image pair of the plant or the components of the plant to a failure state of the plant. The failure prediction program 200 determines the operating time to a failure state of the plant by correlating similarity coefficients of segmented regions of the captured image pair of the plant from an inspection with similarity coefficients of a plurality of historical image pairs of the plant captured over time and including historical image pairs of inspection as well as image pairs of the failure state of the plant.The failure prediction program 200 segments each image of an image pair into "n×n" regions of pixels to obtain a more detailed representation of the plant and components of the plant. The image pairs include a normal image from the human visible light range and an infrared image from the human invisible light range, but indicating heat levels of the equipment and equipment components. The failure prediction program 200 determines similarity coefficients by applying similarity functions to the segmented regions of each pair of images and to corresponding images of the pair of images that represent the plant in a proper state state. The failure prediction program 200 includes the image reconstruction module 117 in which a model is trained to obtain an image pair input from historical or real-time inspection and to reconstruct or reconstruct the received input in a state that reflects the plant in a proper state state. The failure prediction program 200 determines the similarity coefficients between the segmented regions of the received image pair and the reconstructed image pair, and performs the determination of similarity coefficients for the plurality of historical image pairs, respectively, as well as real-time inspection.The failure prediction program 200 includes a second model that calculates the operating time to the failure state of the plant from a real-time inspection, wherein the input of a pair of images includes a time stamp of the acquisition of the pair of images. The plurality of historical image pairs consisting of an infrared image pair and a normal image, as well as the time stamp and information associated with each of the historical image pairs, provide the training data for the second model. Training the second model enables learning of the correlations of temporal inspection plans and states and the similarity coefficients associated with the image pair regions of the inspection with the plurality of historical image pairs and associated temporal plans and states.The image reconstruction module 117 receives the image pairs 120 of a plant in a proper state as training data to perform a reconstruction of received images resulting in image data representing the plant in a proper state state or nearly in a proper state state. In some embodiments, the image reconstruction module 117 includes an autoencoder convolutional neural network model that receives training by inputting selected images that are in a proper state state for the plant 140. In some embodiments, training of the image reconstruction module 117 enables image reconstruction to be performed for both normal images and infrared images. In other embodiments, the image reconstruction module 117 includes a separate trained normal image neural network and a separate trained infrared image neural network. When appropriately trained, the image reconstruction module 117 receives image pairs of the plurality of historical image pairs of the plant 160 from the historical intervention datasets 130, and reconstructs the respective image pairs (or a copy of the respective historical image pair) to generate normal and infrared reference images of the respective historical image pairs. Generating similarity coefficients of respective historical image pairs involves comparing similarity distances between features of image segments of the reconstructed reference images and the historical image pair that reflect the state of the plant 140. The image restoration module 117 receives images of the current equipment inspection 160 to generate a similarity coefficient base for obtaining similarity coefficients of current inspection and enable prediction of an operation time to a failure state of the equipment performed by the failure prediction program 200.The image similarity module 119 calculates similarity coefficients between segmented regions of a single image of one image pair and the corresponding segmented regions of the same image type of another image pair. In some embodiments, the similarity coefficients correspond to a Euclidean distance calculated from the pixels of a region of a first image and the corresponding region of a second image. For example, segmenting a first infrared image of a historical image pair results in each image of the image pair being divided into sixteen regions and inputting a digital copy of the first infrared image regions to the image reconstruction module 117 serving as a first model of the failure prediction program 200. The image reconstruction module 117 reconstructs the input image regions to approximate a proper state state of the plant. The image similarity module 119 determines the Euclidean distances between pixels of the corresponding regions for the first infrared image of the historical image pair and the reconstructed image regions representing a plant in a healthy state. A larger calculated distance between regions reflects a lower similarity and a higher probability that the plant is in a less perfect state.The image pairs 120 of a plant in a proper condition provide a source of selected infrared and normal images of the plant that include a condition that does not provide any evidence of degradation, wear, consumption, damage or other particular characteristics from images of a nearly new plant in operation. The image reconstruction module 117 receives, from the image pairs 120 of a plant in a proper state, image pairs as training data that enables the image reconstruction module 117 to reconstruct received images as infrared and normal images of the plant in a proper state state.The historical intervention datasets 130 include a collection of inspection, maintenance, repair, and replacement activities performed for the plant 140. The historical intervention datasets 130, in some embodiments, include image pairs, a time stamp at which images are captured, also referred to as the time of image capture, as well as data associated with the intervention. In some embodiments, the interventions are associated with a work order or other document that is recorded in a digital format and includes the digitized infrared image, the plant normal image, as well as the activities of the intervention, the duration of the intervention, and the costs associated with the intervention. In some embodiments, a failure condition of the plant results in repair, replacement, or maintenance intervention or other corrective action. In other embodiments, inspection of the plant after a repair intervention and prior to a plant failure condition generates intermediate pair of image and time mark information.The current equipment inspection 140 stands for real-time intervention activity, which includes generation of an image pair including an infrared image and a normal image of the equipment, and a time mark of the image pair acquisition as well as other information related to the inspection are included in a data set of the current equipment inspection. In some embodiments, the failure prediction program 200 processes the pair of images generated by the current plant inspection 140 to predict an operating time to a failure state of the plant from the time frame of the time stamp of the pair of images of the current inspection. In some embodiments, a drone propelled from a remote location performs the real-time equipment inspection of the current equipment inspection 140, thereby preventing a human from being exposed to risky conditions. In other embodiments, one or a set of cameras positioned to capture an infrared image and a normal image of the equipment at certain time intervals provide the real-time inspection image pair to the current equipment inspection 140.The plant 160 performs an operating function on which positive results are dependent. In embodiments of the present invention, operation of the plant 160 creates degradation or consumption of the plant 160 that requires intervention to continue operation of the plant 160. In example embodiments, the plant 160 may be, but is not limited to, an engine, a turbine, a vehicle component, a power generation system, or components of a packet processing plant.FIG. 2 is a flow chart illustrating the operations of the failure prediction program 200 that includes the image reconstruction module 117 and an image similarity module 119 as components operating in the distributed computer computing environment of FIG. 1, in accordance with embodiments of the present invention. Failure prediction program 200 receives a selection of plant perfect condition image pairs. The image pairs of a correct condition contain a normal image and an infrared image of the system.The failure prediction program 200 segments the received perfect state image pairs and a plurality of historical image pairs into regions (step 210). The received good state image pairs reflect a near-new type of equipment state. The acquisition of the plurality of historical image pairs associated with historical interventions including inspection performed between instances of a failure state of the plant and images captured during an intervention to repair or replace the plant occurs at approximately the same time. The image pairs contain a normal image from the visible light range and an infrared image of the system. The historical image pairs of the plant include a time stamp of the image acquisition and information associated with the determined intervention activity.The failure prediction program 200 segments the image pairs into a square matrix of regions characterized by an "n×n" format, where "n" is a positive non-zero integer. Segmenting the images enables a more detailed representation of the installation and installation components. Each region of an image contains a number of pixels and serves as the basis for calculating similarity coefficients by performing a similarity function between a region of an image and the corresponding region of the reconstructed version of the image of the first model (i.e., an image of a plant in a correct state).The failure prediction program 200 includes training a first model to reconstruct the image input to resemble images of the plant in a proper state state (step 220). The failure prediction program 200 includes the image reconstruction module 117 that receives an input of the plant and reconstructs the image based on training that reconstructs a receipt of a selection of images of a proper state of the plant to resemble an image of a proper state of the plant.Thus, training the image reconstruction module 117 includes exclusively using images representing a proper state state of the plant as training data. In some embodiments, the reconstruction function of the image reconstruction module 117 of the failure prediction program 200 generates a plant reference base in a proper state that provides a reference point for determining similarity coefficients of historical image pairs and real-time image pairs from plant inspection. The use of reconstruction of an image of a particular pair of images of the plant, rather than reuse an instance of an image of a plant in a perfect condition, improves the consistency and accuracy of similarity coefficients and prevents additional pixel noise due to differences in exposure, angles and other conditions given in the acquisition of images.The failure prediction program 200 generates similarity coefficients between respective historical image pairs and the corresponding reconstructed historical image pairs (step 230). The failure prediction program 200 applies a similarity function to the segmented regions of each image of the image pair of the plurality of historical image pairs. The similarity function determines a similarity metric between the segmented regions of each image of a respective historical image pair and the segmented regions of each image of the historical image pair that is passed to the first model to reconstruct the image so that it resembles an image of the plant in a proper state state as close as the trained model does so. In some embodiments, determining a Euclidean distance between corresponding regions of the images provides a similarity coefficient metric. In other embodiments, other similarity functions, e.g., a cosine similarity function, provide the similarity coefficient metric. Embodiments of the present invention are not limited by the nature or operation of similarity functions used in determining a similarity coefficient between regions of images. In embodiments of the present invention, the failure prediction program 200 provides similarity coefficients for infrared images and normal images through the image similarity module 119.The failure prediction program 200 trains a second model to predict an operating time until a failure state of the plant (step 240). The failure prediction program 200 uses the time stamp and other information associated with the respective image pairs of the historical interventions of the plant. Similarity coefficients of the plurality of historical image pairs and the time stamp and intervention information associated with a respective historical image pair provide the training basis for the second model to correlate the similarity coefficients and an operating time between the time stamp of the detection of the historical image pair and the time stamp of the next subsequent failure state of the plant. By training the second model with the entire plurality of historical image pairs that include plant failure conditions and plant inspection between failures, the second model correlates between conditions represented by the similarity coefficients of the infrared and normal images and the expected operating time to a plant failure condition.In some embodiments, the additional information associated with historical interventions of the plant, as logged in connection with a work order for the intervention, provides predictions of the duration and cost associated with an intervention performed prior to the predicted failure state of the plant.The failure prediction program 200 receives a real-time image pair of an inspection of the plant and segments the received real-time image pair (step 250). In various embodiments, the plant owners schedule inspection of the plant during operation of the plant. The inspection includes acquisition of an infrared image and a normal image of the equipment forming a real-time image pair associated with the inspection and including a time stamp of the image pair acquisition. The failure prediction program 200 receives the real-time pair of images and continues processing the pair of images to allow prediction of an operating time to a failure state of the plant from the time of image pair detection from the real-time inspection of the plant. The failure prediction program 200 segments the image pairs into regions.The failure prediction program 200 generates real-time similarity coefficients between the real-time image pair and the reconstruction of the image pair from the first model (step 260). The failure prediction program 200 applies a similarity function to the images of the real-time image pair to generate similarity coefficients for the segmented regions. The failure prediction program 200 delivers an infrared image and a normal image as a digitized file to the first model, the image reconstruction module 117, to reconstruct the infrared image and the normal image from the real-time inspection as images corresponding to a proper state status of the equipment. The failure prediction program 200 generates similarity coefficients between the reconstructed image regions that resemble the plant with a proper state status and the regions of the real-time infrared image and normal image of the inspection image pair.The failure prediction program 200 determines a predicted operating time until a failure state of the plant and presents the predicted operating time in a notification (step 270). The failure prediction program 200 provides the generated similarity coefficients to the trained second model that correlates the similarity coefficients and time stamp of the regions of the real-time image pair with the similarity coefficients of the regions of the historical image pair and time stamps used to train the second model. The failure prediction program 200 determines a prediction of the operating time from the time of acquiring the real-time inspection image pair to the expected failure state of the plant. In some embodiments of the present invention, the failure prediction program 200 includes the prediction duration until a failure state of the plant in a notification presented (e.g., on a display). In some embodiments, the failure prediction program 200 presents the notification to one or more designated recipients by a predetermined manner of provision, such as an SMS message, an e, a pop-up display on a monitor associated with the plant, or other medium for providing a notification.After the predicted operating time until a plant failure condition is determined and the predicted operating time is submitted in a notification, the failure prediction program 200 ends. To further clarify aspects of the present invention, use cases are provided that describe embodiments of the present invention.In one embodiment, an inspection is performed on an autonomous elevator and yields a pair of images including a visible light range image and an infrared image. Embodiments of the present invention do without a repetitive need for repetitive human inspection and thus prevent excessive down time. A smaller number of human inspection in elevator shafts also reduces the risk for humans and accidents associated with personal inspection, since images can be recorded by mounted camera systems or camera recording of plant images from safe or remote locations can be carried out. Predicting the operating time up to a failure state of the plant reduces or eliminates inspection and maximizes the operating time by precisely planning maintenance interventions.Embodiments of the present invention provide a direct advantage for autonomous elevator and travel walkway inspection. Installation of image capture sensors / cameras inside the facilities allows precise prediction of an operating time to a failure state of the facilities and enables efficient scheduling of activities of maintenance intervention. As a result, operating time and costs are saved and safe conditions for human interactions are supported.Trains, subway, buses, and trucks may use scheduled stops as a way to capture images of critical plant components and enable precise scheduled maintenance activity prior to predicted downtime. Scheduled maintenance relies on evidence-based and machine learning data rather than regular time intervals or distance travelled, which allows more efficient operation in the vehicle while preventing plant and / or plant component failures from occurring.FIG. 3 illustrates a block diagram of components of a data processing system that includes a data processing unit configured to operatively perform the failure prediction program including the image reconstruction module and the image similarity module of FIG. 2, in accordance with an embodiment of the present invention.A data processing function 305 includes components and functionality similar to the components of the data processing unit 110 (FIG. 1 ), in accordance with an illustrative embodiment of the present invention. It should be appreciated that FIG. 3 provides merely an illustration of an embodiment and does not imply any limitations with respect to the environments in which various embodiments may be implemented. Rather, numerous modifications may be made to the environment illustrated.Computing device 305 includes a communication structure 302 that provides communication between one or more computer processors 304, memory 306, persistent storage 308, communication device 310, and one or more input / output (I / O) interfaces 312. The communication fabric 302 may be implemented with any architecture designed to pass data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripherals, and any other hardware components within a system. The communication structure 302 may be implemented using one or more buses, for example.The memory 306, cache memory 316, and persistent storage 308 are computer readable storage media. In this embodiment, memory 306 includes random access memory (RAM) 314. Generally, memory 306 may include any suitable transitory or non-transitory computer readable storage media.In one embodiment, the failure prediction program 200 is stored in the persistent storage 308 to be executed by one or more of the respective computer processors 304 via one or more memories of the memory 306. In this embodiment, persistent storage 308 includes a magnetic hard disk drive. Alternatively or in addition to a magnetic hard disk drive, persistent memory 308 may include a semiconductor hard disk, a semiconductor memory unit, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), flash memory, or any other computer readable storage medium capable of storing program instructions or digital information.The media used by the persistent storage 308 may also be removable. For example, a removable hard disk drive may be used as the persistent storage 308. Other examples include optical and magnetic disks, USB sticks, and smart cards inserted into a drive to transfer to another computer readable storage medium that is also part of persistent memory 308.The communication device 310, in these examples, provides communication with other computing systems or devices, e.g., with resources of the distributed computing computing device 100. In these examples, communication device 310 includes one or more network interface cards. Communication device 310 may provide communication over both physical and wireless communication links. The failure prediction program 200 may be downloaded to the persistent storage 308 by the communication unit 310.The one or more I / O interfaces 312 enable input and output of data to or from other devices that may be connected to the data processing system 300. For example, the I / O interface 312 may provide a connection to the external devices 318, such as a keyboard, keypad, touch screen, and / or other suitable input device. The external devices 318 may also include portable computer readable storage media such as USB sticks, portable optical or magnetic disks, and memory cards. Software and data for practicing embodiments of the present invention, such as failure prediction program 200, may be stored on such portable computer readable storage media and loaded into persistent memory 308 via the one or more I / O interfaces 312. The one or more I / O interfaces 312 are also connected to a display 320.The display 320 provides a mechanism for displaying data to a user, and may be, for example, a computer monitor.The above description is an example of embodiments of the invention, and variations and substitutions can be made on the implementations without departing from the novel aspects of the embodiments.The programs described herein are identified based on the application for which they are implemented in a specific embodiment of the invention. However, it should be understood that any specific program nomenclature referred to herein is used for purposes of simplicity only and that the invention is not to be limited to being used exclusively in any specific application specified and / or implied by such nomenclature.The present invention may be a system, method and / or computer program product with any possible degree of technical integration. The computer program product may include a computer readable storage medium (or computer readable storage media) having computer readable program instructions stored thereon for causing a processor to carry out aspects of the present invention.The computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage unit, a magnetic storage unit, an optical storage unit, an electromagnetic storage unit, a semiconductor storage unit, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable CD-ROM, a DVD (Digital Versatile Disk), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions stored thereon, and any suitable combination thereof. A computer readable storage medium, as used herein, is not intended to be transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses conducted through an optical fiber cable), or electrical signals transmitted through a wire.Computer readable program instructions described herein may be downloaded from a computer readable storage medium to respective computing / processing units or to an external computer or storage unit via a network such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network. The network may include copper transmission cables, lightwave transmission conductors, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing unit receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the corresponding computing / processing unit.Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or both source code and object code written in any combination of one or more programming languages, including object oriented programming languages such as Smalltalk, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGA) or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuits to perform aspects of the present invention.Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It is noted that each block of the flowcharts and / or the block diagrams, as well as combinations of blocks in the flowcharts and / or the block diagrams, may be executed by computer readable program instructions.These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions / steps specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored on a computer readable storage medium that can control a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium on which instructions are stored comprises an article of manufacture including instructions that implement aspects of the function / step specified in the flowchart and / or block diagram block or blocks.The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of process steps to be performed on the computer or other programmable apparatus or device to produce a computer implemented process such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / steps specified in the flowchart and / or block diagram block or blocks.The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions that include one or more executable instructions for executing the particular logical function or functions. In some alternative implementations, the functions specified in the block may occur in a different order than shown in the figures. For example, two blocks shown in succession may in fact be performed as one step, performed simultaneously, substantially simultaneously, partially, or fully overlapping in time, or the blocks may sometimes be performed in the reverse order depending on the appropriate functionality. It is further noted that each block of the block diagrams and / or flowcharts, as well as combinations of blocks in the block diagrams and / or flowcharts, may be implemented by special purpose hardware-based systems that perform the specified functions or steps, or perform combinations of special purpose hardware and computer instructions.

Claims

A computer-implemented method for predicting an operating time to a failure condition of a plant (140), the computer-implemented method comprising: receiving, by one or more processors (304), a plurality of historical image pairs of a plant (140), such as image pairs of a failure condition of the plant (140), wherein respective image pairs include an infrared image and a corresponding normal image; receiving, by the one or more processors (304), a first model trained to reconstruct respective images of one of the plant (140) image pairs of the plant (140) in a proper condition state (120) of the plant (140); generating a similarity coefficient between a historical image pair and a reconstruction of the historical image pair by the first model for the plurality of historical image pairs by the one or more processors (304), the historical image pair including a time stamp of a detection of the historical image pair; receiving a second model by the one or more processors (304) trained to predict an operating time until a failure condition of the plant (140) occurs based on the similarity coefficients and a time stamp of a detection of the plurality of historical image pairs, respectively; in response to receiving, by the one or more processors (304), a real-time image pair of the plant (140), generating real-time similarity coefficients between the real-time image pair and the real-time image pair reconstructed by the first model, the real-time image pair including a time stamp of acquisition of the real-time image pair; and calculating, by the one or more processors (304), a predicted operating time until the failure state of the plant (140) based on a correlation between the similarity coefficients of the real-time image pair and the time mark of the acquisition of the real-time image pair and a historical image pair of the plurality of historical image pairs having corresponding similarity coefficients and a time delta determined from the time mark of the acquisition of the real-time image pair and a time mark of the failure state of the plant (140) after the historical image pair having the corresponding similarity coefficients.The computer-implemented method of claim 1, wherein the first model is a set of autoencoder convolutional networks that receives infrared images and normal images of the plant (140) as an input.The computer-implemented method of claim 1, further comprising: segmenting, by the one or more processors (304), the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the asset (140) into regions; and generating, by the one or more processors (304), similarity coefficients for the regions of the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the asset (140).The computer-implemented method of claim 3, wherein the regions are segmented into a square matrix of regions.The computer-implemented method of claim 1, wherein the similarity coefficients include a multi-dimensional vector of a Euclidean distance between segmented regions of respective images of the image pair.The computer-implemented method of claim 1, wherein the first model and the second model further comprise: receiving, by one or more processors (304), a sample from a plant (140), image pairs of a plant (140) having a proper state status (120) from the plurality of historical image pairs of the plant (140), e.g., a span of a state of the plant (140) from a re-exchanged state to a failure state of the plant (140); training, by the one or more processors (304), the first model to reconstruct respective images of an image pair based on training data from the sample from the plant (140), image pairs of the plant (140) having the proper state status (120); generating, by the one or more processors (304), a second model to predict an operating time until the failure state of the plant (140); and training, by the one or more processors (304), the second model based on the similarity coefficients of the plurality of historical image pairs of the plant (140) and a time stamp of a detection of the plurality of historical image pairs, respectively.The computer-implemented method of claim 1, wherein the plurality of received historical image pairs of the plant (140) includes a range of states between the proper state status and the failure state of the plant (140).A computer program product for predicting an operating time to a failure condition of a plant (140), the computer program product comprising: one or more computer readable storage media and program instructions stored in common on the one or more computer readable storage media, the program instructions comprising: program instructions for receiving a plurality of historical image pairs of a plant (140), such as image pairs of a failure condition of the plant (140), respective image pairs including an infrared image and a corresponding normal image; program instructions for receiving a first model trained to reconstruct respective images of one of the plant (140) image pairs of the plant (140) in a proper condition state (120) of the plant (140); Program instructions for generating a similarity coefficient between a respective historical image pair and a reconstruction of the historical image pair by the first model for the plurality of historical image pairs, the historical image pair including a time stamp of a detection of the historical image pair; program instructions for receiving a second model trained to predict an operating time until a failure state of the plant (140) occurs based on the respective similarity coefficients and a time stamp of a detection of the plurality of historical image pairs; in response to receiving a real-time image pair of the plant (140), program instructions for generating real-time similarity coefficients between the real-time image pair and the real-time image pair reconstructed by the first model, the real-time image pair including a time stamp of a detection of the real-time image pair; and program instructions for calculating a predicted operating time until the failure state of the plant (140) based on a correlation between the similarity coefficients of the real-time image pair and the time mark of the acquisition of the real-time image pair and a historical image pair of the plurality of historical image pairs having corresponding similarity coefficients and a time delta determined from the time mark of the acquisition of the real-time image pair and a time mark of the failure state of the plant (140) after the historical image pair having the corresponding similarity coefficients.The computer program product of claim 8, wherein the first model is a set of autoencoder convolutional networks that receives infrared images and normal images of the plant (140) as an input.The computer program product of claim 8, further comprising: program instructions for segmenting the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the plant (140) into regions; and program instructions for generating similarity coefficients for the regions of the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the plant (140).The computer program product of claim 10, wherein the regions are segmented into a square matrix of regions.The computer program product of claim 8, wherein the similarity coefficients include a multi-dimensional vector of a Euclidean distance between segmented regions of respective images of the image pair.The computer program product of claim 8, wherein the first model further comprises: program instructions for receiving a sample from a plant (140), image pairs of a plant (140) having a proper state status (120), from the plurality of historical image pairs of the plant (140), e.g., a span of a state of the plant (140) from a re-exchanged state to a failure state of the plant (140); program instructions for training the first model to reconstruct respective images of an image pair based on training data from the sample from the plant (140), image pairs of the plant (140) having the proper state status (120); program instructions for generating a second model to predict an operating time to the failure state of the plant (140); and program instructions for training the second model based on the similarity coefficients of the plurality of historical image pairs of the plant ( 140) and a time stamp of a detection of the plurality of historical image pairs, respectively.The computer program product of claim 8, wherein the plurality of received historical image pairs of the plant (140) includes a range of states between the proper state state and the failure state of the plant (140).A computer system for predicting an operating time to a failure condition of a plant (140), the computer system comprising: one or more processors (304); one or more computer readable storage media; and program instructions stored in common on the one or more computer readable storage media, the program instructions comprising: program instructions for receiving a plurality of historical image pairs of a plant (140), such as image pairs of a failure condition of the plant (140), respective image pairs including an infrared image and a corresponding normal image; program instructions for receiving a first model trained to reconstruct respective images of one of the plant (140) image pairs of the plant (140) in a proper condition state (120) of the plant (140); program instructions for generating a similarity coefficient between a historical image pair and a reconstruction of the historical image pair by the first model for the plurality of historical image pairs of the plant (140), the historical image pair including a time stamp of a detection of the historical image pair; program instructions for receiving a second model trained to predict an operating time until a failure state of the plant (140) occurs based on the similarity coefficients and a time stamp of a detection of the plurality of historical image pairs, respectively; in response to receiving a real-time image pair of the plant (140), program instructions for generating real-time similarity coefficients between the real-time image pair and the real-time image pair reconstructed by the first model, the real-time image pair including a time stamp of acquisition of the real-time image pair; and program instructions for calculating a predicted operating time until the failure state of the plant (140) based on a correlation between the similarity coefficients of the real-time image pair and the time mark of the acquisition of the real-time image pair and a historical image pair of the plurality of historical image pairs of the plant (140) having corresponding similarity coefficients and a time delta determined from the time mark of the acquisition of the real-time image pair and a time mark of the failure state of the plant (140) after the historical image pair having the corresponding similarity coefficients.The computer system of claim 15, wherein the first model is a set of autoencoder convolutional networks that receives infrared images and normal images of the plant (140) as an input.The computer system of claim 15, further comprising: program instructions for segmenting the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the plant (140) into regions, wherein the regions are segmented into a square matrix of regions; and program instructions for generating similarity coefficients for the regions of the sample of image pairs, the plurality of historical image pairs, and the real-time image pairs of the plant (140).The computer system of claim 17, wherein the first model further comprises: program instructions for receiving a sample from a plant (140), image pairs of a plant (140) having a proper state status (120) from the plurality of historical image pairs of the plant (140), e.g., a span of a state of the plant (140) from a re-exchanged state to a failure state of the plant (140); program instructions for training the first model to reconstruct respective images of an image pair based on training data from the sample from the plant (140), image pairs of the plant (140) having the proper state status (120); program instructions for generating a second model to predict an operating time to the failure state of the plant (140); and program instructions for training the second model based on the similarity coefficients of the plurality of historical image pairs of the plant ( 140) and a time stamp of a detection of the plurality of historical image pairs, respectively.The computer system of claim 15, wherein the similarity coefficients include a multi-dimensional vector of a Euclidean distance between segmented regions of respective images of the image pair.The computer system of claim 15, wherein the plurality of received historical image pairs of the plant (140) includes a range of states between the proper state status and the failure state of the plant (140).

Images