Joint prediction of repair time and measures from text failure reports based on ai

Abstract

A system applies an AI-based model pipeline for joint prediction of repair time and repair actions based on received fault reports related to manufacturing automation machines and devices. The model pipeline includes a first encoder configured as a Transformer neural network trained to transform fault report text into latent fault report representations that group semantically similar fault texts. Another Transformer encoder is trained to transform the latent fault report representations into latent repair report representations. A repair time prediction module with a feedforward neural network is trained to generate repair time predictions from the latent repair report representations against actual repair times. A repair action prediction module configured as a Transformer decoder neural network is trained to generate repair action predictions from the latent repair report representations against actual repair actions. End-to-end training of the pipeline is achieved by minimizing a combined loss function of the joint predictions.

Description

Technical Field

[0001] This application relates to artificial intelligence (AI). More specifically, this application relates to the application of AI-based methods to jointly predict maintenance times and maintenance measures for machinery and equipment in automated systems from text-based fault reports. Background Technology

[0002] Machines and other equipment used for manufacturing automation require regular maintenance, but even with regular maintenance, malfunctions are inevitable at some point due to wear, tear, or other reasons. At the production plant level, effective maintenance planning and scheduling are crucial to ensure minimal downtime and maintain high levels of overall equipment efficiency (OEE) during and after a failure.

[0003] The ability to execute effective maintenance planning hinges on accurately predicting the time and type of maintenance required when a failure occurs. Accurate maintenance time estimates enable the planning team to decide whether production should be diverted to alternative machines or production lines, and what other remedial measures should be taken. Recommending potential maintenance actions helps the maintenance team prepare for and execute maintenance more efficiently (e.g., by ordering parts in advance or bringing the correct maintenance equipment). Especially for mechanical maintenance, knowing in advance which equipment may or may not be necessary can save time and resources. Therefore, the technical challenge is to predict maintenance time (in hours) and required maintenance actions based on text-based failure reports.

[0004] While several AI-based approaches have been proposed to address any of these problems, existing methods do not account for the correlation between the two expected predictions, thus failing to provide accurate and consistent forecasts in many cases. For example, solving each problem's predictions in parallel can lead to inconsistent results, such as predicting a rapid repair time for a complex maintenance procedure. Such errors may arise because each prediction pipeline relies on a different data domain.

[0005] Predictive maintenance is an increasingly prevalent AI-based approach used to predict machine failures before they occur, based on the machine's history, including information such as current uptime and previous failure history. Predicting failures before they happen is a more challenging problem and is prone to producing more uncertain results.

[0006] In most cases, repair time and action prediction are currently addressed manually by experts evaluating failure reports and making estimates. However, this approach requires significant manpower and can become a bottleneck in production environments with a large number of devices and failure cases. Furthermore, to make accurate estimates, experts need considerable experience and familiarity with a wide variety of equipment types.

[0007] In addition to manually predicting maintenance time and measures, several AI-based methods have been proposed in the literature. However, these methods only address one problem at a time (maintenance time prediction or maintenance measure prediction), which may lead to inconsistent and therefore inaccurate predictions. Summary of the Invention

[0008] In contrast to the conventional methods described above, this disclosure proposes a joint prediction approach that jointly predicts maintenance times and maintenance actions within a single AI-based pipeline. By doing so, the proposed framework can leverage the relationship between maintenance times and maintenance actions. For example, a predicted maintenance time might exclude several maintenance actions that might require longer durations. Furthermore, knowing which maintenance actions might be necessary allows for more accurate predictions of maintenance times.

[0009] In one aspect, a system employs an AI-based model pipeline for jointly predicting repair times and maintenance actions based on received fault reports related to manufacturing automation machines and equipment. The model pipeline includes a first encoder configured as a Transformer neural network trained to transform fault report text into a potential fault report representation that groups semantically similar fault text. Another Transformer encoder is trained to transform the potential fault report representation into a potential maintenance report representation. A maintenance time prediction module with a feedforward neural network is trained to generate maintenance time predictions from the potential maintenance report representations against actual maintenance times. A maintenance action prediction module configured as a Transformer decoder neural network is trained to generate maintenance action predictions from the potential maintenance report representations against actual maintenance actions. End-to-end training of the pipeline is achieved by minimizing the combined loss function of the joint predictions. Attached Figure Description

[0010] Non-limiting and non-exhaustive embodiments of this disclosure are described with reference to the following accompanying drawings, wherein, unless otherwise specified, the same reference numerals refer to the same elements throughout the drawings.

[0011] Figure 1 An example of an AI-based pipeline for jointly predicting maintenance times and maintenance measures for machinery in an automated system during pre-training, according to an embodiment of this disclosure, is shown.

[0012] Figure 2 A flowchart example of the pre-training and training process of an AI-based pipeline for jointly predicting maintenance time and maintenance measures of machinery in an automated system, according to an embodiment of the present disclosure, is shown.

[0013] Figure 3An example of an AI-based pipeline for jointly predicting maintenance times and maintenance actions for machinery in an automated system during end-to-end training, according to an embodiment of this disclosure, is shown.

[0014] Figure 4 An example of an AI-based pipeline for jointly predicting maintenance times and maintenance measures for machinery in an automated system, according to embodiments of the present disclosure, is shown.

[0015] Figure 5 Examples of computing environments in which embodiments of the present disclosure can be implemented. Detailed Implementation

[0016] Figure 1 An example framework for an AI-based model pipeline for jointly predicting maintenance time and maintenance actions based on fault reports related to machinery in an automated system, according to embodiments of this disclosure, is shown. The innovations provided by the disclosed embodiments include building a model capable of learning to make accurate predictions of maintenance time and maintenance actions, both based on the same underlying representation of the fault report text. Figure 1 As shown, model pipeline 100 is configured as an AI-based pipeline, comprising encoders 101, 102, and 103, a Time-to-Protect (RTP) module 111, and a Remedial Action Prediction (RAP) module 112, each trained as part of a multi-step process. Each of encoders 101, 102, and 103 is configured as a Transformer neural network model capable of automatically transforming one type of input into another type of output. These encoders encode input text words into vectors using embedding layers and Transformer layers. RTP module 111 is configured as a simple feedforward neural network. RAP module 112 is configured as a Transformer neural network decoder.

[0017] The source information of the trained model pipeline 100 is historical fault report text presented in natural language. x (i.e., a sequence of language tokens). Typical fault report information includes a description of the fault (e.g., symptoms) and a description of the affected machine or component. Basic preprocessing (e.g., stop word removal and / or stemming) can be performed on this text of the fault report text x.

[0018] The dependencies for pre-training the Transformer neural network during the initialization of model pipeline 100 are shown by arrows. During the initialization and pre-training phases, other given inputs include a set of historical fault cases. For each fault case, accompanying data includes a fault report and a corresponding repair report generated from the response. Therefore, in addition to the fault text... xIn addition, the actual repair time is also provided. And natural language descriptions of the maintenance measures performed, obtained from the corresponding maintenance report. Similar to error text x Some basic text preprocessing is applied to the description of maintenance measures.

[0019] Figure 2 A flowchart example of the pre-training and training process of an AI-based pipeline for jointly predicting maintenance time and maintenance measures of machinery in an automated system, according to an embodiment of the present disclosure, is shown. Figure 2 The following steps are as follows: Figure 1 and Figure 3 The pipeline 100 shown is used to describe the components.

[0020] The initialization of model pipeline 100 begins with the pre-training of Transformer encoders 101, 102, and 103 and Transformer decoder RAP 112. At 201, encoder 101 is initialized using publicly available pre-trained Transformer models (e.g., BERT, RoBERTa), which are already capable of learning semantic representations of general text. Then, historical fault texts are processed... x The encoder is fine-tuned on a set of data in a self-supervised manner, where encoder 101 will output fault text. x Mapping to the notation The semantic embedding vectors are latent numerical vector representations of the fault report text. Here, "numerical" vectors refer only to real-valued vectors. The Transformer encoder 101 takes the representation of the input text (basically a sequence of tokens) and maps it to real-valued embedding vectors. The similarity between these vectors (e.g., based on Euclidean distance) corresponds to the semantic similarity of the texts corresponding to these vectors. Therefore, the encoder 101 uses the numerical vectors to group semantically similar fault texts within the learned latent space. Accordingly, the encoder 101 performs the transformation of the input text into latent vector representations. h 1.

[0021] At point 202, encoder 103 is pre-trained as a Transformer model, similar to encoder 101, except that the training data for self-supervised fine-tuning is a set of historical maintenance action descriptions obtained from maintenance reports. Instead of fault reports, historical maintenance action descriptions correspond to fault report texts because for each historical fault text, there exists a corresponding maintenance report. During pre-training, the output of encoder 103 is a latent text vector representation of the maintenance report, denoted as... Accordingly, encoder 103 executes the input text... Transform into latent vector representation h 2.

[0022] In this implementation, encoder 101 and encoder 103 are pre-trained independently and in parallel.

[0023] At 203a, the vector generated by encoder 101 is used. h 1 and the vector generated by encoder 103 h 2. Pre-train encoder 102 to process specific historical fault text (for specific fault reports). x Embedded vector h 1. Mapping to the embedding vector of the corresponding historical maintenance report h 2. The transformation learned by encoder 102 is a mapping from one latent vector space to another. In the transformed latent vector space, fault texts with similar repair times and repair measures are ultimately grouped together.

[0024] During the pre-training of encoder 102 in step 203a, it is important to maintain the correspondence between fault texts belonging to the same specific fault case and related maintenance report information. For a specific set (or batch) of historical cases, encoder 101 embeds the fault text, and encoder 102 embeds the maintenance report. This process can be performed in parallel. Alternatively, the embeddings can be pre-computed after the pre-training of encoders 101 and 103 is completed and before the pre-training of encoder 102 begins. Encoder 102 is then trained to map the embedded fault texts to their corresponding embedded fault reports.

[0025] At step 203b, based on the potential maintenance report vector from the pre-trained encoder 103 h The learned repair measures embedded in section 2 are used to pre-train the decoder of RAP module 112. The goal of pre-training RAP module 112 is to reliably represent potential repair reports. h 2. The mapping is returned to the natural text description of the maintenance measures. Based on experience, the model with the lowest complexity was determined for RAP module 112 while still reliably performing this mapping. In the implementation, refer to... Figure 1 The pre-training of RAP module 112 begins with the text of actual maintenance procedures. The latent representation is fed into the pre-trained encoder 103 to obtain the latent representation. h 2. The latent representation is then fed into the decoder RAP module 112 to learn how to decode the embedded maintenance action text into the predicted maintenance action text. y 2.

[0026] Once the pre-training used for initialization is complete, at step 204, end-to-end training of the model pipeline 100 is performed for the joint prediction objective. The training process is as follows: Figure 3 As shown. The training process updates all model parameters except those of encoders 101 and 103. The parameters of encoder 101 are frozen after pre-training, and encoder 103 is no longer needed after model initialization. Training is conducted in a supervised manner based on actual repair times and repair measures from historical case data. The combined loss function of joint prediction is minimized using the backpropagation algorithm. During forward propagation, fault texts used as training inputs are processed using encoders 101 and 102 respectively. x Mapped to its potential fault report h 1 and potential repair reports indicate h 2. Based on the potential repair report. h 2. Train the feedforward neural network of RTP module 111 to predict maintenance time. y 1 as a scalar output The predicted repair time y 1. Actual repair time By comparing, we can obtain the first loss. L 1. Additionally, according to the same potential repair report... h 2. Use the Transformer decoder of RAP module 112 to generate a predicted description of maintenance measures. y 2. And describe it in conjunction with the actual maintenance measures. By comparison, a second loss is obtained. L 2. The training of RAP module 112 is controlled to avoid over-parameterization of the Transformer neural network decoder. In this implementation, a combined loss function is used. L Combine the individual loss functions. Individual loss for each prediction. L 1 and L The balance of 2 can be achieved through hyperparameters. For example, the combined loss function L can be expressed as follows: , Among them, cross-entropy loss l This is a description of the predicted maintenance measures. y 2. Description of actual maintenance measures The bag-of-words encoding representation is computed, and the hyperparameter α is balanced for maintenance time prediction. y 1. Predictions for maintenance measures y 2. Two loss functions. By minimizing the combined loss function. L The model learns the latent representation of fault text, enabling consistent and highly accurate prediction of repair time and repair measures.

[0027] Figure 4 An example of an AI-based pipeline trained according to an embodiment of this disclosure for jointly predicting maintenance times and maintenance actions for machinery in an automated system is shown. After the model pipeline 100 is trained, joint prediction for new cases is performed by using encoders 101 and 102 to process fault text... x Mapped to its potential fault report h 1 and potential repair reports indicate h 2 (e.g.) Figure 4 (as shown), then h 2. Feeds are sent separately through RTP module 111 and RAP module 112 to jointly obtain the maintenance time. y 1. Description of measures y The prediction for 2.

[0028] Figure 5 An example of a computer environment in which embodiments of the present disclosure may be implemented is shown. The computing device 510 includes one or more processors 515 and a memory 511 (e.g., a non-transitory computer-readable medium) on which various computer applications, modules, or executable programs are stored. The processor 515 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other processor known in the art. In this embodiment, encoders 101, 102, 103, and RAP decoder 112 are stored on the GPU and trained / executed thereon to achieve accelerated processing.

[0029] More generally, a processor as described herein is an apparatus for executing machine-readable instructions stored on a computer-readable medium for performing tasks, and may include any one or a combination of hardware and firmware. A processor may also include memory storing machine-readable instructions executable for performing tasks. A processor acts on information by manipulating, analyzing, modifying, transforming, or transferring information for use by an executable program or information apparatus, and / or by routing information to an output device. A processor may use or include the capabilities of, for example, a computer, controller, or microprocessor, and may be modulated using executable instructions to perform special functions not performed by a general-purpose computer. A processor may include any type of suitable processing unit, including but not limited to a central processing unit, microprocessor, reduced instruction set computer (RISC) microprocessor, complex instruction set computer (CISC) microprocessor, microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SoC), digital signal processor (DSP), etc. Furthermore, the processor 520 can have any suitable microarchitecture design, including any number of components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read / write operations on cache memory, branch predictors, etc. The processor's microarchitecture design can be capable of supporting any instruction set across a wide range of instruction sets. The processor can be coupled (electrically and / or including executable components) to any other processor, enabling interaction and / or communication between them. The user interface processor or generator is a known element including electronic circuitry or software, or a combination of both, for generating display images or portions thereof. The user interface includes one or more display images, enabling the user to interact with the processor or other device.

[0030] In this embodiment, the memory 511 includes one or more of the following modules: Transformer encoders 101, 102, 103; RTP module 111; and Transformer decoder RAP module 112. Figure 5 As shown, as an alternative computer implementation of encoders 101, 102, 103, RTP 111, and RAP 112, one or more such modules can be deployed as cloud-based or network-based operations in one or more remote modules 541, or deployed as a partitioned operation shared by local modules 101, 102, 103, 111, 112 and network-based modules 541.

[0031] Network 560 (such as a local area network (LAN), wide area network (WAN), or Internet-based network) connects training data 551 to modules 101, 102, 103, 111, 112, and modules 541, 542 of computing device 510. Training data 551 corresponds to historical fault reports and maintenance reports, as well as latent representations generated during the initialization of model pipeline 100. h 1. h 2.

[0032] User interface module 514 provides an interface between modules 101, 102, 103, 111, 112 and user interface devices 530 (such as display device 531 and user input device 532). GUI engine 513 drives the interactive user interface to be displayed on display device 531, thereby allowing users to receive visualizations of analysis results and assisting users in inputting learning objectives and domain constraints of modules 101, 102, 103, 111, 112, 541.

[0033] Computer-readable medium instructions used to perform the operations of this disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, as part of a standalone software package on the user's computer, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet through an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute the computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry for performing aspects of this disclosure.

[0034] This document describes aspects of the disclosure with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented using computer-readable medium instructions.

[0035] Figure 5The program modules, applications, computer-executable instructions, code, etc., described as stored in system memory 511 are merely illustrative and not exclusive, and are described as being supported by any particular module, which may alternatively be distributed across multiple modules or executed by different modules. Additionally, various program modules, scripts, plug-ins, application programming interfaces (APIs), or any other suitable computer-executable code may be provided locally hosted on computer system 510, remote network devices of storage modules 541, 542, and / or hosted on one or more other accessible computing devices via network 560 to support the functionality and / or additional or alternative functionality provided by the program modules, applications, or computer-executable code. Furthermore, functionality may be modularized in different ways, such that what is described as being supported by... Figure 5 The processing collectively supported by the set of program modules described herein can be executed by fewer or more modules, or functionality described as being supported by any particular module can be at least partially supported by another module. Furthermore, the program modules supporting the functionality described herein can form parts of one or more applications executable across any number of systems or devices according to any suitable computational model (such as, for example, client-server models, peer-to-peer models, etc.). Additionally, the processing described as being supported by... Figure 5 Any functionality supported by any program modularity described herein can be implemented, at least in part, in the hardware and / or firmware across any number of devices.

[0036] It should also be understood that, without departing from the scope of this disclosure, computer system 510 may include alternative and / or additional hardware, software, or firmware components other than those described or depicted. More specifically, it should be understood that the software, firmware, or hardware components depicted as forming part of computer system 510 are merely illustrative, and some components may be absent or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory 511, it should be understood that functionality described as supported by program modules can be implemented by any combination of hardware, software, and / or firmware. It should also be understood that, in various embodiments, each of the modules mentioned above may represent a logical division of supported functionality. Such logical division is depicted for ease of interpretation of functionality and may not represent the structure of the software, hardware, and / or firmware used to implement functionality. Accordingly, it should be understood that, in various embodiments, functionality described as provided by a particular module may be provided at least partially by one or more other modules. Furthermore, in some embodiments, one or more depicted modules may be absent, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and / or additional functionality. Furthermore, while some modules may be described and depicted as submodules of another module, in some implementations, such modules may be provided as independent modules or submodules of other modules.

[0037] While specific embodiments of this disclosure have been described, those skilled in the art will recognize that numerous other modifications and alternative implementations are within the scope of this disclosure. For example, any functionality and / or processing capability described with respect to a particular device or component can be performed by any other device or component. Furthermore, while various illustrative implementations and architectures have been described according to embodiments of this disclosure, those skilled in the art should understand that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. Additionally, it should be understood that any operation, element, component, data, etc., described herein as being based on another operation, element, component, data, etc., may be additionally based on one or more other operations, elements, components, data, etc. Therefore, the phrase "based on" or variations thereof should be interpreted as "at least partially based on".

[0038] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may not occur in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

Claims

1. A computer-based system for jointly predicting maintenance times and maintenance actions for machines and equipment used in manufacturing automation based on received fault reports, said system comprising: processor; as well as A non-transitory memory storing modules executed by the processor, the modules comprising: AI-based model pipeline, including: The first encoder is configured as a Transformer neural network, which is trained to transform fault report text into a potential fault report representation that groups semantically similar fault texts. The second encoder is configured as a Transformer neural network, which is trained to transform the potential fault report representation into a potential maintenance report representation. The maintenance time prediction module is configured as a feedforward neural network, which is trained to generate maintenance time predictions from the potential maintenance report representation using a first loss function based on the actual maintenance time. The maintenance action prediction module is configured as a Transformer decoder neural network, which is trained to generate maintenance action predictions from the potential maintenance report representation using a second loss function based on actual maintenance actions. The model pipeline is trained end-to-end by minimizing the combined loss function of the first loss function and the second loss function.

2. The system according to claim 1, further comprising: The third encoder is configured as a Transformer neural network, which is pre-trained to transform a set of historical maintenance action descriptions into a latent text vector representation of maintenance reports. The second encoder is pre-trained by mapping the potential fault report representation to the potential maintenance report representation.

3. The system according to claim 2, wherein, The third encoder is excluded from end-to-end training of the model pipeline.

4. The system according to claim 2, wherein, The maintenance action prediction module is pre-trained using the latent text vector representation of maintenance reports from a pre-trained third encoder.

5. The system according to claim 1, wherein, The first encoder is pre-trained to learn semantic representations of general text and fine-tuned in a self-supervised manner on a set of historical fault texts.

6. The system according to claim 1, wherein, The parameters of the first encoder remain frozen during end-to-end training of the model pipeline.

7. The system according to claim 1, wherein, The combined loss function uses hyperparameters and cross-entropy loss calculated on bag-of-words encoded representations of the predicted and actual maintenance action descriptions to balance the first and second loss functions.

Images