Method, apparatus and system for performing predictive maintenance of monitored systems

Abstract

The present application relates to a computer-implemented method, which includes: receiving a predictive telemetry data object from a query initiating device configured to display a user interface, the predictive telemetry data object being associated with a plurality of predictive measurement values, each of the plurality of predictive measurement values ​​being associated with a sensor device among one or more sensor devices in a monitored system; generating a maintenance prediction for the associated sensor device based on the predictive telemetry data and one or more models of the monitored system; generating one or more maintenance requirement notification outputs based on the maintenance prediction; and causing the query initiating device to display the one or more maintenance requirement notification outputs using the user interface.

Description

Background Art

[0001] Various methods, apparatus, and systems are configured to provide techniques for performing predictive maintenance data based on telemetry data. Summary of the invention

[0002] In general, embodiments disclosed herein provide methods, apparatuses, systems, computing devices, etc. configured to enable predictive maintenance data to be performed based on telemetry data. For example, certain embodiments disclosed herein provide methods, apparatuses, systems, computing devices, etc. configured to perform predictive maintenance data based on telemetry data using at least one of a predictive maintenance convolutional neural network, a maintenance data object, a predictive input channel per sensor, and a heat map per input channel.

[0003] According to one aspect, a method is provided. In one embodiment, the method includes: obtaining a training telemetry data object for a monitored system; determining a training input data object based on the training telemetry data object, wherein the training input data object includes one or more training input channels, and wherein each of the one or more training input channels is associated with a sensor device in one or more sensor devices, and further wherein each of the one or more training input channels is determined based on a first subset of a plurality of per-time-unit per-sensor training measurements associated with the sensor device of the training input channel; obtaining a maintenance data object for the monitored system, wherein the maintenance data object identifies one or more maintenance-critical time units in the one or more training time units; and generating a trained predictive maintenance convolutional neural network based on the training input data object and the maintenance data object, wherein the trained predictive maintenance convolutional neural network is configured to process the predictive input data object for the monitored system to generate a maintenance prediction for the monitored system.

[0004] According to another aspect, a computer program product is provided. The computer program product may include at least one computer-readable storage medium having a computer-readable program code portion stored therein, the computer-readable program code portion including an executable portion, the executable portion configured to: obtain a training telemetry data object for a monitored system; determine a training input data object based on the training telemetry data object, wherein the training input data object includes one or more training input channels, and wherein each of the one or more training input channels is associated with a sensor device in one of the one or more sensor devices, and further wherein each of the one or more training input channels is determined based on a first subset of a plurality of per-time-unit per-sensor training measurements associated with the sensor device of the training input channel; obtain a maintenance data object for the monitored system, wherein the maintenance data object identifies one or more maintenance-critical time units in the one or more training time units; and generate a trained predictive maintenance convolutional neural network based on the training input data object and the maintenance data object, wherein the trained predictive maintenance convolutional neural network is configured to process the predictive input data object for the monitored system to generate a maintenance prediction for the monitored system.

[0005] According to another aspect, an apparatus is provided, the apparatus comprising at least one processor and at least one memory comprising computer program code. In one embodiment, the at least one memory and the computer program code can be configured to use the processor to cause the apparatus to: obtain a training telemetry data object of a monitored system; determine a training input data object based on the training telemetry data object, wherein the training input data object includes one or more training input channels, and wherein each of the one or more training input channels is associated with a sensor device in one or more sensor devices, and further wherein each of the one or more training input channels is determined based on a first subset of a plurality of per-time-unit per-sensor training measurements associated with the sensor device of the training input channel; obtain a maintenance data object of the monitored system, wherein the maintenance data object identifies one or more maintenance-critical time units in the one or more training time units; and generate a trained predictive maintenance convolutional neural network based on the training input data object and the maintenance data object, wherein the trained predictive maintenance convolutional neural network is configured to process the predictive input data object of the monitored system to generate a maintenance prediction for the monitored system. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Having thus generally described some embodiments, reference will now be made to the accompanying drawings, which are not drawn to scale, and in which:

[0007] Figure 1is a block diagram of an exemplary architecture in which at least some embodiments disclosed herein may operate.

[0008] Figure 2 is a block diagram of an exemplary predictive maintenance computing device in accordance with at least some embodiments disclosed herein.

[0009] Figure 3 is a block diagram of an exemplary client computing device in accordance with at least some embodiments disclosed herein.

[0010] Figure 4 is a block diagram of an exemplary telemetry server computing device in accordance with at least some embodiments disclosed herein.

[0011] Figure 5 is a flow chart of an exemplary process for training a predictive maintenance convolutional neural network according to at least some embodiments disclosed herein.

[0012] Figure 6 Examples of operations of telemetry data objects are provided in accordance with at least some embodiments disclosed herein.

[0013] Figure 7 Examples of operations on input data objects are provided in accordance with at least some embodiments disclosed herein.

[0014] Figure 8 Examples of operations for maintaining data objects are provided in accordance with at least some embodiments disclosed herein.

[0015] Fig. 9 An example of operation of a predictive maintenance convolutional neural network according to at least some embodiments disclosed herein is provided.

[0016] Fig.10 is a flow chart of an exemplary process for performing predictive inference using a trained predictive maintenance convolutional neural network according to at least some embodiments disclosed herein.

[0017] Fig.11 An example of the operation of a per-input object heat map according to at least some embodiments disclosed herein is provided.

[0018] Fig.12 is a flow chart of an exemplary process for generating a per-input channel heat map according to at least some embodiments disclosed herein. DETAILED DESCRIPTION

[0019] Various embodiments described herein will now be described more fully hereinafter with reference to the accompanying drawings, in which some but not all embodiments of the present disclosure are shown. In fact, the present disclosure can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided in order for the present disclosure to meet applicable legal requirements. Unless otherwise indicated, the term "or" is used herein in both alternative and combined senses. The terms "illustrative", "example" and "exemplary" are examples for which there is no indication of a quality level. Throughout the content, similar reference numerals refer to similar elements.

[0020] Overview

[0021] Various embodiments of the present disclosure present techniques for training a convolutional neural network to generate maintenance predictions for a monitored system based on telemetry data associated with the monitored system, and techniques for extracting explanatory metadata of maintenance-related predictive inferences based on outputs generated by the mentioned convolutional neural network. By utilizing the mentioned techniques, various embodiments of the present disclosure enable automatic predictive maintenance of the monitored system to be performed in an effective, efficient and interpretable manner. The mentioned techniques also enable learning the desired configuration of the mentioned monitored system based on cross-time analysis of past fault history logs and / or past maintenance history logs associated with a large-scale monitored system (e.g., a large-scale monitored system with a large number of affected users). By doing so, various embodiments of the present invention make substantial contributions to improving the efficiency and effectiveness of monitoring and maintaining a variety of physical systems (such as heating, ventilation and air conditioning systems).

[0022] Exemplary System Architecture

[0023] The methods, apparatus, and computer program products disclosed herein may be embodied by any of a variety of devices. For example, the methods, apparatus, and computer program products of the exemplary embodiments may be embodied by a networked device (e.g., an enterprise platform) (such as a server or other network entity) configured to communicate with one or more devices (such as one or more query initiating computing devices). Additionally or alternatively, the computing device may include a fixed computing device, such as a personal computer or a computer workstation. Further, the exemplary embodiments may be embodied by any of a variety of mobile devices, such as a portable digital assistant (PDA), a mobile phone, a smart phone, a laptop computer, a tablet computer, a wearable device, or any combination of the foregoing devices.

[0024] Figure 1An exemplary system architecture 100 is shown in which embodiments disclosed herein may operate. The architecture 100 includes a predictive maintenance system 105 configured to interact with one or more client computing devices 102A-C, such as client computing device 102A, client computing device B 102B, and client computing device C 102C. The predictive maintenance system 105 may be configured to receive requests from client computing devices 102A-C to perform predictive maintenance data analysis based on telemetry data, perform predictive maintenance data analysis based on telemetry to generate maintenance predictions, and provide the generated maintenance predictions to the client computing devices 102A-C.

[0025] Telemetry data is associated with one or more monitored systems (such as one or more heating, ventilation, and air conditioning systems). In some embodiments, telemetry data refers to data obtained by recording readings of one or more sensor devices that are configured to monitor one or more monitored systems (e.g., heating, ventilation, and air conditioning systems). Examples of sensor devices whose readings are used to generate telemetry data include bag filter sensors, coil-on temperature sensors, supply air temperature sensors, ambient humidity sensors, fan angular motion sensors, etc. In some embodiments, telemetry data refers to data generated based on one or more operating commands for a monitored system.

[0026] The predictive maintenance system 105 can communicate with the client computing devices 102A-C using the network 104. The network 104 may include any wired or wireless communication network, including, for example, a wired or wireless local area network (LAN), a personal area network (PAN), a metropolitan area network (MAN), a wide area network (WAN), etc., and any hardware, software, and / or firmware required to implement it (such as, for example, a network router, etc.). For example, the network 104 may include a cellular phone, an 802.11, 802.16, 802.20, and / or WiMax network. In addition, the network 104 may include a public network (such as the Internet), a private network (such as an intranet), or a combination thereof, and may utilize a variety of networking protocols that are currently available or developed later, including but not limited to networking protocols based on the Transmission Control Protocol / Internet Protocol (TCP / IP). For example, the networking protocol can be customized to accommodate the needs of a group-based communication system. In some embodiments, the protocol is a customized protocol for a JavaScript Object Notation (JSON) object sent via a Websocket channel. In some embodiments, the protocol is JSON based on RPC, JSON based on REST / HTTP, etc.

[0027] The predictive maintenance system 105 includes a predictive maintenance computing device 106 and a storage subsystem 108. The predictive maintenance computing device 106 can be configured to perform predictive maintenance based on telemetry data. The predictive maintenance computing device 106 includes a training unit 111, a prediction unit 112, and an explanatory metadata unit 113. Figures 5 to 12 The functionality of the mentioned components 111 - 113 of the predictive maintenance computing device 106 is described.

[0028] The storage subsystem 108 is configured to store telemetry data and data associated with one or more predictive models utilized by the predictive maintenance computing device 106. The storage subsystem 108 may include one or more storage units, such as multiple distributed storage units connected by a computer network. Each storage unit in the storage subsystem 108 may store at least one of one or more data assets and / or one or more data regarding computing characteristics of one or more data assets. In addition, each storage unit in the storage subsystem 108 may include one or more non-volatile storage devices or memory media, including but not limited to hard disks, ROMs, PROMs, EPROMs, EEPROMs, flash memory, MMCs, SD memory cards, memory sticks, CBRAMs, PRAMs, FeRAMs, NVRAMs, MRAMs, RRAMs, SONOSs, FJG RAMs, Millipede memories, racetrack memories, and the like.

[0029] The predictive maintenance system 105 may receive telemetry data from at least one of the client computing devices 102A-C and the telemetry server computing device 107. For example, the telemetry server computing device 107 may be a server device configured to monitor the readings of one or more sensor devices associated with one or more monitored systems. An example of a telemetry server computing device 107 is a server device associated with a heating, ventilation, and air conditioning system. The telemetry data may be stored in a storage subsystem 108 of the predictive maintenance system 105. Examples of sensor devices whose readings are recorded and transmitted by the telemetry server computing device 107 include bag filter sensors, coil-on temperature sensors, supply air temperature sensors, ambient humidity sensors, fan angular motion sensors, and the like. In some embodiments, the predictive maintenance system 105 generates telemetry data based on telemetry signals received from the telemetry server computing device 107.

[0030] Exemplary Predictive Maintenance Computing Device

[0031] The predictive maintenance computing device 106 is composed of one or more computing systems (such as Figure 2200) is embodied in the apparatus 200 shown. The apparatus 200 may include a processor 202, a memory 204, an input / output circuit 206, and a communication circuit 208. The apparatus 200 may be configured to perform the operations described herein. Although these components 202-208 are described with respect to functional limitations, it should be understood that a specific implementation must include the use of specific hardware. It should also be understood that some of these components 202-208 may include similar or common hardware. For example, both groups of circuits may use the same processor, network interface, storage medium, etc. to perform their associated functions, so that each group of circuits does not require repeated hardware.

[0032] In some embodiments, the processor 202 (and / or a coprocessor or any other processing circuit that assists the processor or is otherwise associated with the processor) can communicate with the memory 204 via a bus for transferring information between components of the device. The memory 204 is non-transitory and can include, for example, one or more volatile and / or non-volatile memories. In other words, for example, the memory 204 can be an electronic storage device (e.g., a computer-readable storage medium). The memory 204 can be configured to store information, data, content, applications, instructions, etc. for enabling the device to perform various functions according to the exemplary embodiments disclosed herein.

[0033] The processor 202 may be embodied in a variety of different ways, and may, for example, include one or more processing devices configured to execute independently. In some preferred and non-limiting embodiments, the processor 202 may include one or more processors configured in series via a bus to enable independent execution of instructions, pipelining, and / or multithreading. The use of the term "processing circuit" may be understood to include a single-core processor, a multi-core processor, multiple processors within a device, and / or a remote or "cloud" processor.

[0034] In some preferred and non-limiting embodiments, the processor 202 may be configured to execute instructions stored in the memory 204 or otherwise accessible to the processor 202. In some preferred and non-limiting embodiments, the processor 202 may be configured to perform hard-coded functions. Therefore, whether configured by hardware methods or software methods, or by a combination of them, the processor 202 may represent an entity (e.g., physically embodied in circuit form) that is capable of performing operations according to the embodiments disclosed herein while being configured accordingly. Alternatively, for example, when the processor 202 is embodied as an executor of software instructions, the instructions may specifically configure the processor 202 to perform the algorithms and / or operations described herein when executing the instructions.

[0035] In some embodiments, the device 200 may include an input / output circuit 206, which may in turn communicate with the processor 202 to provide output to a user, and in some embodiments, receive an indication of a user input. The input / output circuit 206 may include a user interface, and may include a display, and may include a web user interface, a mobile application, a query initiating computing device, an information kiosk, etc. In some embodiments, the input / output circuit 206 may also include a keyboard, a mouse, a joystick, a touch screen, a touch area, soft keys, a microphone, a speaker, or other input / output mechanism. The processor and / or user interface circuitry including the processor may be configured to control one or more functions of one or more user interface elements via computer program instructions (e.g., software and / or firmware) stored on a memory accessible to the processor (e.g., memory 204, etc.).

[0036] The communication circuit 208 can be any device, such as a device or circuit embodied in hardware or a combination of hardware and software, which is configured to receive and / or transmit data from and / or to a network and / or any other device, circuit or module in communication with the device 200. In this regard, the communication circuit 208 may include, for example, a network interface for implementing communications with a wired or wireless communication network. For example, the communication circuit 208 may include one or more network interface cards, antennas, buses, switches, routers, modems and supporting hardware and / or software, or any other device suitable for implementing communications via a network. Additionally or alternatively, the communication circuit 208 may include circuitry for interacting with one or more antennas to send signals via one or more antennas or process the reception of signals received via one or more antennas.

[0037] It should also be noted that all or some of the information discussed herein may be based on data received, generated, and / or maintained by one or more components of the device 200. In some embodiments, one or more external systems (such as remote cloud computing and / or data storage systems) may also be utilized to provide at least some of the functionality discussed herein.

[0038] Exemplary Client Computing Device

[0039] Reference now Figure 3 The client computing devices 102A-C are composed of one or more computing systems (such as Figure 3300) is embodied in the apparatus 300 shown. The apparatus 300 may include a processor 302, a memory 304, an input / output circuit 306, and a communication circuit 308. Although these components 302-308 are described with respect to functional limitations, it should be understood that a particular implementation necessarily includes the use of specific hardware. It should also be understood that some of these components 302-310 may include similar or common hardware. For example, both groups of circuits may use the same processor, network interface, storage medium, etc. to perform their associated functions, so that each group of circuits does not require duplicate hardware.

[0040] In some embodiments, the processor 302 (and / or a coprocessor or any other processing circuit that assists the processor or is otherwise associated with the processor) can communicate with the memory 304 via a bus for transferring information between components of the device. The memory 304 is non-transitory and can include, for example, one or more volatile and / or non-volatile memories. In other words, for example, the memory 304 can be an electronic storage device (e.g., a computer-readable storage medium). The memory 304 can include one or more databases. In addition, the memory 304 can be configured to store information, data, content, applications, instructions, etc. for enabling the device 300 to perform various functions according to the exemplary embodiments disclosed herein.

[0041] The processor 302 may be embodied in a variety of different ways, and may, for example, include one or more processing devices configured to execute independently. In some preferred and non-limiting embodiments, the processor 302 may include one or more processors configured in series via a bus to enable independent execution of instructions, pipelining, and / or multithreading. The use of the term "processing circuit" may be understood to include a single-core processor, a multi-core processor, multiple processors within a device, and / or a remote or "cloud" processor.

[0042] In some preferred and non-limiting embodiments, the processor 302 may be configured to execute instructions stored in the memory 304 or otherwise accessible to the processor 302. In some preferred and non-limiting embodiments, the processor 302 may be configured to perform hard-coded functions. Therefore, whether configured by hardware methods or software methods, or by a combination of them, the processor 302 may represent an entity (e.g., physically embodied in circuit form) that is capable of performing operations according to the embodiments disclosed herein while being configured accordingly. Alternatively, for example, when the processor 302 is embodied as an executor of software instructions (e.g., computer program instructions), the instructions may specifically configure the processor 302 to perform the algorithms and / or operations described herein when executing the instructions.

[0043] In some embodiments, the device 300 may include an input / output circuit 306, which may in turn communicate with the processor 302 to provide output to a user, and in some embodiments, receive an indication of a user input. The input / output circuit 306 may include a user interface, and may include a display, and may include a web user interface, a mobile application, a query initiating computing device, an information kiosk, etc. In some embodiments, the input / output circuit 306 may also include a keyboard (e.g., also referred to herein as a keypad), a mouse, a joystick, a touch screen, a touch area, soft keys, a microphone, a speaker, or other input / output mechanism. The processor and / or user interface circuitry including the processor may be configured to control one or more functions of one or more user interface elements via computer program instructions (e.g., software and / or firmware) stored on a memory accessible to the processor (e.g., memory 304, etc.).

[0044] The communication circuit 308 can be any device, such as a device or circuit embodied in hardware or a combination of hardware and software, which is configured to receive and / or transmit data from and / or to a network and / or any other device, circuit or module in communication with the device 300. In this regard, the communication circuit 308 may include, for example, a network interface for implementing communications with a wired or wireless communication network. For example, the communication circuit 308 may include one or more network interface cards, antennas, buses, switches, routers, modems and supporting hardware and / or software, or any other device suitable for implementing communications via a network. Additionally or alternatively, the communication circuit 308 may include circuitry for interacting with one or more antennas to send signals via one or more antennas or process the reception of signals received via one or more antennas.

[0045] It should also be noted that all or some of the information discussed herein may be based on data received, generated, and / or maintained by one or more components of the device 300. In some embodiments, one or more external systems (such as remote cloud computing and / or data storage systems) may also be utilized to provide at least some of the functionality discussed herein.

[0046] Exemplary Telemetry Server Computing Device

[0047] Reference now Figure 4 The telemetry server computing device 107 is composed of one or more computing systems (such as Figure 4400). The device 400 may include a processor 402, a memory 404, an input / output circuit 406, and a communication circuit 408. Although these components 402-408 are described with respect to functional limitations, it should be understood that a particular implementation must include the use of specific hardware. It should also be understood that some of these components 402-408 may include similar or common hardware. For example, two groups of circuits can use the same processor, network interface, storage medium, etc. to perform their associated functions, so that each group of circuits does not require duplicate hardware.

[0048] In some embodiments, the processor 402 (and / or a coprocessor or any other processing circuit that assists the processor or is otherwise associated with the processor) can communicate with the memory 404 via a bus for transferring information between components of the device. The memory 404 is non-transitory and can include, for example, one or more volatile and / or non-volatile memories. In other words, for example, the memory 404 can be an electronic storage device (e.g., a computer-readable storage medium). The memory 404 can include one or more databases. In addition, the memory 404 can be configured to store information, data, content, applications, instructions, etc. for enabling the device 400 to perform various functions according to the exemplary embodiments disclosed herein.

[0049] Processor 402 can be embodied in a variety of different ways, and can, for example, include one or more processing devices configured to execute independently. In some preferred and non-limiting embodiments, processor 402 can include one or more processors configured in series via a bus to achieve independent execution of instructions, pipelining and / or multithreading. The use of the term "processing circuit" can be understood to include a single-core processor, a multi-core processor, multiple processors within a device, and / or a remote or "cloud" processor.

[0050] In some preferred and non-limiting embodiments, the processor 402 may be configured to execute instructions stored in the memory 404 or otherwise accessible to the processor 402. In some preferred and non-limiting embodiments, the processor 402 may be configured to perform hard-coded functions. Therefore, whether configured by hardware methods or software methods, or by a combination thereof, the processor 402 may represent an entity (e.g., physically embodied in circuit form) that is capable of performing operations according to the embodiments disclosed herein while being configured accordingly. Alternatively, for example, when the processor 402 is embodied as an executor of software instructions (e.g., computer program instructions), the instructions may specifically configure the processor 402 to perform the algorithms and / or operations described herein when executing the instructions.

[0051] In some embodiments, the device 400 may include an input / output circuit 406, which may then communicate with the processor 402 to provide output to the user, and in some embodiments, receive an indication of a user input. The input / output circuit 406 may include a user interface, and may include a display, and may include a web user interface, a mobile application, a query initiating computing device, an information kiosk, etc. In some embodiments, the input / output circuit 406 may also include a keyboard (e.g., also referred to as a keypad herein), a mouse, a joystick, a touch screen, a touch area, soft keys, a microphone, a speaker, or other input / output mechanism. The processor and / or user interface circuitry including the processor may be configured to control one or more functions of one or more user interface elements via computer program instructions (e.g., software and / or firmware) stored on a memory accessible to the processor (e.g., memory 404, etc.).

[0052] The communication circuit 408 can be any device, such as a device or circuit embodied in hardware or a combination of hardware and software, which is configured to receive and / or transmit data from and / or to a network and / or any other device, circuit or module in communication with the device 400. In this regard, the communication circuit 408 may include, for example, a network interface for implementing communications with a wired or wireless communication network. For example, the communication circuit 408 may include one or more network interface cards, antennas, buses, switches, routers, modems and supporting hardware and / or software, or any other device suitable for implementing communications via a network. Additionally or alternatively, the communication circuit 408 may include circuitry for interacting with one or more antennas to send signals via one or more antennas or process the reception of signals received via one or more antennas.

[0053] It should also be noted that all or some of the information discussed herein may be based on data received, generated, and / or maintained by one or more components of the device 400. In some embodiments, one or more external systems (such as remote cloud computing and / or data storage systems) may also be utilized to provide at least some of the functionality discussed herein.

[0054] Example Data Flow and Operations

[0055] Various embodiments of the present disclosure present techniques for training a convolutional neural network to generate maintenance predictions for a monitored system based on telemetry data associated with the monitored system, and techniques for extracting explanatory metadata of maintenance-related predictive inferences based on outputs generated by the mentioned convolutional neural network. By utilizing the mentioned techniques, various embodiments of the present disclosure enable automatic predictive maintenance of the monitored system to be performed in an effective, efficient and interpretable manner. The mentioned techniques also enable learning the desired configuration of the mentioned monitored system based on cross-time analysis of past fault history logs and / or past maintenance history logs associated with a large-scale monitored system (e.g., a large-scale monitored system with a large number of affected users). By doing so, various embodiments of the present invention make substantial contributions to improving the efficiency and effectiveness of monitoring and maintaining a variety of physical systems (such as heating, ventilation and air conditioning systems).

[0056] Training a Convolutional Neural Network for Predictive Maintenance

[0057] Figure 5 5 is a flowchart of an exemplary process 500 for training a predictive maintenance convolutional neural network. Through the various steps of process 500, the training unit 111 of the predictive maintenance computing device 106 can efficiently and effectively train a predictive maintenance convolutional neural network that is configured to perform efficient, effective, and interpretable predictive inferences related to maintenance needs of a monitored system (e.g., a monitored heating, ventilation, and air conditioning system).

[0058] As used herein, a convolutional neural network is any machine learning model that processes its input data according to one or more convolutional layers, each of which is followed by a pooling layer (immediately or after one or more intermediate layers), wherein the final outputs of the convolutional and pooling layers may then be supplied to one or more discriminative layers, such as one or more fully connected discriminative layers. A predictive maintenance convolutional neural network is a convolutional neural network for performing predictive maintenance on a monitored system based on telemetry data associated with the monitored system. Although various embodiments of the present disclosure describe performing predictive maintenance with respect to a single monitoring system, a person of ordinary skill in the relevant art will recognize that the disclosed techniques may be used to perform predictive maintenance on any number of monitored systems.

[0059] Process 500 begins at step 501, when the training unit 111 obtains a training telemetry data object for the monitored system. As used herein, a telemetry data object is a data object that includes measurements of a group of one or more sensor devices across a group of one or more time units. Therefore, the telemetry data object includes a plurality of per-time unit per-sensor measurements, wherein each of the plurality of per-time unit per-sensor measurements is associated with one of the one or more time units and one of the one or more sensor devices. The training telemetry data object is a data object for training a predictive maintenance convolutional neural network. Therefore, the training telemetry data object includes a plurality of per-time unit per-sensor training measurements, wherein each of the plurality of per-time unit per-sensor training measurements is associated with one of the one or more training time units and one of the one or more sensor devices. In some embodiments, the telemetry data includes data determined based on measurements from one or more sensor devices.

[0060] Figure 6 An example of the operation of the telemetry data object 600 is presented in FIG. Figure 6 As depicted in FIG. 6 , telemetry data object 600 includes a time series graph that depicts differential pressure measurements of three sensor devices 601 during a time unit defined by the x-axis of the time series graph. Figure 6 As further depicted, telemetry data object 600 includes a plurality of per-sensor measurements per time unit, wherein each per-sensor measurement per time unit is associated with one of one or more training time units defined by an x-axis of the time series graph and one of three sensor devices 601. For example, per-sensor measurement per time unit 621 is associated with a bag filter sensor device and time unit 602. As another example, per-sensor measurement per time unit 622 is associated with a bag filter sensor device and time unit 602. As yet another example, per-sensor measurement per time unit 623 is associated with an on-coil temperature sensor device and time unit 602.

[0061] return Figure 5, at operation 502, the training unit 111 determines a training input data object based on the training telemetry data object. As used herein, an input data object is a data object configured to be supplied to a predictive maintenance convolutional neural network. The input data object has a data format defined by an input structure of the predictive maintenance convolutional neural network. In some embodiments, the data format of the input data object is a data format known in the relevant art that facilitates the capabilities and structure of a machine learning model based on a convolutional neural network. For example, the input data object may have an image format, an audio format, etc. The training input data object is an input data object for training a predictive maintenance convolutional neural network.

[0062] The input data object has one or more input channels, wherein each input channel is associated with a corresponding sensor device and is determined based on a subset of per-sensor measurements per time unit of a telemetry data object associated with the corresponding sensor device. In some embodiments, if the input data object is a visual data object (e.g., an image data object), the one or more input channels of the input data object are defined based on one or more color-based channels of the image data object, wherein the one or more color-based channels are defined by a color encoding scheme of the image data object.

[0063] For example, if the image data object has a grayscale color encoding scheme, the input channels of the image data object include a first input channel corresponding to a black-based channel and a second input channel corresponding to a white-based channel. As another example, if the image data object has a red-green-blue (RGB) color encoding scheme, the input channels of the image data object include a first input channel corresponding to a red-based channel, a second input channel corresponding to a green-based channel, and a third input channel corresponding to a blue-based channel.

[0064] As noted above, the training input data object is an input data object for training a predictive maintenance convolutional neural network. Thus, the training input data object is associated with one or more training input channels, wherein a training input channel of the one or more training input channels is associated with a sensor device of the one or more sensor devices associated with the training telemetry data object, and wherein each training input channel is determined based on a subset of the training measurements per sensor per time unit of the training telemetry data object associated with the corresponding sensor device.

[0065] Although various embodiments of the present disclosure describe input channels, each input channel is associated with a single sensor device and is determined based on telemetry data associated with a single sensor device, a person of ordinary skill in the relevant art will recognize that each input channel can be associated with any number of sensor devices and is determined based on telemetry data associated with any number of sensor devices. In addition, although various embodiments of the present disclosure describe input channels of image-based input data objects, a person of ordinary skill in the relevant art will recognize that input channels can be created in data objects having any data format. In addition, although various embodiments of the present disclosure describe color-based input channels of image-based input data objects, a person of ordinary skill in the relevant art will recognize that image-based data objects can include non-color-based input channels, such as feature-based input channels, wherein each feature-based input channel itself can be an image, and wherein an image-based input data object is generated by superimposing feature-based input channels on each other to generate an image-based input data object.

[0066] Figure 7 An example of the operation of input data object 700 is presented in FIG. Figure 7 As depicted, input data object 700 includes three input channels 701-703, wherein each of the three input channels 701-703 is associated with Figure 6 600 of the telemetry data object 600. Specifically, input channel 701 is associated with a bag filter sensor device and is determined based on a subset of the per-sensor measurement values ​​per time unit of the telemetry data object 600 associated with the mentioned bag filter sensor device; input channel 702 is associated with an on-coil temperature sensor device and is determined based on a subset of the per-sensor measurement values ​​per time unit of the telemetry data object 600 associated with the mentioned bag filter sensor device; and input channel 703 is associated with a supply air temperature sensor device and is determined based on a subset of the per-sensor measurement values ​​per time unit of the telemetry data object 600 associated with the mentioned bag filter sensor device.

[0067] return Figure 5, at operation 503, the training unit 111 obtains a maintenance input data object. As used herein, a maintenance input data object is a data object that identifies one or more time units (e.g., one or more time units of a training time unit associated with a training telemetry data object) as maintenance critical time units. In some embodiments, in addition to identifying the maintenance critical time units, the maintenance input data object also provides one or more maintenance time unit characteristics for each maintenance critical time unit, such as the source and / or urgency of a maintenance requirement identifier associated with each maintenance critical time unit, an affected tenant identifier associated with each maintenance critical time unit, and the like. The maintenance data object can be used to train one or more predictive maintenance convolutional neural networks and / or to perform convolutional neural network-based predictive maintenance operations using a trained predictive maintenance convolutional neural network. The content of the maintenance data object is generated using maintenance data associated with a monitored system and according to the structure of a maintenance prediction generated by a predictive maintenance convolutional neural network trained using the maintenance data object.

[0068] For example, to train a predictive maintenance convolutional neural network configured to generate maintenance predictions representing a predictive maintenance need with any level of urgency for a single monitored system (e.g., a heating, ventilation, and air conditioning system for a particular building), the training unit 111 may generate a maintenance data object that identifies all timestamps of maintenance logs associated with the single monitored system as maintenance critical time units. As another example, to train a predictive maintenance convolutional neural network configured to generate maintenance predictions representing a predictive maintenance need with any level of urgency for multiple monitored systems, the training unit 111 may generate a maintenance data object that identifies all timestamps of maintenance logs associated with any one or more of the monitored systems as maintenance critical time units. As yet another example, in order to train a predictive maintenance convolutional neural network configured to generate a maintenance prediction representing a predictive maintenance need with a specific urgency and / or a specific source for a single monitored system, the training unit 111 may generate a maintenance data object identifying all timestamps of maintenance logs with the specific urgency and / or the specific source associated with the single monitored system. As a further example, in order to generate a maintenance data object identifying all timestamps of maintenance logs with any one of multiple specific urgency and / or multiple specific sources for a single monitored system, the training unit 111 may generate a maintenance data object identifying all timestamps of maintenance logs with at least one of the multiple specific urgency and / or at least one of the multiple specific sources associated with the single monitored system. As a further example, in order to generate a maintenance data object that identifies all timestamps of maintenance logs that meet specific criteria and the predicted features of each maintenance log for the identified timestamp (e.g., source, urgency, etc.), the training unit 111 may generate a maintenance data object that identifies all timestamps of maintenance logs that meet specific criteria and the mentioned predicted features of each maintenance log for the identified timestamp.

[0069] exist Figure 8 An example of the operation of maintaining data object 800 is presented in FIG. Figure 8, maintenance data object 800 includes one or more entries each identifying a maintenance critical time unit. For example, entry 801 of maintenance data object 800 identifies 18:08 on October 3, 2016 as a maintenance critical time unit. As another example, entry 802 of maintenance data object 800 identifies 16:43 on January 1, 2017 as a maintenance critical time unit. As yet another example, entry 803 of maintenance data object 800 identifies 16:53 on January 1, 2017 as a maintenance critical time unit. As a further example, entry 804 of maintenance data object 800 identifies 17:00 on June 14, 2017 as a maintenance critical time unit.

[0070] return Figure 5 At operation 504, the training unit 111 generates a trained predictive maintenance convolutional neural network based on the training input data object and the maintenance data object. In some embodiments, to generate the trained predictive maintenance convolutional neural network, the training unit 111 utilizes one or more training algorithms, such as an optimization-based training algorithm. An example of an optimization-based training algorithm suitable for generating a trained predictive maintenance convolutional neural network is gradient descent with back-propagation.

[0071] For example, in some embodiments, to generate a trained predictive maintenance convolutional neural network, the training unit 111 provides a training input data object to an untrained predictive maintenance convolutional neural network. The training unit 111 then processes the training input data object according to the untrained predictive maintenance convolutional neural network to generate an inferred maintenance prediction for the untrained predictive maintenance convolutional neural network. The training unit 111 then generates an error measure for the untrained predictive maintenance convolutional neural network based on the deviation between the inferred maintenance prediction indicated by the maintenance data object and the maintenance data. The training unit 111 then determines the gradient of the error measure relative to the parameters of the untrained predictive maintenance convolutional neural network. The training unit 111 then back-propagates the error measure to update the parameters of the untrained predictive maintenance convolutional neural network to minimize the error measure. Although various embodiments of the present disclosure disclose training algorithms that seek to minimize an error function, a person of ordinary skill in the relevant art will recognize that an optimization-based training algorithm may seek to maximize a utility function rather than minimize a cost function.

[0072] Fig. 9 An example of the operation of the predictive maintenance convolutional neural network 900 is depicted in FIG. Fig. 9As depicted, the predictive maintenance convolutional neural network 900 includes a set of feature extraction layers 901 configured to obtain an input data object 931 having one or more input channels, process the input data object 931 to generate a feature map 932 for each of the one or more input channels, and provide the feature map 932 to a set of fully connected layers 902 of the predictive maintenance convolutional neural network 900. Fig. 9 As further depicted, a set of feature extraction layers 901 includes one or more sequentially ordered subgroups, wherein each sequentially ordered subgroup includes a convolution layer (e.g., a one-dimensional convolution layer), followed by a batch normalization layer, followed by an activation layer (e.g., an activation layer using a rectified linear unit activation function), and then a pooling layer. Fig. 9 As further depicted in , the predictive maintenance convolutional neural network 900 includes a set of fully connected layers 902 configured to receive feature maps 932 from a set of feature extraction layers 901 and process the feature maps 932 to generate maintenance predictions 933. Although various embodiments of the present disclosure describe convolutional neural networks with fully connected discriminative layers, a person of ordinary skill in the relevant art will recognize that other types of machine learning models may be used as the discriminative layer of the convolutional neural network.

[0073] Performing Predictive Disturbance Using a Trained Predictive Maintenance Convolutional Neural Network

[0074] Once trained, the predictive maintenance convolutional neural network can be used to perform predictive inference using telemetry data associated with a monitored system to generate a maintenance prediction for the monitored system. A maintenance prediction is any prediction about an expected maintenance need for a monitored system. For example, a maintenance prediction can be a prediction about whether a monitored system will have a maintenance need in the near future and / or within a specific time frame. As another example, a maintenance prediction can be a prediction about whether a monitored system will have a maintenance need of a specific type and / or a specific urgency in the near future and / or within a specific time frame. As yet another example, a maintenance prediction can be a prediction about the predicted type and / or predicted urgency of an existing maintenance need. As a further example, the maintenance prediction can describe one or more affected hardware units of the hardware units associated with the monitored system, wherein the one or more affected hardware units are determined based on processing the predictive telemetry data using the trained predictive maintenance convolutional neural network.

[0075] Fig.101 is a flow chart of an exemplary process 1000 for performing predictive inference using a trained predictive maintenance convolutional neural network. Through the various operations of process 1000, the prediction unit 112 of the predictive maintenance computing device 106 can efficiently and effectively perform efficient, effective and interpretable predictive inferences related to the maintenance needs of the monitored system (e.g., the monitored heating, ventilation, air conditioning system). Although various embodiments of the present disclosure describe training and predictive inference processes performed by different units of a single computing device, a person of ordinary skill in the relevant art will recognize that the mentioned processes can each be performed by a separate computing device in a group of computing devices that are connected to each other via a network and / or can be configured to operate in a distributed manner.

[0076] Process 1000 begins at operation 1001, when prediction unit 112 obtains a predictive telemetry data object. A predictive telemetry data object is a telemetry data object used to perform predictive inference (e.g., having a Figure 6 The predictive telemetry data object is a telemetry data object of the structure depicted), the predictive inference utilizing a trained predictive maintenance convolutional neural network. Thus, the predictive telemetry data object is associated with a plurality of per-time-unit-per-sensor predictive measurements, wherein each per-time-unit-per-sensor predictive measurement in the plurality of per-time-unit-per-sensor predictive measurements is associated with a sensor device in the one or more sensor devices and a predictive time unit in the one or more predictive time units. In some embodiments, the prediction unit 112 obtains at least a portion of the predictive telemetry data object from the client computing device 102A-C.

[0077] At operation 1002, the prediction unit 112 determines a predictive input data object based on the predictive telemetry data object. The predictive input data object is an input data object (e.g., having a similar Figure 7 Thus, the predictive input data object comprises one or more predictive input channels, wherein each predictive input channel of the one or more predictive input channels is associated with a sensor device of one or more sensor devices, measurements of the one or more sensor devices being captured by the predictive telemetry data object, and wherein the predictive input channel of the one or more predictive input channels is determined based on a second subset of the plurality of per-time-unit-per-sensor predictive measurements in the predictive telemetry data object associated with the sensor device for the predictive input channel.

[0078] At operation 1003, the prediction unit 112 maintains a convolutional neural network (e.g., having Fig. 9The predictive input data object is processed by a trained predictive maintenance convolutional neural network of the depicted architecture to generate a maintenance prediction for the monitored system. In some embodiments, the prediction unit 112 uses the output of the final discriminative layer of the trained predictive maintenance convolutional neural network as the maintenance prediction. In some embodiments, the prediction unit 112 performs one or more operations (e.g., one or more softmax normalization operations) on the output of the final discriminative layer of the trained predictive maintenance convolutional neural network to generate a normalized output value, wherein the normalized output value can then be used to generate a maintenance prediction.

[0079] At operation 1004, the prediction unit 112 performs one or more prediction-based actions based on the generated maintenance prediction. Examples of prediction-based actions include setting configuration parameters of the monitored system to accommodate the maintenance needs indicated by the maintenance prediction, generating one or more maintenance need notifications, adjusting the performance of one or more sensor devices based on the maintenance prediction to accommodate the maintenance needs indicated by the maintenance prediction, automatically scheduling one or more maintenance appointments to address the maintenance needs indicated by the maintenance prediction, etc.

[0080] Generate explanatory metadata for predictive inference performed using a trained predictive maintenance convolutional neural network

[0081] By utilizing per-sensor feature maps that can be generated by a trained predictive maintenance convolutional neural network, various embodiments of the present disclosure can efficiently and effectively generate per-input channel heat maps that each highlight a specific portion of the corresponding input channel as having a relatively higher degree of predictive importance for a particular predictive relevance compared to other portions of the corresponding input channel. Fig.11 An example of the operation of such a per-input channel heat map 1100 is provided in FIG. Fig.11 As depicted, each input channel heat map 1100 will Figure 7 Portions 1101 of input channels 701 are indicated as having greater predictive importance for a particular predictive relevance.

[0082] According to various embodiments of the present disclosure, because each input channel can be associated with one or more corresponding sensor devices (e.g., a single corresponding sensor device), explanatory metadata can be generated at a per-sensor level and / or a per-sensor group level. This is an important distinction between various embodiments of the present disclosure and many prior art techniques for generating metadata, which generate explanatory metadata at a cross-input channel level. By generating per-input channel explanatory metadata, various embodiments of the present disclosure can present highly granular explanatory metadata about the relevance of sensor measurements and generated maintenance predictions.

[0083] Fig.1212 is a flow chart of an exemplary process 1200 for generating a per-input channel heat map for a predictive input channel of a predictive input data object. Through the various operations of process 1200, the explanatory metadata unit 113 of the predictive maintenance computing device 106 can efficiently and effectively generate metadata that associates granular sensor measurements with maintenance predictions, thereby making the predictive inference performed using the trained predictive inference convolutional neural network more interpretable. Although various embodiments of the present disclosure describe training processes, predictive inference processes, and explanatory metadata generation processes performed by different units of a single computing device, a person of ordinary skill in the relevant art will recognize that the mentioned processes can be performed by any number of computing devices that can be connected to each other via a network and / or can be configured to operate in a distributed manner.

[0084] Process 1200 begins at operation 1201, when the explanatory metadata unit 113 determines a per-input channel weight value for a predictive input channel relative to a target classification category associated with a maintenance prediction. In some embodiments, the target classification category is a candidate classification category for a maintenance state of a monitored system that may be indicated by the maintenance prediction. For example, if the trained predictive maintenance convolutional neural network is configured to generate a maintenance prediction describing whether a monitored system is predicted to have a maintenance requirement within a time range, the target classification category for the maintenance prediction may be a candidate classification category indicating a positive prediction of a maintenance requirement within the mentioned time range or a candidate classification category indicating a negative prediction of a maintenance requirement within the mentioned time range.

[0085] In some embodiments, in order to determine the per-input channel weight value of the predictive input channel relative to the target classification category, the explanatory metadata unit 113 first determines the gradient of the target classification category score associated with the target classification category relative to the feature map of the predictive input category, and then determines the per-input channel weight value of the predictive input channel relative to the target classification category based on the mentioned determined gradient of the target classification category score associated with the target classification category relative to the feature map of the predictive input category.

[0086] In some embodiments, to determine the per-input channel weight value based on the determined gradient of the target classification category score relative to the feature map, the explanatory metadata unit 113 applies a global-max-pooling and / or global-average-pooling operation to the determined gradient to determine the per-input channel weight value. In some embodiments, to determine the per-input channel weight value of the predictive input channel relative to the target classification category, the explanatory metadata unit 113 utilizes the operation described in the following document: Selvaraju et al., Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization, Formula 1 on page 4 and accompanying description, arXiv:1610.02391[cs.CV](2016), available online at https: / / arxiv.org / pdf / 1610.02391.pdf (hereinafter referred to as Selvaraju).

[0087] At operation 1202, the explanatory metadata unit 113 generates a per-input channel heat map for a predictive input channel based on the per-input channel weight values ​​of the sensor device and the feature map of the predictive input channel generated by the trained predictive maintenance convolutional neural network. The per-input channel of the predictive input channel indicates a per-component prediction importance value for each component of a plurality of components of the predictive input channel (e.g., each pixel, each audio second, each image portion, etc.), wherein the per-component prediction importance value indicates a relative prediction relevance of the corresponding component to generating a particular maintenance prediction. In some embodiments, generating the per-input channel heat map for the predictive input channel includes performing guided back-propagation, for example, in accordance with some aspects of the technology disclosed by Selvaraju.

[0088] In some embodiments, generating a per-input channel heat map for a predictive input channel includes determining a per-input channel activation output based on a per-input channel weight value for the predictive input channel and a feature map for the predictive input channel; and applying an activation function (e.g., a rectified linear unit activation function) to the per-input channel activation output to generate a per-input channel heat map for the predictive input channel. In some embodiments, generating a per-sensor heat map for a predictive input channel includes performing an operation described by the following formula:

[0089] L k =ReLu(a k *A k )

[0090] Formula 1

[0091] In the above formula 1, L kis a per-input channel heat map of the k-th predictive input channel (and / or the sensor device associated with the k-th predictive input channel), ReLu is the rectified linear unit function, and a k is the per-input channel weight value of the kth predictive input channel, and A k is the feature map of the kth predictive input channel.

[0092] Additional implementation details

[0093] Although exemplary processing systems have been described in the text and figures, specific implementations of the subject matter and functional operations described herein may be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them.

[0094] The embodiments of the subject matter and operations described herein may be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. The embodiments of the subject matter described herein may be implemented as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer-readable storage medium for information / data processing devices to execute or control the operation of the information / data processing devices. Alternatively or in addition, program instructions may be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical signal, optical signal, or electromagnetic signal), which is generated to encode information / data for transmission to a suitable receiver device for execution by the information / data processing device. The computer-readable storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them, or may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Furthermore, although a computer-readable storage medium is not a propagated signal, a computer-readable storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. A computer-readable storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0095] The operations described herein may be implemented as operations performed by an information / data processing apparatus on information / data stored on one or more computer-readable storage devices or received from other sources.

[0096] The term "data processing device" covers all kinds of devices, equipment and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a plurality of the foregoing devices or a combination thereof. The device may include a dedicated logic circuit (e.g., a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). In addition to hardware, the device may also include code that creates an execution environment for the computer program under consideration (e.g., code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform operating environment, a virtual machine, or a combination of one or more of them). The device and the execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing infrastructures, and grid computing infrastructures.

[0097] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative languages, or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or information / data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing portions of one or more modules, subroutines, or code). A computer program may be deployed to execute the computer program on one or more computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0098] The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input information / data and generating output. By way of example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors and any one or more processors of any kind of digital computer. Generally speaking, the processor will receive instructions and information / data from a read-only memory, a random access memory, or both. The basic elements of a computer are a processor for performing actions according to instructions and one or more memories for storing instructions and data. Generally speaking, a computer will also include one or more mass storage devices (e.g., disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to the one or more mass storage devices to receive information / data from the one or more mass storage devices or transfer information / data to the one or more mass storage devices, or both. However, a computer does not need to have such devices. Devices suitable for storing computer program instructions and information / data include all forms of non-volatile memory, media, and storage devices, including by way of example semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0099] To provide interaction with a user, embodiments of the subject matter described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information / data to the user, as well as a keyboard and pointing device (e.g., a mouse or trackball through which the user can provide input to the computer). Other types of devices may also be used to provide interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including sound, voice, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser in response to a request received by the web browser on a computing device from a query initiating the user.

[0100] Embodiments of the subject matter described herein may be implemented in a computing system that includes a back-end component (e.g., as an information / data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a query-initiating computing device with a graphical user interface or a web browser through which a user can interact with a specific implementation of the subject matter described herein), or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital information / data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), interconnected networks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0101] The computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is obtained by means of a computer program running on each computer and having a client-server relationship with each other. In some embodiments, the server transmits information / data (e.g., a hypertext markup language (HTML) page) to a query initiating computing device (e.g., for displaying information / data to a user interacting with the query initiating computing device and receiving user input from a user interacting with the query initiating computing device). Information / data (e.g., results of user interaction) generated at the query initiating computing device can be received from the query initiating computing device at the server.

[0102] Although this specification includes many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or the content that can be claimed, but should be interpreted as descriptions of specific features for the specific embodiments of a specific invention. Certain features described herein in the context of a separate embodiment may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as acting in certain combinations and even initially claimed, in some cases, one or more features from the claimed combination may be removed from the combination, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0103] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be construed as requiring that such operations be performed in the particular order shown or in an increasing order, or that all of the operations shown be performed to achieve the desired result, unless otherwise described. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or grouped into multiple software products.

[0104] Thus, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired results. In addition, unless otherwise specified, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the increasing order to achieve the desired results. In some specific implementations, multitasking and parallel processing may be advantageous.

[0105] in conclusion

[0106] Many modifications and other embodiments of the inventions set forth herein will occur to those skilled in the art having the benefit of the foregoing description and the teachings presented in the associated drawings. Therefore, it should be understood that the invention is not limited to the particular embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, unless otherwise indicated, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. A computer-implemented method for performing predictive maintenance of a monitored system, the computer-implemented method comprising: obtaining a training telemetry data object corresponding to telemetry data indicative of time-series sensor measurement data collected from a plurality of sensors of the monitored system; determining a training input data object based on the training telemetry data object, the training input data object having a data format defined by an input structure of a predictive maintenance convolutional neural network, wherein the training input data object includes one or more training input channels based on a type of the training input data object, and wherein each training input channel is associated with a sensor of the plurality of sensors and is based on a subset of the telemetry data; obtaining a maintenance data object for the monitored system, wherein the maintenance data object identifies one or more maintenance critical timestamps of the time series sensor measurement data; as well as A trained predictive maintenance convolutional neural network is generated based on the training input data objects and the maintenance data objects, wherein the trained predictive maintenance convolutional neural network is configured to process the predictive input data objects for the monitored system to generate a maintenance prediction for the monitored system.

2. The computer-implemented method of claim 1 , further comprising: obtaining a predictive telemetry data object corresponding to predictive telemetry data indicative of predictive time series sensor measurement data for the monitored system; determining a predictive input data object based on the predictive telemetry data object, wherein the predictive input data object includes one or more predictive input channels, and wherein each of the one or more predictive input channels is associated with the sensor of the plurality of sensors, and further, wherein each of the one or more predictive input channels is determined based on a first subset of the predictive time series sensor measurement data associated with the sensor for the predictive input channel; as well as The predictive input data object is processed according to the trained predictive maintenance convolutional neural network to generate the maintenance prediction for the monitored system, and wherein the monitored system is a heating, ventilation, and air conditioning system, and wherein the trained predictive maintenance convolutional neural network is generated using a gradient descent training routine.

3. The computer-implemented method of claim 2, further comprising: for each of the one or more predictive input channels, determining a per-input channel weight value for the predictive input channel relative to a target classification category associated with the maintenance prediction; as well as Based on the per-input channel weight value of the predictive input channel and the feature map of the predictive input channel, a per-input channel heat map of the predictive input channel is generated, wherein the per-input channel heat map of the predictive input channel indicates a per-component prediction importance value of each of a plurality of components of the predictive input channel.

4. The computer-implemented method of claim 3, wherein generating the per-input channel heat map of the predictive input channels comprises: determining a per-input channel activation output based on the per-input channel weight value for the predictive input channel and the feature map for the predictive input channel; as well as An activation function is applied to the per-input channel activation output to generate the per-input channel heat map for the predictive input channel, and wherein the activation function is a rectified linear unit activation function. 5 . The computer-implemented method of claim 3 , wherein generating the per-input channel heatmap for the predictive input channels comprises performing guided back-propagation.

6. An apparatus for performing predictive maintenance of a monitored system, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code being configured to, with the processor, cause the apparatus to at least: obtaining a training telemetry data object corresponding to telemetry data indicative of time-series sensor measurement data collected from a plurality of sensors of the monitored system; determining a training input data object based on the training telemetry data object, the training input data object having a data format defined by an input structure of a predictive maintenance convolutional neural network, wherein the training input data object includes one or more training input channels based on a type of the training input data object, and wherein each training input channel is associated with a sensor of the plurality of sensors and is based on a subset of the telemetry data; obtaining a maintenance data object for the monitored system, wherein the maintenance data object identifies one or more maintenance critical timestamps of the time series sensor measurement data; as well as A trained predictive maintenance convolutional neural network is generated based on the training input data objects and the maintenance data objects, wherein the trained predictive maintenance convolutional neural network is configured to process the predictive input data objects for the monitored system to generate a maintenance prediction for the monitored system.

7. The apparatus of claim 6, wherein the program code is further configured to, with the processor, cause the apparatus to at least: obtaining a predictive telemetry data object corresponding to predictive telemetry data indicative of predictive time series sensor measurement data for the monitored system; determining a predictive input data object based on the predictive telemetry data object, wherein the predictive input data object comprises one or more predictive input channels, and wherein each of the one or more predictive input channels is associated with the sensor of the plurality of sensors, and further, wherein each of the one or more predictive input channels is determined based on a first subset of the predictive time series sensor measurement data associated with the sensor for the predictive input channel; and The predictive input data object is processed according to the trained predictive maintenance convolutional neural network to generate a maintenance prediction for the monitored system, and wherein the monitored system is a heating, ventilation, and air conditioning system.

8. The apparatus of claim 7, wherein the program code is further configured to, with the processor, cause the apparatus to at least: for each of the one or more predictive input channels, determining a per-input channel weight value for the predictive input channel relative to a target classification category associated with the maintenance prediction; and Based on the per-input channel weight value of the predictive input channel and the feature map of the predictive input channel, a per-input channel heat map of the predictive input channel is generated, wherein the per-input channel heat map of the predictive input channel indicates a per-component prediction importance value of each of a plurality of components of the predictive input channel.

9. The apparatus of claim 8, wherein generating the per-input channel heat map of the predictive input channels comprises: determining a per-input channel activation output based on the per-input channel weight value for the predictive input channel and the feature map for the predictive input channel; as well as An activation function is applied to the per-input channel activation output to generate the per-input channel heat map for the predictive input channel.

10. The apparatus of claim 8, wherein generating the per-input channel heatmap for the predictive input channel comprises performing guided back-propagation.

Images