Predicting electrical component failure

Abstract

Methods, systems, and apparatus, including medium-encoded computer program products, for predicting electrical component faults. A first sensor measurement of a component of an electrical grid taken at a first time may be obtained. A second sensor measurement of the component taken at a second time may be identified, and the second time may be after the first time. Input, which may include a first sensor measurement and a second sensor measurement, may be processed using a machine learning model configured to process the component based on one or more changes in one or more characteristics of the component compared to the first sensor measurement as depicted in the second sensor measurement, a prediction is generated representing a likelihood that the component will experience a type of fault during the time interval. Data indicative of the prediction may be provided for presentation by a display.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 350,174, filed on June 8, 2022. The disclosure of the prior application is considered part of and incorporated by reference into the disclosure of the present application. Technical Field

[0003] The present description relates to electrical grids, and in particular to processes for predicting failures of components of the electrical grid. Background Art

[0004] Electric utility companies have hundreds of thousands of assets deployed in the field. When an asset fails (e.g., a transformer explodes), the failure can cause widespread power outages and present a life-threatening hazard. To prevent failures, simple heuristics can be used to determine when upgrades and replacements are recommended. For example, a utility company may have a policy to replace transformers after a fixed operating period (e.g., 20 years). However, while simple heuristics can be used to make approximate predictions, they may over-predict and under-predict failures. In the case of over-predicted failures, equipment is replaced prematurely, resulting in wasted costs and materials; for under-predicted failures, equipment fails unexpectedly, with potentially catastrophic consequences. For example, time-based heuristics can be used to determine when to replace a transformer, but the heuristics may over-predict failures of lightly loaded transformers in milder environments, or under-predict failures of high-load transformers in hot environments. Summary of the invention

[0005] Generally speaking, the present specification relates to processes for predicting failures of components of an electrical grid, and more specifically, the present disclosure relates to using two or more time series sensor measurements as input to a machine learning model configured to predict component failures.

[0006] One aspect is characterized by obtaining a first sensor measurement of a component of a power grid taken at a first time. A second sensor measurement of the component taken at a second time can be identified, and the second time can be after the first time. An input that can include the first sensor measurement and the second sensor measurement can be processed using a machine learning model, the machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement compared to the first sensor measurement. The time interval can be a time period after the second time. Data indicative of the prediction can be provided for presentation by a display.

[0007] In some embodiments, the sensor measurement may be an image, such as an optical image or a thermal image. In some embodiments, the sensor measurement may be an acoustic recording.

[0008] One or more of the following features may be included. The machine learning model may include a defect detection machine learning model and a fault prediction machine learning model. The machine learning model may include a fault prediction machine learning model. The fault prediction machine learning model may include a defect detection hidden layer. The prediction may include one or more of the probability that the component will fail in a single time period, the probability that the component will fail in each of a plurality of time periods, the mean time to failure, the probability distribution of failures, or the most likely period during which the component will fail. The characteristics of the component may include one or more of bulges, tilts, loose fasteners, missing fasteners, cracks, burn marks, rust, oil leaks, missing insulation or damaged insulation, operating sounds, or thermal quality. The machine learning model may be a recurrent neural network. The recurrent neural network may be a long short-term memory machine learning model or a transformer model based on cross-attention. The input may also include features of the component and features of the operating environment. Features of the operating environment may include a series of temperature values ​​measured at or around the location of the component. A machine learning model may be used to process inputs that may include features of a first sensor measurement and features of the operating environment, the machine learning model being configured to generate a prediction representing a recommended time for capturing one or more subsequent sensor measurements of the component.

[0009] In some embodiments, the first sensor measurement and the second sensor measurement are images of the component. A first acoustic recording of a component of the electrical network taken at a first time may be obtained. A second acoustic recording of the component taken at a second time may be identified. A second input, which may include the first acoustic recording and the second recording, may be processed using a second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second acoustic recording compared to the first acoustic recording. Data provided for presentation by a display may be determined based on a weighted combination of the prediction and the second prediction.

[0010] In some embodiments, the first sensor measurement and the second sensor measurement are optical images of the component. A first thermal image of a component of the electrical grid taken at a first time may be obtained. A second thermal image of the component taken at a second time may be identified. A second input including the first thermal image and the second thermal image may be processed using a second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second thermal image compared to the first thermal image. Data provided for presentation by a display may be determined based on a weighted combination of the prediction and the second prediction.

[0011] Particular embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. The techniques described below can be used to predict component failures using a series of sensor measurements (such as images) of the component taken over a period of time. By using multiple images of the component, the system can determine changes in defects of the component, including rates of change, to produce more accurate reliability predictions. The system can also produce more accurate reliability predictions by using predictions based on different types of sensor measurements (such as images and audio recordings) or different types of images. The system can also produce more accurate reliability predictions by using features of the operating environment of the component.

[0012] The details of one or more embodiments of the subject matter described in this specification are set forth in the drawings and the description below. Other features, aspects, and advantages of the invention will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 and Figure 2 is a graphical representation of component defects over time.

[0014] Figure 3A-3B is a diagram of an example system for predicting electrical component failure.

[0015] Figure 4 is a flow chart of an example process for predicting electrical component failure.

[0016] Figure 5 is a graphical representation of component defects detectable in thermal images over a period of time.

[0017] Figure 6 is a block diagram of an example computer system. DETAILED DESCRIPTION

[0018] This specification describes techniques for predicting the likelihood that a component of an electrical grid will fail within one or more specified time periods. The techniques may include evaluating sensor measurements of the component taken at multiple times. For example, the sensor measurements may include image data. For example, Figure 1 and Figure 2 is a graphical representation of component defects visible in the image over time. Figure 1 Transformer 100 is depicted at five time periods, 1990, 1995, 2000, 2005, and 2010, and the amount of rust 110, 120, 130a, 130b, 140a, 140b, 140c increases over time. For example, in 1990, transformer 100 had no rust. In 1995, transformer 100 had one smaller rust spot 110. In 2000, transformer 100 showed a larger rust spot 120. By 2005, transformer 100 included a larger rust spot 130a and a second smaller rust spot 130b. In 2010, transformer 100 included a very large rust spot 140a and two smaller rust spots 140b, 140c.

[0019] Both the presence of a defect (in this case corrosion) and the rate of change of the defect can be used to predict component failure. Figure 1 In the rust model, the amount of corrosion increases over time, which can predict failure, for example if a component is no longer able to function properly, or is unlikely to function properly, if the corrosion coverage exceeds a threshold.

[0020] on the contrary, Figure 2 A transformer 200 is shown with corrosion 210a, 210b, 220a, 220b at two time periods 1990 and 2020. Although the amount of corrosion is significant, it changes very little over the 30 year period. If the unit has not failed due to corrosion during this 30 year period, the slow rate of spread may indicate a lower probability that corrosion will cause a failure in the next few years. And although Figure 1 and Figure 2 Rust is shown as an example, but a wide variety of defects are contemplated. Examples of defects visible in the image may include bulges, tilts, loose or missing fasteners, cracks, burn marks, rust, oil leaks (e.g., oil stains), missing or damaged insulation or thermal mass, etc.

[0021] For these reasons, a system that considers only a single image and therefore cannot assess not only the presence of a defect but also the rate of change of the defect may miss predictive signals of a fault or non-fault. Accordingly, the present specification describes techniques for determining predictions by using a machine learning model that evaluates signals from multiple time periods.

[0022] In addition, a system that considers other types of sensor measurements can evaluate more predictive signals. For example, the system can consider image data, such as thermal images or audio recording data.

[0023] Figure 3A is a diagram of an example of a system 300 for predicting electrical component failures. In short, the system 300 can use a defect detection machine learning model to process inputs including sensor measurement data to determine which defects, if any, exist on the component. For example, the defect detection machine learning model can be a classification machine learning model, such as a convolutional neural network or any other suitable type of classification model. The system 300 can provide the sensor measurements to the defect detection machine learning model, and the defect detection machine learning model can determine an output including an encoding of the sensor measurements. The encoding can include an indication of the presence and type of the defect. The system 300 can use the defect detection machine learning model to process sensor measurements of the component taken at different times, and use multiple outputs as inputs to the fault prediction machine learning model, as described below.

[0024] A sensor measurement of a component may be obtained by a sensor for a specific point in time. For example, a sensor measurement may be an image taken of the component, or an audio recording taken near the component. The audio recording may capture, for example, sounds made by the component.

[0025] exist Figure 3A In the example of , the sensor measurement is an image. The image can be an optical image. In some embodiments, the image can be another type of image, such as a thermal image.

[0026] Thus, the system 300 can process inputs including images using a defect detection machine learning model to determine which defects, if any, are present on a component. The system 300 can provide the image to the defect detection machine learning model, and the defect detection machine learning model can determine an output including an encoding of the image. The encoding can include an indication of the presence and type of defects. The system 300 can process images of the component taken at different times using the defect detection machine learning model, and use multiple outputs as inputs to a fault prediction machine learning model, as described below.

[0027] Examples of defects may include bulges, tilting, loose or missing fasteners, cracks, burn marks, rust, oil leaks (e.g., oil stains), missing or damaged insulation, operating sounds or thermal mass, etc.

[0028] Image data may be obtained from a variety of sources. For example, the owner of the component may capture images at periodic intervals. Images may be obtained from other parties, such as vehicles that include cameras (such as self-driving cars), photo sharing sites (assuming the owner of the photo approves such use), etc.

[0029] To determine the likelihood of failure, the system may process input comprising the output of a defect detection machine learning model for two images of a component using a fault prediction machine learning model configured to generate a prediction associated with failure of the component within a time period.

[0030] The inputs may also include a grid map, characteristics of the components, and characteristics of the operating environment. Characteristics of the operating environment may include, but are not limited to, the number and timing of power outages, brownouts, lightning strikes, and blown fuses, and weather conditions (e.g., temperature and humidity). In addition, the characteristics of the operating environment may include one or more series of values. For example, such a series may include temperature values ​​measured at or around the location of the component at multiple points in time.

[0031] In embodiments where the sensor measurements include thermal images, the system can use characteristics of the operating environment to distinguish changes in components from changes in the environment. For example, thermal images can be taken at different times of the year or under different environmental conditions. Different environmental conditions may affect the temperatures present in the thermal images. Thus, the system can use characteristics such as the temperature of the environment to compare the thermal quality of the component at different points in time, isolated from changes in the environment.

[0032] In embodiments where the sensor measurements include thermal images, for example, the system can use characteristics of the operating environment to determine the thermal mass of a component. For example, the system can use temperature values ​​measured at or around the location of the component (taken at a point in time within the same time window as the thermal image of the component) to determine the ambient temperature of the environment of the component. Thus, the system can obtain temperature information by comparing the temperature present in the thermal image with the ambient temperature. As another example, the system can use weather conditions such as humidity to perform a humidity analysis. For example, humid air or air with higher humidity has a higher heat capacity and is a better conductor of heat than dry air. The humidity condition of the air surrounding the component can affect the temperature of the component. Thus, the system can use thermal images and humidity information to determine the thermal mass of the component in the context of the environment.

[0033] Characteristics of the components may include, but are not limited to, brand, model, duration of use, rating, thermal constants, winding type, and load metrics (maximum load, average load, time at maximum load, etc.) A grid diagram may include, for example, the components present in the grid, their interconnection pattern, and the distances between elements.

[0034] The defect detection machine learning model can be a neural network. In some embodiments, the defect detection machine learning model is a long short-term memory (LSTM) model. LSTM models differ from feed-forward models in that they can process sequences of data, such as sensor measurements of a component (or outputs from processing sensor measurements) over multiple time periods. In some embodiments, the defect detection machine learning model is a transformer model based on cross-attention.

[0035] Examples of predictions of the fault prediction machine learning model may include, but are not limited to, the likelihood that a component will fail in a single time period, the likelihood that a component will fail in each of a plurality of time periods, the mean time to failure, and the most likely period during which a component will fail. Furthermore, the fault prediction machine learning model may be configured to produce one or more of these outputs.

[0036] Fault prediction machine learning models can be evaluated in response to various triggers. For example, the model can be evaluated whenever new data (e.g., images of components) arrives, at periodic intervals, and when a user requests an evaluation (e.g., during a maintenance planning exercise).

[0037] In some embodiments, the defect detection machine learning model is a component of the fault prediction machine learning model (described above). For example, defect detection can be performed by one or more hidden layers within the fault prediction machine learning model, and the outputs from those layers can be used by other layers of the fault prediction machine learning model.

[0038] The system 300 can train a fault prediction machine learning model using training examples including feature values ​​and results. The result can indicate whether a component failed during a given time period. For example, a value of "1" can indicate a failure, and a value of "0" can indicate no failure. The feature values ​​can include two or more images of the component, a grid map, features of the component, and features of the operating environment, as described above.

[0039] The system 300 may include a feature acquisition engine 310, an image recognition engine 320, an evaluation engine 330, and a prediction providing engine 340. The engines 310, 320, 330, and 340 may be provided as one or more computer executable software modules, hardware modules, or a combination thereof. For example, one or more of the engines 310, 320, 330, and 340 may be implemented as a software code block having instructions for causing one or more processors of the system 300 to perform the operations described herein. Additionally or alternatively, one or more of the engines 310, 320, 330, and 340 may be implemented in an electronic circuit, such as, for example, a programmable logic circuit, a field programmable logic array (FPGA), or an application specific integrated circuit (ASIC).

[0040] The feature acquisition engine 310 may acquire feature data associated with component failures. The feature data may include, but is not limited to, images 305a, 305b of electrical components and elements associated with potential failures of the electrical components (such as structural support elements). Examples of components may include, but are not limited to, transformers, fuses, wires, and related structures such as poles, cross arms, insulators, and lightning arresters.

[0041] Visual indicators associated with component failure that may be present in images 305a, 305b may include defects in the component itself, such as corrosion (eg, Figure 1 and Figure 2 ), cracks, holes, deformities, etc., defects in any supporting structure (e.g., a utility pole that may have begun to lean over time), or a combination thereof. For example, indicators associated with component failure that may be present in a thermal image may include higher than normal operating temperatures or hot spots on the component. The image may be encoded in any suitable format, including but not limited to Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), or a lossless format such as RAW.

[0042] In some embodiments, the feature acquisition engine 310 can obtain additional feature data. For example, the additional feature data can include a grid map, features of the components, and features of the operating environment. Features of the operating environment can include, but are not limited to, the number and timing of power outages, brownouts, lightning strikes, and blown fuses, as well as weather and environmental conditions (e.g., temperature, humidity, vegetation level). Features of components can include, but are not limited to, brand, model, duration of use, rating, thermal constant, winding type, service history, and load metrics (maximum load, average load, time at maximum load, etc.). The grid map can include, for example, the components present in the grid, their interconnection patterns, and the distances between elements.

[0043] The feature data may also include metadata describing the feature data, such as a timestamp of the feature data (e.g., the date and time the image was captured), a timestamp of when the feature data was obtained, a location (e.g., the location of the image capture device and / or the location of an object captured in an image provided by GPS or other components), a provider of the feature data, an asset identifier (e.g., provided by the person who captured the image of the asset), and the like.

[0044] The feature acquisition engine 310 may use various techniques to obtain feature data. In some embodiments, the feature acquisition engine 310 retrieves feature data from a data repository such as a database and a file system. The feature acquisition engine 310 may collect feature data at regular intervals (e.g., daily, weekly, monthly, etc.) or upon receiving an indication of a data change. In some embodiments, the feature acquisition engine 310 may include an application programming interface (API) through which feature data may be provided to the feature acquisition engine 310. For example, the API may be a Web service API.

[0045] Image recognition engine 320 may accept an image of an electrical component and determine whether one or more other images depict the same electrical component. Image recognition engine 320 may include an object recognition machine learning model, such as a convolutional neural network (CNN) or a Barlow Twins model, configured to recognize objects in an image.

[0046] In some implementations, the image recognition engine 320 may evaluate metadata associated with the characteristics of the electrical components. For example, if the metadata includes the location of the asset, and the image recognition engine 320 determines that the locations of the two assets are different, the image recognition engine 320 may determine that the images depict different electrical components. Similarly, if the metadata includes an asset identifier of the asset, and the image recognition engine 320 determines that the asset identifiers of the two assets are different, the image recognition engine 320 may determine that the images depict different electrical components.

[0047] The evaluation engine 330 can accept the feature data (as described above) and evaluate one or more machine learning models to generate predictions related to electrical component failures. Examples of predictions of the failure prediction machine learning model can include, but are not limited to, the likelihood that a component will fail in a single time period, the likelihood that a component will fail in each of a plurality of time periods, the mean time to failure, the distribution of failure probabilities, and the most likely time period during which a component will fail.

[0048] The assessment engine 330 may include one or more machine learning models. In some embodiments, the assessment engine 330 includes a fault prediction neural network 334 configured to accept inputs and generate predictions, such as the prediction types listed above. In some embodiments, the assessment engine 330 includes one fault prediction neural network 334 that generates one or more prediction types. In some embodiments, the assessment engine 330 includes multiple fault prediction neural networks 334, each fault prediction neural network 334 generating one or more prediction types.

[0049] As described above, the input may include images of the asset at multiple time periods. In addition, the input features may also include, but are not limited to, the grid map, the characteristics of the components, and the characteristics of the operating environment. The characteristics of the operating environment may include, but are not limited to, the number and timing of power outages, brownouts, lightning strikes, and blown fuses, as well as weather conditions (e.g., temperature and humidity). The characteristics of the components may include, but are not limited to, brand, model, duration of use, rating, thermal constant, winding type, and load metrics (maximum load, average load, time at maximum load, etc.). The grid map may include, for example, the components present in the grid, their interconnection pattern, and the distances between elements.

[0050] In some embodiments, the assessment engine 330 includes a defect detection machine learning model 332 and one or more fault prediction machine learning models 334. To determine which defects, if any, are present on a component, the system can process inputs including one or more images of the component using the defect detection machine learning model 332. The defect detection machine learning model 332 can be a neural network, and in some embodiments, the defect detection machine learning model 332 is a recurrent neural network (e.g., a long short-term memory (LSTM) model) or another type of sequential machine learning model. Recurrent models differ from feed-forward models in that they can process sequences of data, such as images of components (or outputs from processing images), over multiple time periods.

[0051] The system can provide inputs (which include images) to the defect detection machine learning model 332, and the defect detection machine learning model 332 can generate outputs including an encoding of the image. The encoding can include an indication of the presence and type of defects. The system can process images of the component taken at different times using the defect detection machine learning model 332, and use one or more outputs as inputs to the fault prediction machine learning model 334. The system can then process the inputs including the outputs of the defect detection machine learning model and other feature data (described above) using a machine learning model configured to generate a prediction describing the likelihood of a fault.

[0052] In some embodiments, the defect detection machine learning model is a component of the fault prediction machine learning model 334. For example, defect detection can be performed by one or more hidden layers within the fault prediction machine learning model 334, and the outputs from those layers can be used by other layers of the fault prediction machine learning model.

[0053] The prediction providing engine 340 can provide one or more predictions generated by the evaluation engine 330. In some embodiments, the prediction providing engine 340 can generate user interface presentation data 345 that, when rendered by a client device, causes the client device to display the predictions. In some embodiments, the prediction providing engine 340 can send the one or more predictions to a network-connected device, including a storage device and a database.

[0054] Figure 3B is a diagram of an example of a system 350 for predicting electrical component failures. The system 350 is similar to Figure 3A The system 300, however, can process inputs including different types of sensor measurement data.

[0055] The system 350 may include a feature acquisition engine 310, an image recognition engine 320, an audio feature acquisition engine 371, an audio recognition engine 371, an evaluation engine 380, and a prediction providing engine 340. The engines 361, 371, and 380 may be provided as one or more computer executable software modules, hardware modules, or a combination thereof. For example, one or more of the engines 361, 371, and 380 may be implemented as a software code block having instructions that cause one or more processors of the system 350 to perform the operations described herein. Additionally or alternatively, one or more of the engines 361, 371, and 380 may be implemented in an electronic circuit, such as, for example, a programmable logic circuit, a field programmable logic array (FPGA), or an application specific integrated circuit (ASIC).

[0056] The audio feature acquisition engine 361 is similar to the feature acquisition engine 310 and can obtain audio feature data related to component failure. The audio feature data can include, but is not limited to, audio recordings 306a, 306b of electrical components and audio recordings 306a, 306b of elements related to potential failure of the electrical components (such as structural support elements).

[0057] Audio indicators associated with component failure may be present in audio recording 306a or 306b, and may include defects in the component itself, such as abnormal operating sounds (such as humming), or defects in any supporting structure (e.g., clanging sounds from loose connections), or a combination thereof. The audio recording may be encoded in any suitable format, including but not limited to a spectrogram or other audio format.

[0058] For example, audio recording 306b may include audio characteristics that indicate that the operation of a component is louder or abnormal compared to normal operation or the audio characteristics of audio recording 306a.

[0059] In some implementations, the audio feature derivation engine 361 can obtain additional feature data, as described with reference to the feature derivation engine 310. The feature data can include metadata describing the feature data, such as a timestamp of the feature data (e.g., the date and time the audio recording was captured), a timestamp when the feature data was obtained, a location (e.g., the location of the audio recording capture device and / or the location of an object captured in the audio recording provided by GPS or other means), a provider of the feature data, an asset identifier (e.g., provided by the person who captured the audio recording of the asset), and the like.

[0060] The audio feature acquisition engine 361 may obtain feature data using various techniques as described with reference to the feature acquisition engine 310 .

[0061] The audio recognition engine 371 is similar to the image recognition engine 320 and can accept an audio recording of an electrical component and determine whether one or more other audio recordings depict the same electrical component. The audio recognition engine 371 can include a machine learning model configured to recognize the sounds made by the electrical component in the audio recording.

[0062] In some implementations, the audio recognition engine 371 can evaluate metadata associated with the characteristics of the electrical component. For example, if the metadata includes the location where the audio recording was captured, the audio recognition engine 371 can determine that the location of the audio recording is different at a threshold distance, and the image recognition engine 371 can determine that the audio recording captures a different electrical component. Similarly, if the metadata includes an asset identifier for an asset, and the audio recognition engine 371 determines that the asset identifiers of two assets are different, the audio recognition engine 371 can determine that the image depicts a different electrical component.

[0063] Evaluation engine 380 is similar to evaluation engine 330, but may include additional machine learning models. For example, evaluation engine 380 may include a fault prediction neural network configured to accept inputs and generate predictions. In some embodiments, evaluation engine 380 may include separate fault prediction neural networks, such as fault prediction neural network 334 and fault prediction neural network 384, which are configured to generate predictions for different types of inputs.

[0064] As described above, the input to the fault prediction neural network 334 may include images of the asset at multiple time periods. The input to the separate fault prediction neural network 384 may include audio recordings of the asset at multiple time periods. In addition, the input features may also include, but are not limited to, the grid map, the characteristics of the components, and the characteristics of the operating environment. The characteristics of the operating environment may include, but are not limited to, the number and timing of power outages, brownouts, lightning strikes, and blown fuses, as well as weather conditions (e.g., temperature and humidity). The characteristics of the components may include, but are not limited to, brand, model, duration of use, rating, thermal constant, winding type, and load metrics (maximum load, average load, time at maximum load, etc.). The grid map may include, for example, the components present in the grid, their interconnection patterns, and the distances between elements.

[0065] In some embodiments, the assessment engine 380 includes one or more defect detection machine learning models (such as defect detection machine learning model 332 and defect detection machine learning model 382) and one or more fault prediction machine learning models (such as 334 and 384). To determine which defects, if any, exist on a component, the system can use the defect detection machine learning model 332 to process inputs including one or more images of the component. To determine which defects, if any, exist on a component, the system can use the defect detection machine learning model 382 to process inputs including one or more audio recordings of the component. The defect detection machine learning model 382 can be a neural network, and in some embodiments, the defect detection machine learning model 382 is a recurrent neural network (e.g., a long short-term memory (LSTM) model) or another type of sequential machine learning model.

[0066] The system may provide inputs (which include images or audio recordings) to the corresponding defect detection machine learning model 332 or defect detection machine learning model 382. The defect detection machine learning model 332 may generate an output including an encoding of the image. The defect detection machine learning model 382 may generate an output including an encoding of the audio recording. The encoding may include an indication of the presence and type of defects. The system may process images of components taken at different times using the defect detection machine learning model 332 and use one or more outputs as inputs to the fault prediction machine learning model 334. The system may process audio recordings of components taken at different times using the defect detection machine learning model 382 and use one or more outputs as inputs to the fault prediction machine learning model 384. Next, the system may process inputs including the output of the defect detection machine learning model 332 and other feature data (described above) using a machine learning model configured to generate a first prediction describing the likelihood of a fault. Next, the system may process inputs including the output of the defect detection machine learning model 382 and other feature data (described above) using a machine learning model configured to generate a second prediction describing the likelihood of a fault. The system may determine a final prediction based on a weighted combination of the first prediction and the second prediction.

[0067] In some embodiments, as described above, the defect detection machine learning model is a component of the fault prediction machine learning model 334 or the fault prediction machine learning model 384.

[0068] Figure 4 4 is a flow chart of an example process for predicting electrical component failure. For convenience, process 300 will be described as being performed by a system for predicting electrical component failure (e.g., system 300 for predicting electrical component failure of FIG. 3 ), which is appropriately programmed to perform the process. The operations of process 400 may also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and execution of the instructions by one or more data processing devices may cause one or more data processing devices to perform the operations of process 400. One or more other components described herein may perform the operations of process 400.

[0069] The system obtains (410) a first sensor measurement of a component of the electrical grid taken at a particular time. The sensor measurement may include, for example, an image or an audio recording. The sensor measurements, including the first sensor measurement, may be obtained from a variety of sources. For example, an owner of the component may capture images at periodic intervals. In another example, the images may be obtained from other parties, such as vehicles including cameras, such as self-driving cars, drones, photo sharing websites (assuming the owner of the photo approves such use), etc.

[0070] The system identifies (420) a second sensor measurement of the component taken at a later time. The system can process each sensor measurement in the set of the first sensor measurement and the second sensor measurement using a machine learning model that is configured to determine whether the electrical component in the first sensor measurement is also present in the second sensor measurement. For example, if the sensor measurement is an image, the system can use an object detection machine learning model that is configured to determine whether the electrical component depicted in the first image is also present in the second image. For each of one or more second images (extracted from the set), the system can use the machine learning model to determine a predicted likelihood that the component is present in the second image. If the system determines that the predicted likelihood meets a threshold, the system determines that the second image contains the component. In some embodiments, the system can process the machine learning model using all second images and the first image in the set.

[0071] In some embodiments, the system may use metadata from each sensor measurement in a set of first sensor measurements and second sensor measurements to determine whether an electrical component in the first sensor measurement is also present in the second sensor measurement. For example, the system may use metadata from each image in a set of first images and second images to determine whether an electrical component depicted in the first image is also present in the second image. For example, location data (e.g., a GPS reading) of the first image may be compared to location data of each image in a set of second images. If the locations of the images are the same or within a threshold distance, the system may determine that the component is depicted in both images. The threshold distance may be predefined or calculated based on the geographic distribution of similar assets within a geographic area. For example, a larger threshold distance may be used for more rural areas with fewer transformers per unit area, while a smaller threshold distance may be used for urban areas with more transformers per unit area.

[0072] The machine learning model may obtain a set of second sensor measurements using the techniques of operation 410 or similar techniques. Additionally, in some embodiments, once the sensor measurements obtained in operation 410 have been evaluated using process 400, the sensor measurements may be retained in operation 420 for future use.

[0073] In some embodiments, the system is provided with a first sensor measurement and a second sensor measurement having a component, and thus identifies the second sensor measurement when providing the sensor measurement. For example, a user may call an API provided by the system to provide the first sensor measurement and the second sensor measurement.

[0074] The system optionally obtains (430) additional characteristic data related to the electrical component failure. The additional characteristic data may include a grid map, characteristics of the component, and characteristics of the operating environment, as described above.

[0075] The system may obtain additional feature data using various means. The system may retrieve data from an information source using an API provided by the data source. The system may retrieve data from various databases using structured query language (SQL) operations. The system may retrieve data from a file system using conventional file system operations. The system may provide an API, and a user of the system (which may be a computing device) may call the API to provide data.

[0076] The system processes (440) inputs including at least a first sensor measurement and a second sensor measurement using one or more machine learning models, the one or more machine learning models being configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement compared to the first sensor measurement, wherein the time interval is a time period after the second time.

[0077] To determine which defects, if any, exist on a component, the system may use a defect detection machine learning model to process inputs including sensor measurements. The system may provide two or more sensor measurements of a component to the defect detection machine learning model, and the defect detection machine learning model may determine an output including an encoding of the sensor measurements. The encoding may include an indication of the presence and type of defect. The system may use a defect detection machine learning model to process sensor measurements of a component taken at different times, and use the multiple outputs as inputs to a fault prediction machine learning model.

[0078] To determine the likelihood of a failure, the system may process an input including an output of a fault prediction machine learning model for two or more images of a component using a fault prediction machine learning model, the fault prediction machine learning model being configured to generate a prediction associated with a failure of the component within a certain time period. As described above, the input may also include additional feature data.

[0079] The fault prediction machine learning model can be evaluated in response to various triggers. For example, the model can be evaluated whenever new data (e.g., images of components) arrives, at periodic intervals, and when a user requests an evaluation (e.g., during a maintenance planning exercise).

[0080] In some embodiments, the system may process inputs including first sensor measurements and other feature data (e.g., features of the component and features of the operating environment) without second sensor measurements. In such embodiments, the system may employ one or more machine learning models configured to generate predictions representing the likelihood that a component will experience a type of failure during a time interval. Such a machine learning model may be trained using back propagation on examples, where each example includes sensor measurements of the component, other feature data, and results. The other feature data may include features of the component and features of the operating environment. If the component fails during the time interval, the result may represent a failure, and if the component does not fail during the interval, the result may represent a success. Note that the features of the component may allow the machine learning model to learn which components fail under similar circumstances. For example, components of the same brand and model may fail under similar circumstances, and such failures will be present in the training data, allowing the machine learning model to learn failure patterns. In addition, components of the same type (e.g., transformers) may follow similar failure patterns, even if the patterns are slightly different due to differences in brand and model. Such an approach may provide initial failure predictions before a second image is available. Note that a "new" asset may be an asset that has been installed in the grid for some time but is newly entered into the system for predicting electrical component failures.

[0081] In some embodiments, the system can process inputs including a first sensor measurement and other feature data (e.g., features of the component and features of the operating environment) using one or more machine learning models configured to generate a prediction representing a recommended time for capturing one or more subsequent sensor measurements of the component. The machine learning model can be trained on examples including sensor measurements, other feature data, and labels. The label can represent a recommended duration before obtaining the next sensor measurement of the component.

[0082] To configure the model, the system can train a fault prediction machine learning model using training examples including feature values ​​and results. The result can indicate whether a component failed during a given time period. For example, a value of "1" can indicate a failure, and a value of "0" can indicate no failure. The feature values ​​can include two or more images of the component, a grid map, features of the component, and features of the operating environment, as described above.

[0083] In some embodiments, the first sensor measurement and the second sensor measurement can be images, and the system can also obtain a first acoustic recording of the component. For example, the acoustic recording can be taken at a location near the component so that the audio recording includes any sounds made by the component, such as operating sounds. The first acoustic recording can be taken at a specific time when the first image is taken. For example, the first acoustic recording can be taken at a time before or after the specific time when the first image is taken within a predefined time window. For example, the first acoustic recording can be taken a few seconds, minutes, hours, or days before or after the specific time when the first image is taken. The system can also identify a second acoustic recording of the component taken at a later time after the second image is taken. For example, the second acoustic recording can be taken at a time before or after the later time when the second image is taken within a predefined time window. For example, the second acoustic recording can be taken a few seconds, minutes, hours, or days before or after the later time when the second image is taken. The system can use a machine learning model to process each audio recording in the set of the first audio recording and the second audio recording, and the machine learning model is configured to determine whether the electrical component in the first audio recording is also present in the second audio recording.

[0084] In these embodiments, the system may process inputs including at least a first image and a second image using one or more machine learning models configured to generate, based on the image, a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second image compared to the first image, wherein the time interval is a time period after the second time. The system may process a second input including at least a first acoustic recording and a second acoustic recording using one or more machine learning models configured to generate, based on the audio recording, a second prediction representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second acoustic recording compared to the first acoustic recording.

[0085] In some embodiments, the first sensor measurement and the second sensor measurement can be optical images, and the system can also obtain a first thermal image of the component. The first thermal image can be obtained at a specific time when the first optical image is obtained. For example, the first thermal image can be obtained before or after the specific time when the first optical image is obtained within a predefined time window. For example, the first thermal image can be obtained a few seconds, minutes, hours, or days before or after the specific time when the first optical image is obtained. The system can also identify a second thermal image of the component obtained at a later time after the second optical image is obtained. For example, the second thermal image can be obtained before or after the later time when the second optical image is obtained within a predefined time window. For example, the second thermal image can be obtained a few seconds, minutes, hours, or days before or after the later time when the second optical image is obtained. The system can use a machine learning model to process each thermal image in the set of the first thermal image and the second thermal image, and the machine learning model is configured to determine whether the electrical component in the first thermal image is also present in the second thermal image.

[0086] In these embodiments, the system can process inputs including at least a first optical image and a second optical image using one or more machine learning models configured to generate a prediction based on the optical images representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second optical image compared to the first optical image, wherein the time interval is a time period after the second time. The system can process a second input including at least a first thermal image and a second thermal image using one or more machine learning models configured to generate a second prediction based on the thermal images representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second thermal image compared to the first thermal image. The second input can also include, for example, features of the operating environment, such as the temperature of the environment near the component.

[0087] In these embodiments, the system may determine data indicating the prediction based on a weighted combination of the prediction and the second prediction. For example, the system may multiply the prediction and the second prediction by a predetermined weight and add the weighted prediction to the weighted second prediction to determine the final prediction.

[0088] The system provides (450) data indicative of the prediction for presentation by the display.The system can provide the presentation data by sending the data to the client device via a network or storing the presentation data in a data store (eg, a file system or a database).

[0089] In embodiments where the system obtains sensor measurements as images and audio recordings, the system may provide data indicative of a final prediction based on a weighted combination of a prediction based on the images and a second prediction based on the audio recording. In embodiments where the system obtains optical images and thermal images, the system may provide data indicative of a final prediction based on a weighted combination of a prediction based on the optical images and a second prediction based on the thermal images.

[0090] Figure 5 is a graphical representation of component defects detectable in thermal images over a period of time. Figure 5 Insulator 500 is depicted at two time periods 1990 and 1995, and areas of different temperatures or hot spots 510 increase over time. For example, in 1990, insulator 500 did not have any hot spots. In 1995, insulator 510 had one smaller hot spot 510. Hot spots 510 may indicate tracking or degradation on the surface of insulator 500 that negatively affects the function of insulator 500. Hot spots 510 may be detectable or present in a thermal image of insulator 500.

[0091] Both the presence of a defect (in this example indicating a tracked hot spot) and the rate of change of the defect can be used to predict component failure. Figure 5 In FIG. 5 , hot spots 510 increase over time, which may predict failure, for example, if a component is no longer able to function properly, or is less likely to function properly, if the number or area of ​​hot spots exceeds a threshold.

[0092] And although Figure 5 Tracking is described as an example, but a wide variety of defects may be considered. Examples of defects detectable in thermal images may include missing or damaged insulation, operating hot spots or thermal mass (such as the operating temperature of a component), and the like.

[0093] A system that considers the thermal history of a component or the thermal mass of a component at different points in time can utilize predictive signals of failure or non-fault based on the thermal history. For example, a component that is exposed to higher temperatures in the environment or operates at higher temperatures may wear faster than a component that is exposed to lower temperatures or operates at lower temperatures. A component that is exposed to higher temperatures for a longer period of time may wear faster than a component that is exposed to higher temperatures for a shorter period of time. A component that is exposed to a higher rate of temperature change may wear faster than a component that is exposed to a slower rate of temperature change.

[0094] Figure 66 is a block diagram of an example computer system 600 that can be used to perform the above operations. System 600 includes a processor 610, a memory 620, a storage device 630, and an input / output device 640. Each of components 610, 620, 630, and 640 can be interconnected, for example, using a system bus 650. Processor 610 is capable of processing instructions for execution within system 600. In one embodiment, processor 610 is a single-threaded processor. In another embodiment, processor 610 is a multi-threaded processor. Processor 610 is capable of processing instructions stored in memory 620 or on storage device 630.

[0095] The memory 620 stores information within the system 600. In one embodiment, the memory 620 is a computer readable medium. In one embodiment, the memory 620 is a volatile memory unit. In another embodiment, the memory 620 is a non-volatile memory unit.

[0096] The storage device 630 can provide mass storage for the system 600. In one embodiment, the storage device 630 is a computer-readable medium. In various embodiments, the storage device 630 may include, for example, a hard disk device, an optical disk device, a storage device shared by multiple computing devices over a network (e.g., a cloud storage device), or some other mass storage device.

[0097] The input / output device 640 provides input / output operations for the system 600. In one embodiment, the input / output device 640 may include one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-252 port), and / or a wireless interface device (e.g., an 802.11 card). In another embodiment, the input / output device may include a driver device configured to receive input data and send output data to other input / output devices (e.g., a keyboard, a printer, and a display device 660). However, other embodiments may also be used, such as mobile computing devices, mobile communication devices, set-top TV client devices, etc.

[0098] Although already Figure 6 An example processing system is described in the specification, but the subject matter and implementation of the functional operations described in this specification may be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them.

[0099] The embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented using one or more modules of computer program instructions encoded on a computer-readable medium, which are used to be executed by a data processing device or control the operation of the data processing device. The computer-readable medium can be a manufactured product, such as a hard drive in a computer system or an optical disk sold through a retail channel, or an embedded system. The computer-readable medium can be obtained separately and later encoded with one or more modules of computer program instructions, such as by delivering one or more modules of computer program instructions via a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

[0100] The term "data processing apparatus" encompasses all apparatus, devices and machines for processing data, including, for example, a programmable processor, a computer or a plurality of processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, a runtime environment, or a combination of one or more of them. Furthermore, the apparatus may employ a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0101] A computer program (also referred to as a program, software, software application, script, or code) may be written in any suitable form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any suitable form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0102] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0103] Processors suitable for executing computer programs include, for example, dedicated microprocessors. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (such as magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to receive data from one or more mass storage devices or transfer data to it or both. However, a computer need not have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0104] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computing device capable of providing information to a user. Information may be provided to the user in any form of sensory format, including visual, auditory, tactile, or a combination thereof. The computing device may be coupled to a display device, such as an LCD (liquid crystal display) display device, an OLED (organic light emitting diode) display device, another monitor, a head-mounted display device, or the like, for displaying information to the user. The computing device may be coupled to an input device. The input device may include a touch screen, a keyboard, and an indicating device, such as a mouse or a trackball, through which a user can provide input to the computing device. Other types of devices may also be used to provide interaction with a user; for example, feedback provided to the user may be any suitable form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any suitable form, including acoustic, voice, or tactile input.

[0105] A computing system may include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by means of a computer program running on a corresponding computer and having a client-server relationship with each other. The embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser, through which a user can interact with the implementation of the subject matter described in this specification), or includes any combination of one or more such back-end, middleware or front-end components. The components of the system may be interconnected by digital data communication (e.g., a communication network) in any suitable form or medium. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), interconnected networks (e.g., the Internet), and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0106] In addition to the above embodiments, the following embodiments are also innovative:

[0107] Embodiment 1 is a method for predicting power grid asset failures, comprising:

[0108] obtaining a first sensor measurement of a component of the electrical grid taken at a first time;

[0109] identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time;

[0110] processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and

[0111] Data indicative of the prediction is provided for presentation by a display.

[0112] Embodiment 2 is a power grid asset fault prediction method according to Embodiment 1, wherein the machine learning model includes a defect detection machine learning model and a fault prediction machine learning model.

[0113] Embodiment 3 is a method for predicting faults of power grid assets according to any one of Embodiments 1-2, wherein the machine learning model includes a fault prediction machine learning model.

[0114] Embodiment 4 is a method for predicting faults of power grid assets according to Embodiment 3, wherein the fault prediction machine learning model includes a defect detection hidden layer.

[0115] Embodiment 5 is a method for predicting power grid asset failures according to any one of embodiments 1-4, wherein the prediction includes one or more of the following: the likelihood that a component will fail in a single time period, the likelihood that a component will fail in each of multiple time periods, the mean failure time, the distribution of failure probabilities, or the most likely time period when a component will fail.

[0116] Embodiment 6 is a grid asset fault prediction method according to any one of embodiments 1-5, wherein the characteristics of the component include one or more of the following: protrusions, tilts, loose fasteners, missing fasteners, cracks, burns, rust, oil leaks, missing insulation, damaged insulation, operating sounds, or thermal mass.

[0117] Embodiment 7 is a method for predicting power grid asset failures according to any one of embodiments 1 to 6, wherein the machine learning model is a recurrent neural network.

[0118] Embodiment 8 is a grid asset fault prediction method according to Embodiment 7, wherein the recurrent neural network is a long short-term memory machine learning model or a cross-attention based converter model.

[0119] Embodiment 9 is a method for predicting faults in power grid assets according to any one of embodiments 1-8, wherein the input also includes characteristics of the components and characteristics of the operating environment of the components.

[0120] Embodiment 10 is a method for predicting faults in power grid assets according to embodiment 9, wherein the characteristics of the operating environment include a series of temperature values ​​measured at or around the location of the component.

[0121] Embodiment 11 is a method for predicting faults of power grid assets according to any one of embodiments 1-10, wherein the sensor measurement is an acoustic recording of the component.

[0122] Embodiment 12 is a method for predicting faults in power grid assets according to any one of embodiments 1-11, wherein the sensor measurement is an image of the component.

[0123] Embodiment 13 is a method for predicting faults in power grid assets according to any one of embodiments 1-12, wherein the sensor measurement is an image of the component, and the method further comprises:

[0124] obtaining a first acoustic recording of a component of the electrical network taken at a first time;

[0125] identifying a second acoustic recording of the component taken at a second time;

[0126] processing a second input comprising the first acoustic recording and the second acoustic recording using a second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second acoustic recording as compared to the first acoustic recording; and

[0127] The data is determined based on a weighted combination of the prediction and the second prediction.

[0128] Embodiment 14 is a method for predicting faults in power grid assets according to any one of embodiments 1-13, wherein the sensor measurement is an optical image of the component, and the method further comprises:

[0129] obtaining a first thermal image of a component of the electrical grid taken at a first time;

[0130] identifying a second thermal image of the component taken at a second time;

[0131] processing a second input comprising the first thermal image and the second thermal image using a second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second thermal image compared to the first thermal image; and

[0132] The data is determined based on a weighted combination of the prediction and the second prediction.

[0133] Embodiment 15 is a method for predicting power grid asset failures according to any one of embodiments 1 to 14, further comprising:

[0134] Inputs including the first sensor measurement and a characteristic of the operating environment are processed using a machine learning model configured to generate a prediction representing a recommended time for capturing one or more subsequent sensor measurements of the component.

[0135] Embodiment 16 is a system comprising one or more computers and one or more storage devices storing instructions, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations comprising:

[0136] obtaining a first sensor measurement of a component of the electrical grid taken at a first time;

[0137] identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time;

[0138] processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and

[0139] Data indicative of the prediction is provided for presentation by a display.

[0140] Embodiment 17 is a system according to embodiment 16, wherein the machine learning model includes a defect detection machine learning model and a fault prediction machine learning model.

[0141] Embodiment 18 is a system according to any one of embodiments 16-17, wherein the machine learning model includes a fault prediction machine learning model.

[0142] Embodiment 19 is a system according to embodiment 18, wherein the fault prediction machine learning model includes a defect detection hidden layer.

[0143] Embodiment 20 is one or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

[0144] obtaining a first sensor measurement of a component of the electrical grid taken at a first time;

[0145] identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time;

[0146] processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and

[0147] Data indicative of the prediction is provided for presentation by a display.

[0148] Although this specification contains many implementation details, these should not be interpreted as limitations on the scope of what is claimed or may be claimed, but rather as descriptions of features specific to a particular embodiment of the disclosed subject matter. Certain features described in the context of a single embodiment in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination. Therefore, unless otherwise expressly stated, or unless otherwise expressly stated by the knowledge of a person of ordinary skill in the art, any feature of the above-described embodiments may be combined with any other feature of the above-described embodiments.

[0149] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequence, or that all of the operations shown be performed, to achieve the desired results. In some cases, multitasking and / or parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0150] Thus, certain embodiments of the invention have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results.

Claims

1. A method for predicting power grid asset failures, comprising: obtaining a first sensor measurement of a component of the electrical grid taken at a first time; identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time; processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and Data indicative of the prediction is provided for presentation by a display.

2. The method for predicting power grid asset failure according to claim 1, wherein: The machine learning model includes a defect detection machine learning model and a fault prediction machine learning model.

3. The method for predicting power grid asset failure according to claim 1, wherein: The machine learning model includes a fault prediction machine learning model.

4. The method for predicting power grid asset failure according to claim 3, wherein: The fault prediction machine learning model includes a defect detection hidden layer.

5. The method for predicting power grid asset failure according to claim 1, wherein: The prediction includes one or more of the following: the likelihood that a component will fail in a single time period, the likelihood that a component will fail in each of a plurality of time periods, the mean time to failure, a distribution of failure probabilities, or the most likely period during which a component will fail.

6. The method for predicting power grid asset failure according to claim 1, wherein: Characteristics of the component include one or more of the following: bulges, tilting, loose fasteners, missing fasteners, cracks, burn marks, rust, oil leaks, missing insulation, damaged insulation, operating sounds, or thermal mass.

7. The method for predicting power grid asset failure according to claim 1, wherein: The machine learning model is a recurrent neural network.

8. The method for predicting power grid asset failure according to claim 7, wherein: Recurrent Neural Networks are Long Short-Term Memory machine learning models or criss-cross attention based Transformer models.

9. The method for predicting power grid asset failure according to claim 1, wherein: The input also includes characteristics of the component and characteristics of the environment in which the component operates.

10. The method for predicting power grid asset failure according to claim 9, wherein: The operating environment is characterized by a series of temperature values ​​measured at or around the location of the component.

11. The method for predicting power grid asset failure according to claim 1, wherein: The sensor measurement is an acoustic recording of the component.

12. The method for predicting power grid asset failure according to claim 1, wherein: The sensor measurement is an image of the component.

13. The method for predicting power grid asset failure according to claim 1, wherein: The sensor measurement is an image of the component, the method further comprising: obtaining a first acoustic recording of a component of the electrical network taken at a first time; identifying a second acoustic recording of the component taken at a second time; processing a second input comprising the first acoustic recording and the second acoustic recording using a second machine learning model, the second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second acoustic recording as compared to the first acoustic recording; and The data is determined based on a weighted combination of the prediction and the second prediction.

14. The method for predicting power grid asset failure according to claim 1, wherein: The sensor measurement is an optical image of the component, the method further comprising: obtaining a first thermal image of a component of the electrical grid taken at a first time; identifying a second thermal image of the component taken at a second time; processing a second input comprising the first thermal image and the second thermal image using a second machine learning model configured to generate a second prediction representing a likelihood that the component will experience a type of failure during the time interval based on one or more changes in one or more characteristics of the component as depicted in the second thermal image compared to the first thermal image; and The data is determined based on a weighted combination of the prediction and the second prediction.

15. The method for predicting power grid asset failure according to claim 1, further comprising: Inputs including a first sensor measurement and a characteristic of an operating environment are processed using a machine learning model configured to generate a prediction representing a recommended time for capturing one or more subsequent sensor measurements of the component.

16. A system comprising one or more computers and one or more storage devices storing instructions, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations comprising: obtaining a first sensor measurement of a component of the electrical grid taken at a first time; identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time; processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and Data indicative of the prediction is provided for presentation by a display.

17. The system of claim 16, wherein: The machine learning model includes a defect detection machine learning model and a fault prediction machine learning model.

18. The system of claim 16, wherein: The machine learning model includes a fault prediction machine learning model.

19. The system of claim 18, wherein: The fault prediction machine learning model includes a defect detection hidden layer.

20. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising: obtaining a first sensor measurement of a component of the electrical grid taken at a first time; identifying a second sensor measurement of the component taken at a second time, wherein the second time is after the first time; processing inputs including the first sensor measurement and the second sensor measurement using a machine learning model configured to generate a prediction representing a likelihood that the component will experience a type of failure during a time interval based on one or more changes in one or more characteristics of the component as depicted in the second sensor measurement as compared to the first sensor measurement, wherein the time interval is a time period after the second time; and Data indicative of the prediction is provided for presentation by a display.

Images