Underwater machine performance analysis using surface sensors

Abstract

A system, method, and computer program product perform an audiovisual inspection at a liquid surface above a machine working beneath the surface. The audiovisual examination includes feeding surface wave motion into a neural network, feeding a bubble formation pattern into the neural network, feeding bubble size into the neural network, and feeding underwater acoustic information into each of the neural networks. The system, method, and computer program product also identify statistical anomalies indicative of performance anomalies of the machine from the audiovisual check using a neural network.

Description

Background Art

[0001] In various applications, it is advantageous and / or necessary for a machine to operate under a liquid (e.g., underwater). This may include one or more moving parts that actively contact the water (e.g., rather than being within a sealed housing) when they are actuated. Examples of underwater machines include oil extraction machinery, underwater navigation systems, ships, underwater pumps, etc. Summary of the invention

[0002] Aspects of the present disclosure relate to a method, system, and computer program product involving analyzing an underwater machine using a surface sensor. For example, the method includes performing an audio-visual inspection at a surface above a machine that is operating below the surface of a liquid. The audio-visual inspection includes feeding surface wave motion into a neural network, feeding bubble formation patterns into the neural network, feeding bubble sizes into the neural network, and feeding underwater acoustic information into each of the neural networks. The method also includes using the neural network to identify statistical anomalies from the audio-visual inspection that indicate abnormal performance of the machine. Systems and computer programs configured to perform the above methods are also described herein.

[0003] The above summary is not intended to describe each illustrated embodiment or every implementation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The accompanying drawings included in this application are incorporated into the specification and form a part of the specification. They illustrate embodiments of the present disclosure and, together with the specification, are used to explain the principles of the present disclosure. The accompanying drawings only illustrate specific embodiments and do not limit the present disclosure.

[0005] Figure 1 A conceptual diagram depicts an example system in which a controller may analyze performance of an underwater machine using surface sensors.

[0006] Figure 2 Depicted Figure 1 An example flow chart of how a controller may analyze the performance of a subsea machine and determine corrective actions as needed.

[0007] Figure 3 Describes the ability to host and / or include Figure 1 A conceptual block diagram of a controller computing system.

[0008] Although the present invention can have various modifications and alternative forms, its details have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that its purpose is not to limit the present invention to the specific embodiments described. On the contrary, the present invention covers all modifications, equivalents and alternatives that fall within the scope of the present invention. DETAILED DESCRIPTION

[0009] Various aspects of the present disclosure relate to underwater machine analysis, and more specific aspects of the present disclosure relate to using data from surface sensors as a feed to a neural network to analyze the performance of the machine and determine corrective actions as needed. Although the present disclosure is not necessarily limited to such applications, various aspects of the present disclosure may be understood through a discussion of various examples using this context.

[0010] As we hope to settle in new places and make more and more use of the surrounding lakes and oceans, the use of underwater machinery is becoming more and more common. Due to the pressure applied by the water to the machine, the (hot or cold) water temperature, the flow rate of the water, etc., underwater machines may have various problems. For example, the pseudo-constant vibration caused by underwater work may be consistent with the resonant frequency of the material of the machine, and may cause microcracks, unbalanced force distribution, friction with the supporting structure, etc. The problems that may arise from underwater use are countless, such as loose bolts, bends in any particular pipeline, or any other type of mechanical / material / electrical failure that may arise due to the characteristics of underwater work. These problems may appear with different severity, so that some problems may be small and therefore can be ignored (at least for a period of time), while some problems may need to be solved immediately to avoid catastrophic damage to the machine.

[0011] Aspects of the present disclosure can detect these problems. For example, aspects of the present disclosure are configured to perform an audio-visual inspection using sensors at the water surface above the underwater machine. Specifically, if there is a problem with the underwater equipment, the underwater equipment may vibrate in an atypical manner due to how the machine moves underwater. The vibrations generated by the machine may create waves and / or bubbles on the water surface. In addition, the vibrations of the underwater machine may create new sounds, such as due to changes in one or more air columns.

[0012] Aspects of the present disclosure may detect these waves (physical or audio) and / or bubbles through audio-visual inspection via one or more water surface wave motion patterns, bubble formation patterns, bubble size analysis, underwater sound wave propagation patterns, etc. Aspects of the present disclosure are configured to identify any anomalies in the performance of the underwater machine exhibited by these bubbles and / or waves, and accordingly identify whether proactive corrective action is recommended (and if so, determine which corrective actions are recommended).

[0013] One or more computing devices including one or more processing units executing instructions stored on one or more memories can provide functionality to address these issues, where the computing device(s) are referred to herein as a controller. The controller can analyze how the machine generates waves and / or bubbles when creating a profile or baseline for the machine. The controller can then compare the generated waves and / or bubbles to the baseline for the machine, and when the waves and / or bubbles indicate a deviation from the baseline that is greater than a threshold, determine that the performance of the machine is abnormal.

[0014] The controller may further analyze and balance various factors in determining what corrective action to determine and / or determining whether to autonomously perform the corrective action. For example, the controller may determine if there is a danger level above a threshold, a quota that needs to be hit, an operating time frame that is about to expire, etc. in evaluating potential corrective actions. Corrective actions may include repairing the machine, changing the way the machine is operated, removing the machine from the water, etc. The controller may determine this based on the wave formation pattern on the water surface, the size of the bubbles at the water surface, the generation rate of the bubbles and / or waves.

[0015] For example, Figure 1 An environment 100 is depicted in which a controller 110 may analyze a machine 120 using a sensor 140. The controller 110 may use data from the sensor 140 to analyze waves 160 and / or bubbles 150. The waves 160 may be acoustic waves and / or physical waves generated by water 170. It should be noted that while water is the fluid primarily discussed herein, aspects of the present disclosure relate to any fluid in which the machine 120 may operate and through which bubbles 150 and / or waves 160 will propagate.

[0016] The controller 110 may use the neural network 112 to compare the data from the sensor to the historical database 130. The controller 110 may include a memory device (e.g., Figure 3 1 and 12. The memory stores instructions that cause the controller 110 to perform the operations discussed herein. Although the controller 110 is depicted as being structurally distinct from the sensor 140 and the historical database 130, in some embodiments, the controller 110 may share some computing components with the sensor 140 and / or the historical database 130.

[0017] The controller 110 may interact with the sensors 140 and / or the historical database 130 using the network 180. The network 180 may include a computing network through which computing messages may be sent and / or received. For example, the network 140 may include the Internet, a local area network (LAN), a wide area network (WAN), a wireless network such as a wireless LAN (WLAN), etc. The network 180 may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device (e.g., a computing device hosting / including the sensor 140 and / or the historical database 130) may receive messages and / or instructions from and / or through the network 180, and forward the messages and / or instructions for storage or execution, etc. to the corresponding memory or processor of the corresponding computing / processing device. Although for illustrative purposes, the network 180 is not shown in FIG. Figure 1 1 is depicted as a single entity, but in other examples, network 180 may include multiple private and / or public networks.

[0018] Sensor 140 may include a camera that records images of the water surface. When machine 120 is in the water and is operating, machine 120 may generate sound and / or vibration. If machine 120 encounters some problems, machine 120 may generate vibrations due to unbalanced forces. Such vibrations may propagate through the water. Similarly, if the machine generates sound, the sound will also propagate through the water.

[0019] Controller 110 may use any type of microphone, camera, etc. to perform an audio-visual inspection of machine 120 from the surface of the water above machine 120. In some examples, controller 110 may utilize a drone or other unmanned vehicle (e.g., an unmanned aerial vehicle or an unmanned water vehicle) to perform the surface inspection. Sensor 140 may include a microphone array that may be submerged in the water to capture sound propagation in the water.

[0020] The controller 110 may identify an “allowed” vibration range in any machine 120 , so that if the controller 110 detects vibrations outside of that range, the controller 110 may determine if the machine 120 has any problems, such as loose bolts, missing screws, etc.

[0021] In some examples (not depicted), there may be multiple machines 120 in environment 100, and thus each machine 120 may generate vibrations / sounds. Thus, controller 110 may utilize different sensors 140 for each machine 120, and / or controller 110 may use data from some sensors 140 that are capable of monitoring multiple machine systems 120.

[0022] The controller 110 may analyze the dimensions (e.g., size, shape, number, duration) of the bubbles 150. The controller 110 may specifically analyze whether the size of the bubbles 150 changes as they are generated on the water surface. The controller 110 may store data about the machine 120 in a historical database 130. In some examples, the controller 110 may reference the detected bubbles 150 and / or waves 160 (whether physical or acoustic) against data from the historical database 130.

[0023] In some examples, the controller 110 may collect data directly from the machine 120 itself. For example, the controller 110 may collect data from the machine 120 regarding an error code that the machine 120 is generating. The controller 110 may receive the error code from the machine 120 after the controller 110 has received the atypical bubble 150 and / or wave 160 pattern for a period of time, after which the controller 110 may determine that the atypical bubble 150 and / or wave 160 pattern is a predictive indicator of an error code that will occur in the near future.

[0024] The controller 110 may Figure 2 The flowchart 200 depicted in FIG. 1 manages the performance of a subsea machine, and for illustrative purposes, is directed to Figure 1 discuss Figure 2 200, but it should be understood that in other examples, other environments with other components may be used to perform Figure 2 200. In addition, in some examples, the controller 110 may perform the same Figure 2 The flowchart 200 may be different than the method, or the controller 110 may perform a similar method with more or fewer steps in a different order, etc.

[0025] The controller 110 performs an audio-visual inspection at the surface above the machine operating below the surface of the liquid 202. The audio-visual inspection includes feeding surface wave motion into a neural network, feeding bubble formation patterns into a neural network, feeding bubble sizes into a neural network, and feeding underwater acoustic information into a neural network.

[0026] The controller 110 uses the neural network to identify statistical anomalies from the audio-visual inspection that indicate abnormal performance of the machine (204). The neural network 112 can identify anomalies from the audio-visual inspection by learning how various vibrations and sounds of the machine 120 cause the surface waves 160 to move and the bubbles 150 to form patterns and bubble sizes as well as underwater acoustic information to identify changes in the vibration of the machine 120.

[0027] The controller 110 determines a corrective action to be taken to address the performance anomaly of the machine (206). The controller 110 may determine the corrective action by analyzing the correlation between the statistical anomaly and the probability of failure of the machine. Alternatively or additionally, the controller 110 may determine the corrective action by identifying a time frame in which the corrective action must be performed.

[0028] In some examples, the controller 110 autonomously performs the corrective action (208). In other examples, the controller 110 generates a notification to the user detailing the corrective action and the performance anomaly.

[0029] As described above, the controller 110 may include or be part of a computing device that includes a processor configured to execute instructions stored on a memory to perform the techniques described herein. For example, Figure 3 1 is a conceptual block diagram of a computer 101 that may host a controller 110. Although the controller 110 is depicted as a single entity (e.g., within a single housing) for purposes of illustration, in other examples, the controller 110 may include two or more separate physical systems (e.g., within two or more separate housings). The controller 110 may include an interface 210, a processor 220, and a memory 230. The controller 110 may include any number of interface(s) 210, processor(s) 220, and / or memory(s) 230.

[0030] The computing environment 300 includes an example of an environment for executing at least some of the computer codes involved in performing the method of the present invention (e.g., the underwater machine condition analysis technology 399). In addition to the underwater machine condition analysis technology 399, the computing environment 300 includes, for example, a computer 301, a wide area network (WAN) 302, an end user device (EUD) 303, a remote server 304, a public cloud 305, and a private cloud 306. In this embodiment, the computer 301 includes a processor group 310 (including a processing circuit 320 and a cache 321), a communication structure 311, a volatile memory 312, a persistent storage device 313 (including an operating system 322 and the underwater machine condition analysis technology 399, as described above), a peripheral device group 314 (including a user interface (UI) device group 323, a storage device 324, and an Internet of Things (IoT) sensor group 325), and a network module 315. The remote server 104 includes a remote database 330. The public cloud 305 includes a gateway 340 , a cloud orchestration module 341 , a host physical machine group 342 , a virtual machine group 343 , and a container group 344 .

[0031] Computer 301 may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or developed in the future that is capable of running programs, accessing a network, or querying a database such as remote database 330. As is well known in the art of computer technology, and depending on the technology, the execution of the computer-implemented method may be distributed among multiple computers and / or among multiple locations. On the other hand, in this presentation of computing environment 300, the detailed discussion focuses on a single computer, particularly computer 301, to keep the presentation as simple as possible. Computer 301 may be located in the cloud, even though it is not physically present. Figure 3 On the other hand, computer 301 need not be in the cloud unless to any extent that can be positively indicated.

[0032] Processor group 310 includes one or more computer processors of any type known now or developed in the future. Processing circuit 320 can be distributed over multiple packages, such as multiple coordinated integrated circuit chips. Processing circuit 320 can implement multiple processor threads and / or multiple processor cores. Cache 321 is a memory located in the processor chip package, and is generally used for data or code that should be available for fast access by threads or cores running on processor group 310. Cache memory is generally organized into multiple levels according to relative proximity to the processing circuit. Alternatively, some or all of the caches of the processor group may be located "off-chip". In some computing environments, processor group 310 may be designed to work with qubits and perform quantum computing.

[0033] Computer readable program instructions are typically loaded onto the computer 301 to cause the processor group 310 of the computer 301 to execute a series of operating steps to implement a computer-implemented method, such that the instructions so executed will instantiate the method specified in the flowchart and / or narrative description of the computer-implemented method contained in this document (collectively referred to as the "method of the present invention"). These computer readable program instructions are stored in various types of computer readable storage media, such as cache 321 and other storage media discussed below. The program instructions and related data are accessed by the processor group 310 to control and direct the execution of the method of the present invention. In the computing environment 300, at least some of the instructions for executing the method of the present invention may be stored in the underwater machine condition analysis technology 399 in the persistent storage device 313.

[0034] Communications fabric 311 is the signaling pathways that allow the various components of computer 301 to communicate with each other. Typically, the fabric includes switches and conductive pathways, such as those that make up a bus, a bridge, physical input / output ports, etc. Other types of signal communication pathways may be used, such as fiber optic communication pathways and / or wireless communication pathways.

[0035] Volatile memory 312 is any type of volatile memory now known or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 312 is characterized by random access, but this is not required unless expressly stated. In computer 301, volatile memory 312 is located in a single package and is internal to computer 301, but alternatively or additionally, volatile memory can be distributed in multiple packages and / or located external to computer 301.

[0036] The persistent storage device 313 is any form of non-volatile storage device for a computer known now or developed in the future. The non-volatility of the storage device means that the stored data is maintained regardless of whether power is supplied to the computer 301 and / or directly to the persistent storage device 313. The persistent storage device 313 can be a read-only memory (ROM), but typically at least a portion of the persistent storage device allows the writing of data, the deletion of data, and the rewriting of data. Some common forms of persistent storage devices include magnetic disks and solid-state storage devices. The operating system 322 can take several forms, such as various known proprietary operating systems or open source portable operating system interface type operating systems using a kernel. The code included in the underwater machine condition analysis technology 399 typically includes at least some of the computer code involved in executing the method of the present invention.

[0037] The peripheral device group 314 includes the peripheral device group of the computer 301. The data communication connection between the peripheral device and other components of the computer 301 can be implemented in various ways, such as a Bluetooth connection, a near field communication (NFC) connection, a connection by a cable (such as a universal serial bus (USB) type cable), a plug-in type connection (e.g., a secure digital (SD) card), a connection through a local area communication network, and even a connection through a wide area network such as the Internet. In various embodiments, the UI device group 323 may include components such as a display screen, a speaker, a microphone, a wearable device (such as goggles and a smart watch), a keyboard, a mouse, a printer, a touchpad, a game controller, and a tactile device. The storage device 324 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. The storage device 324 can be persistent and / or volatile. In some embodiments, the storage device 324 can take the form of a quantum computing storage device for storing data in the form of quantum bits. In embodiments where the computer 301 needs to have a large amount of storage (e.g., where the computer 301 locally stores and manages a large database), the storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor group 325 includes sensors that may be used in IoT applications. For example, one sensor may be a thermometer, while another sensor may be a motion detector.

[0038] The network module 315 is a collection of computer software, hardware, and firmware that allows the computer 301 to communicate with other computers via the WAN 302. The network module 315 may include hardware (e.g., a modem or Wi-Fi signal transceiver), software for packetizing and / or depacketizing data transmitted over a communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control function and network forwarding function of the network module 315 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software defined networks (SDN)), the control function and forwarding function of the network module 315 are executed on physically separated devices so that the control function manages several different network hardware devices. Computer-readable program instructions for executing the method of the present invention can typically be downloaded to the computer 301 from an external computer or an external storage device via a network adapter card or a network interface included in the network module 315.

[0039] WAN 302 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances by any technology now known or developed in the future for transmitting computer data. In some embodiments, WAN 302 may be replaced and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. A WAN and / or LAN typically includes computer hardware, such as copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0040] End-user device (EUD) 303 is any computer system used and controlled by an end-user (e.g., a customer of an enterprise operating computer 301), and may take any of the forms discussed above in connection with computer 301. EUD 303 typically receives helpful and useful data from the operation of computer 301. For example, in the hypothetical case where computer 301 is designed to provide recommendations to an end-user, the recommendations would typically be transmitted from network module 315 of computer 301 to EUD 303 via WAN 302. In this manner, EUD 303 may display or otherwise present the recommendations to the end-user. In some embodiments, EUD 303 may be a client device, such as a thin client, a heavy client, a mainframe computer, a desktop computer, or the like.

[0041] Remote server 304 is any computer system that provides at least some data and / or functionality to computer 301. Remote server 304 may be controlled and used by the same entity that operates computer 301. Remote server 304 represents a machine that collects and stores helpful and useful data for use by other computers, such as computer 301. For example, in the hypothetical case where computer 301 is designed and programmed to provide recommendations based on historical data, then the historical data may be provided to computer 301 from remote database 330 of remote server 304.

[0042] The public cloud 305 is any computer system available to multiple entities that provides on-demand availability of computer system resources and / or other computer capabilities, particularly data storage (cloud storage) and computing capabilities, without requiring direct active management by users. Cloud computing typically utilizes resource sharing to achieve consistency and economies of scale. Direct active management of the public cloud 305 computing resources is performed by computer hardware and / or software of the cloud orchestration module 341. The computing resources provided by the public cloud 305 are typically implemented by virtual computing environments running on various computers that constitute the host physical machine group 342, which are various physical computers in and / or available for the public cloud 305. Virtual computing environments (VCEs) typically take the form of virtual machines from a virtual machine group 343 and / or containers from a container group 344. It will be appreciated that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCE is instantiated. The cloud orchestration module 341 manages the transmission and storage of images, deploys new instances of VCEs, and manages active instances of VCE deployments. Gateway 340 is a collection of computer software, hardware, and firmware that allows public cloud 305 to communicate over WAN 302 .

[0043] Some further explanation of virtualized computing environments (VCEs) will now be provided. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two familiar types of VCEs are virtual machines and containers. Containers are VCEs that use operating system level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user space instances (called containers) to exist. From the perspective of the programs running in them, these isolated user space instances typically behave like real computers. Computer programs running on a normal operating system can utilize all of the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of that container and the devices assigned to that container, a feature known as containerization.

[0044] Private cloud 306 is similar to public cloud 305, except that the computing resources are only available to a single enterprise. Although private cloud 306 is depicted as communicating with WAN 302, in other embodiments, the private cloud can be completely disconnected from the Internet and can only be accessed through a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), typically implemented separately by different vendors. Each of the multiple clouds remains an independent discrete entity, but the larger hybrid cloud architecture is tied together through standardized or proprietary technologies that support orchestration, management, and / or data / application portability between multiple constituent clouds. In this embodiment, both public cloud 305 and private cloud 306 are part of a larger hybrid cloud.

[0045] In addition to the underwater machine condition analysis technology 399, in some examples, the collected or predetermined data or technology, etc. are used by the processor group 310 to manage the underwater machine performance. For example, the persistent storage device 313 may include the above information collected from the environment 100. Specifically, the memory 313 may include some or all of the data collected from the sensor 140, and / or the persistent storage device may include some or all of the data of the historical database 130.

[0046] In addition, the persistent storage 313 may include threshold and preference data. The threshold and preference data may include thresholds that define the manner in which the controller 110 manages the analysis of the underwater machine. For example, the threshold and preference data may include thresholds, such as user-provided thresholds, for the controller 110 to perform various tasks as described above. For another example, the threshold and performance data may include thresholds for the controller 110 to recommend corrective actions. For example, the controller 110 may be configured to autonomously perform corrective actions when a potential predicted anomaly is expected to have a severity that exceeds a threshold severity.

[0047] The persistent storage 313 may also include machine learning techniques that the controller 110 may use to improve the process of analyzing and managing subsea machine performance over time as described herein. The machine learning techniques may include algorithms or models generated by performing supervised, unsupervised, or semi-supervised training on a data set and then applying the generated algorithms or models to manage subsea machine performance. Using these machine learning techniques, the controller 110 may be able to improve the ability to detect deviations in subsea machine performance and then responsively manage subsea machine performance.

[0048] Machine learning techniques may include, but are not limited to, decision tree learning, association rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, similarity / metric training, sparse dictionary learning, genetic algorithms, rule-based learning, and / or other machine learning techniques. Specifically, the machine learning technique can utilize one or more of the following example techniques: K-nearest neighbor (KNN), learning vector quantization (LVQ), self-organizing map (SOM), logistic regression, ordinary least squares regression (OLSR), linear regression, stepwise regression, multivariate adaptive regression splines (MARS), ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS), probability classifier, naive Bayes classifier, binary classifier, linear classifier, hierarchical classifier, canonical correlation analysis (CCA), factor analysis, independent component analysis (ICA), linear discriminant analysis (LDA), multidimensional scaling (MDS), non-negative factor decomposition (NMF), partial least squares regression (PLSR), principal component analysis (PCA), principal component regression (PCR), Sammon map, t-distributed stochastic neighbor embedding (t-SNE), bootstrap aggregation, Ensemble average, Gradient boosted decision tree (GBRT), Gradient boosting machine (GBM), Inductive bias algorithm, Q-learning, State-Action-Reward-State-Action (SARSA), Temporal difference (LAT) learning, A priori algorithm, Equivalence Class Transformation algorithm (ECLAT) algorithm, Gaussian process regression, Gene Expression Programming, Grouping method for data manipulation (GMDH), Inductive logic programming, Instance-based learning, Logical model tree, Information fuzzy network (IFN), Hidden Markov model, Gaussian naive Bayes, Multinomial naive Bayes, Averaged One Dependence Estimation (AODE), Classification and Regression Tree (CART), Chi-squared Automatic Interaction Detection (CHAID), Expectation-maximization algorithm, Feedforward neural network, Logical learning machine, Self-organizing map, Single linkage clustering, Fuzzy clustering, Hierarchical clustering, Boltzmann machine, Convolutional neural network, Recurrent neural network, Hierarchical temporal memory (HTM), and / or other machine learning algorithms.

[0049] The description of various embodiments of the present disclosure has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are selected to explain the principles of the embodiments, practical applications, or technical improvements over existing technologies on the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

[0050] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any collection of one or more storage media (also referred to as "media") collectively included in a collection of one or more storage devices, which collectively include machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can retain and store instructions for use by a computer processor. A computer-readable storage medium may be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device (such as punched cards or pits / islands formed in the main surface of the disk), or any suitable combination of the above. As the term is used in the present invention, computer readable storage medium is not to be interpreted as storage in the form of transient signals itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, light pulses through optical cables, electrical signals transmitted through wires and / or other transmission media. As understood by those skilled in the art, data is usually moved at certain occasional points in time during normal operation of the storage device, such as during access, defragmentation or garbage collection, but this does not make the storage device temporary because the data is not temporary when it is stored.

[0051] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0052] The computer-readable program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and procedural programming languages ​​(e.g., "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0053] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium in which the instructions are stored includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0054] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0055] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, the two frames shown in succession can actually be implemented as a step, and are performed simultaneously, substantially simultaneously, in a partially or entirely time-overlapping manner, or these frames can sometimes be performed in reverse order, depending on the functions involved.It will also be noted that each frame of the block diagram and / or flow chart illustration and the combination of frames in the block diagram and / or flow chart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0056] Various aspects of the present disclosure are described by narrative text, flow charts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. With respect to any flow chart, depending on the technology involved, the operations may be performed in an order different from the order shown in a given flow chart. For example, again depending on the technology involved, two operations shown in consecutive flow chart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

Claims

1. A computer-implemented method comprising: Performing an audio-visual inspection at a surface above a machine operating below the surface of a liquid, wherein the audio-visual inspection comprises: Feeding surface wave motion into a neural network; feeding the bubble formation pattern into the neural network; feeding bubble sizes into the neural network; and feeding acoustic information to the neural network; and Statistical anomalies indicative of abnormal performance of the machine are identified from the audio-visual inspection using the neural network.

2. The computer-implemented method of claim 1 , further comprising: A corrective action is determined to address the performance anomaly of the machine.

3. The computer-implemented method of claim 2, further comprising: The corrective action is performed autonomously.

4. The computer-implemented method of claim 2, further comprising: A notification is generated to a user detailing the corrective action and the performance anomaly.

5. The computer-implemented method of claim 2, wherein: Determining the corrective action includes analyzing a correlation between the statistical anomaly and a probability of failure of the machine.

6. The computer-implemented method of claim 2, wherein: Determining the corrective action includes identifying a time frame within which the corrective action must be performed.

7. The computer-implemented method of claim 1 , wherein: The neural network identifies the anomalies from the audio-visual inspection including: the neural network learns how various vibrations and sounds of the machine cause surface wave motion and bubble formation patterns and bubble sizes as well as underwater acoustic information to identify changes in the machine's vibrations away from a baseline.

8. A system comprising: processor; as well as a memory in communication with the processor, the memory containing instructions that, when executed by the processor, cause the processor to: Performing an audio-visual inspection at a surface above a machine operating below the surface of a liquid, wherein the audio-visual inspection comprises: Feeding surface wave motion into a neural network; feeding the bubble formation pattern into the neural network; feeding bubble sizes into the neural network; and feeding acoustic information to the neural network; and Statistical anomalies indicative of abnormal performance of the machine are identified from the audio-visual inspection using the neural network.

9. The system of claim 8, the memory containing additional instructions that, when executed by the processor, cause the processor to: determine a corrective action to address the performance anomaly of the machine.

10. The system of claim 9, the memory containing additional instructions that, when executed by the processor, cause the processor to: autonomously perform the corrective action.

11. The system of claim 9, the memory containing additional instructions that, when executed by the processor, cause the processor to: generate a notification to a user detailing the corrective action and the performance anomaly.

12. The system according to claim 9, wherein: Determining the corrective action includes analyzing a correlation between the statistical anomaly and a probability of failure of the machine.

13. The system according to claim 9, wherein: Determining the corrective action includes identifying a time frame within which the corrective action must be performed.

14. The system according to claim 8, wherein: The neural network identifies the anomalies from the audio-visual inspection including: the neural network learns how various vibrations and sounds of the machine cause surface wave motion and bubble formation patterns and bubble sizes as well as underwater acoustic information to identify changes in the machine's vibrations away from a baseline.

15. A computer program product, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions being executable by a computer to cause the computer to: An audio-visual inspection is performed at a surface above a machine operating below the surface of a liquid, wherein: The audio-visual examination includes: Feeding surface wave motion into a neural network; feeding the bubble formation pattern into the neural network; feeding bubble sizes into the neural network; and feeding underwater acoustic information to the neural network; and Statistical anomalies indicative of abnormal performance of the machine are identified from the audio-visual inspection using the neural network.

16. The computer program product of claim 15, the computer-readable storage medium containing additional program instructions that, when executed by the computer, cause the computer to: determine a corrective action to be taken to resolve the performance anomaly of the machine.

17. The computer program product of claim 16, the computer-readable storage medium containing additional program instructions that, when executed by the computer, cause the computer to: autonomously perform the corrective action.

18. The computer program product of claim 16, the computer-readable storage medium containing additional program instructions that, when executed by the computer, cause the computer to: generate a notification to a user detailing the corrective action and the performance anomaly.

19. The computer program product of claim 16, wherein: Determining the corrective action includes analyzing a correlation between the statistical anomaly and a probability of failure of the machine.

20. The computer program product of claim 16, wherein: Determining the corrective action includes identifying a time frame within which the corrective action must be performed.

Images